This wiki is obsolete, see the NorduGrid web pages for up to date information.
NOX/Tests
This page collects links to the test cases for individual components of Nox release. The features tested were collected by main developers of respective components and test cases were then prepared and carried out by independent testers. So that both documentation and software of given component could be evaluated.
Manual testing
A-REX
- Covered by KnowARC D5.4-1 performance tests: D5.4-1_09 deliverable document
- Test report: A-REX test cases are collected on dedicated page: A-REX test cases
Janitor
Jura
- Test report: JURA tests
ISIS
- Covered by KnowARC D5.4-1 performance tests: D5.4-1_09 deliverable document
- Test report: ISIS tests
Chelonia Storage
- Covered by KnowARC D5.4-1 performance tests: D5.4-1_09 deliverable document
- Additional performance tests: Storage plots
- Test report: test cases page
Small services HOPI, ECHO C++ and ECHO Python)
- Test reports:
Charon & security
- Test plan: https://www.knowarc.eu/wiki/images/a/ac/Egerszalok-arc1-security.pdf
- Test report: NOX/Tests/Security test results
ARCLIB and Client
- Covered by KnowARC D5.4-1 performance D5.4-1 deliverable: D5.4-1_09 deliverable document
- Test reports:
Automated tests
This page is dedicated to test planning and detailed specifications. For now it contains just an outline of main components and functionality of ARC to be tested. The more detailed input will be added continuously. Any input is welcome.
An outline of testing process
The testing can be divided into two parts: unit and system/integration testing. The former concentrate on the lowest level components of the software while the latter probes larger subsystems or system as whole. Schematic outline of the testing procedure is on the figure below.
Unit tests
The low level tests are performed automatically after successful builds. Most significant portion of ARC Nox source code is written in C++ programming language. To provide possibility of performing low level unit testing of that portion of code we make use of cppunit (http://sourceforge.net/apps/mediawiki/cppunit) framework. The unit tests so far implemented focus on the most fundamental functionality within the HED: the coverage common libraries (18\% coverage), information system libraries (60% coverage), libarcclient client library (16% coverage) libarcdatadata handling library (~7% coverage). Overall coverage of HED libraries is 9%.
System/integration tests
I have divided system/integration tests into three subgroups: accordingly three test suits will be developed. The test suits should run automatically after successful installation of ARC.
Black box testing
The first and the most important one is "alien" layer which comprise all functional requirements end user could demand from ARC.
Functional tests:
- Site registration & authorization.
- Monitoring and logging.
- Single/bulk job submission.
- Job management (control, kill, clean, restart).
- Storage elements (SE). Copy/move files to/from/between SE.
All of standard commands: apclean, apecho, apget, apinfo, apkill, approxy, apsstat, apstat, apsub, arccp, arcls, arcrm and arcstat will be tested during this step.
Glass (White) box testing
Finer grained testing of distinct components of ARC. Based on specifications in design document (D1.1).
- Hosting environment.
- Information system.
- Execution management capability.
- Data management.
- Resource management.
- Security.
- Self Management.
Performance testing
Tests from deliverable D5.4
- CPU and MEM usage under different conditions.
- Job submission component (jobs submitted per minute, job ratio ...).
- Information system response time.
- Resource discovery
- Data staging.
- Data copy.
Test specifications
The following text specifies tests that are run upon subsequent revisions of ARC1 trunk repository automatically. The test reusults can be viewed at dedicated vls.grid.upjs.sk/testing webpage
AREX
A-rex is tested by submitting, controling, dowloading, and cleaning with following jobs:
- get hostname. If the job finishes successfully a hostname of ARC1 server shall be present in out.txt file.
- shell http. This job stages in an input file which is then processed.
- local stage in. This job stages in a shell script from a client which is then executed on the cluster.
- http stage in. This job stages in an input file which is then processed.
- http stage out. This job produces a file which is then staged out to a remote http server .
- ftp stage in. This job stages in an input file from a remote ftp server which is then processed.
- ftp stage out . This job produces a file which is then staged out to a remote ftp server .
- gsi stage in. This job stages in a python script from a client and an input file from a remote GSI server. The python script uses the input file for creating output which is then staged out to the client.
- gsi stage out. This job stages in a python script from a client. This script generates an output file which is then staged out to a remote GSI server.
AREX SECURITY
The AREX SECURITY feature is tested by submitting the same bunch of jobs to AREX. AREX is running with several different policies one after another.
ECHO
There are currently three forms of ECHO service provided by ARC1. ECHO is available from C++, Python and Java.
Tests for ECHO C++
- Using apecho client tool. Expected output is <message> enclosed in wrapper specified in configuration file. Usage:
$ arcecho service_url <message>
.
- Using curl gnu utility. Usage:
$ curl -d <message>
, where <message> is XML formatted text. For example:
'<?xml version="1.0"?><soap-env:Envelope xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:star="urn:echo"> <soap-env:Body> <star:echo> <star:say>'''HELLO'''</star:say> </star:echo> </soap-env:Body> </soap-env:Envelope>'
.
The expected output is XML formatted text with HELLO string enclosed in wrapper specified in configuration file.
Tests for ECHO python
- Using curl gnu utility. Usage:
$ curl -d <message>
, where <message> is XML formatted text. For example:
'<?xml version="1.0"?><soap-env:Envelope xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:star="urn:echo"> <soap-env:Body> <star:echo> <star:say>'''HELLO'''</star:say> </star:echo> </soap-env:Body> </soap-env:Envelope>'
.
The expected output is XML formatted text with HELLO string enclosed in wrapper specified in configuration file.
Tests for ECHO java (not available now)
- Using curl gnu utility. Usage:
$ curl -d <message>
, where <message> is XML formatted text. For example:
'<?xml version="1.0"?><soap-env:Envelope xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:star="urn:echo"> <soap-env:Body> <star:echo> <star:say>'''HELLO'''</star:say> </star:echo> </soap-env:Body> </soap-env:Envelope>'
.
The expected output is XML formatted text with HELLO string enclosed in wrapper specified in configuration file.
HOPI (previous name HTTPD)
HOPI service is tested by listing of content of published directory and by wget GNU utility:
Listing
- published HTML page is checked for whether every file or directory within published directory is present and accessible via hyperlink.
wget
- The published directory is recursively downloaded and the MD5SUMs of downloaded files are compared to checksums of local files in published directory.
- Structure of downloaded directory is compared to the structure of published directory.
STORAGE
The STORAGE service is tested through its client arc_storage_cli. All its methods (make, stat, list, put, get, del, move, unmake) are being tested.
MAKE
- root
- some collection in root
- recursive collections up to 100
- some collection within recursive tree
- without target
STAT
- empty root
- file
- non empty collection
- empty collection
- bunch of files within collection tree
- non existing path
- root
- without target
LIST
- root
- file
- non empty collection
- empty collection
- non existing path
- without target
PUT
- to root
- to existing collection
- to non existing collection
- to existing file
- bunch of files somwhere in the collection tree
- without target
GET
- bunch of files from the collection tree
- to non existing local path
- to new name
- collection
- without source
DEL
- non existing path
- collection
- file
- bunch of files from the collection tree
- without target
MOVE
- to new name
- non existing file
- to existing file
- to existing collection
- to non existing path
- without target
UNMAKE
- collection in root
- collection within recursive tree
- recursive collection up to 100
- file
- non empty collection
- non existing path
- root
- without target
CHARON (previous name PDP - Policy Decision Point)
For now it only makes sure that service runs with example configuration file (after trivial modifications) in the trunk, and that it returns correct answer based on predefined policy and preformated request sent through its client (arcdecision).
Operating systems
Based on discussion during WP5 meeting the following platforms were selected for integration testing:
- Fedora 8
- OpenSuSE 10.3
- Ubuntu 8.04
- CentOS 5.1
Snapshots and relevant .rpm packages for these operating systems, are downloaded, compiled (in the case of installation from sources), installed and tested daily. Results of tests, actual tests scripts, test jobs and scripts, can be found on dedicated site
Bugtracking system
Results of all tests are archived and accessible on the testing site and discovered bugs will be filled into bugzilla and/or the responsible developer will be contacted directly if one can be identified easily.