This wiki is obsolete, see the NorduGrid web pages for up to date information.

Testing/ARC-Clients

From NorduGrid
Jump to navigationJump to search

Coordination effort for each RC will be performed at the links shown in the Testing page.

Test report

To be attached here.

EMI Component Description and Version

The ARC-Clients components consist of a set of commands necessary for job management, proxy manipulation and data handling. Also, the underlying libraries are included. The compute cli support different job description languages (XRSL, JSDL, JDL), different CE flavours (Classic ARC, A-REX, gLite Cream CE). The data cli handles different storage elements. The arcproxy serves as a general purpose proxy manipulation utility.

Code analysis

  • Tester: Marek
  • due: for every RC
  • Sloccount:
  • CCCC metrics:

Later the results will be split between components.

Unit tests

  • Tester: Anders
  • due: for every RC

Link to results of unit test code coverage for entire ARC goes here.

The text description of results will come later.

Later the results will be split between components.

Deployment tests

CL-D1: clean installation

  • due: for every RC

CL-D2: upgrade installation

  • due: for every RC

System Tests

Regression tests

See RfCs in savannah tracker

Functionality tests

CL-F1: direct job submission

  • Tester: Marek, Jozef, Eva

job submission to a pre-selected resource

  • ngsub
  • arcsub
  • test pre-WS, WS, CREAM, Unicore


  • CL-F1.1: Job submission to pre-WS ARC CE using arcsub and XRSL job description language
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using arcsub command (manual test). XRSL job description can be found at http://vls.grid.upjs.sk/testing/job_descriptions/get_hostname_xrsl.html.
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully submitted
    • Result:


  • CL-F1.2: Job submission to pre-WS ARC CE using arcsub and JSDL job description language
    • Description of the test:
Create simple job in JSDL format and submit it to pre-WS ARC CE using arcsub command (manual test). JSDL job description can be found at http://vls.grid.upjs.sk/testing/job_descriptions/get_hostname.html.
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully submitted
    • Result:


  • CL-F1.3: Job submission to pre-WS ARC CE using arcsub and JDL job description language
    • Description of the test:
Create simple job in JDL format and submit it to pre-WS ARC CE using arcsub command. (manual test). JDL job can be found at http://vls.grid.upjs.sk/testing/job_descriptions/get_hostname_jdl.html
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully submitted
    • Result:


  • CL-F1.4: Job submission to pre-WS ARC CE using ngsub and XRSL job description language
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using ngsub command (manual test).
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully submitted
    • Result:


  • CL-F1.5: Job submission to pre-WS ARC CE using ngsub and JSDL job description language
    • Description of the test:
Create simple job in JSDL format and submit it to pre-WS ARC CE using ngsub command (manual test).
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully submitted
    • Result:

CL-F2: Job migration

  • arcmigrate

CL-F3: job and resource info querying

  • ngstat
  • arcstat
  • arcinfo


  • CL-F3.1: Query the job status submitted to pre-WS ARC CE using arcstat command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using arcsub command. Wait 120 seconds. Query the job status using arcstat command.
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully queried
    • Result:
  • CL-F3.2: Query the job status submitted to pre-WS ARC CE using ngstat command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using ngsub command. Wait 120 seconds. Query the job status using ngstat command.
    • Testbed:
Describe testbed here
    • Expected result:
The job is successfully queried
    • Result:

CL-F4: retrieval of job results/output

  • Tester: Marek, Jozef
  • due: 25 March
  • ngget
  • arcget


  • CL-F4.1: Retrieve the results of successfully finished job submitted to pre-WS ARC CE using arcget command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using arcsub command. Wait until the job is successfully finished (verify it using arcstat command). Retrieve the results of finished job using arcget command.
    • Testbed:
Describe testbed here
    • Expected result:
The job results are successfully retrieved
    • Result:
  • CL-F4.2: Retrieve the results of successfully finished job submitted to pre-WS ARC CE using ngget command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using ngsub command. Wait until the job is successfully finished (verify it using ngstat command). Retrieve the results of finished job using ngget command.
    • Testbed:
Describe testbed here
    • Expected result:
The job results are successfully retrieved
    • Result:

CL-F5: access to stdout/stderr or server logs

  • ngcat
  • arccat

CL-F6: removal of jobs from CE

  • ngclean
  • arcclean
  • CL-F6.1: Removal of finished job from pre-WS ARC CE using arcclean command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using arcsub command. Wait until the job is successfully finished (verify it using arcstat command). Issue an arcclean command with jobID as parameter. Query the job status using arcstat command
    • Testbed:
Describe testbed here
    • Expected result:
The second job status query shall return a warning that the job is no longer available
    • Result:


  • CL-F6.2: Removal of finished job from pre-WS ARC CE using ngclean command
    • Description of the test:
Create simple job in XRSL format and submit it to pre-WS ARC CE using ngsub command. Wait until the job is successfully finished (verify it using ngstat command). Issue an ngclean command with jobID as parameter. Query the job status using ngstat command
    • Testbed:
Describe testbed here
** Expected result:
The second job status query shall return a warning that the job is no longer available
    • Result:

CL-F7: termination of active jobs

  • ngkill
  • arckill

CL-F8: proxy renewal for an active job

  • ngrenew
  • arcrenew

CL-F9: job failure recovery

  • ngresume
  • arcresume

CL-F10: job list population via job discovery

  • ngsync
  • arcsync

CL-F11: test utility check

The most detailed documentation about ngtest (or arctest) is the man page itself. Even the outdated manuals or the usage (--help) output of the commands contain just a subset of the description of features. Since the role of a functional test is to prove that the software is able to provide the documented features and do it under any circumstances, the base of this test will be the man page and the documented capabilities.

To localize the possible weaknesses it worth to distinguish the features and test them separated from each other. So the following test cases will follow this approach by testing them separately.

Test environment:

The clients were installed to a dedicated server with a brand new SL5 installation used the RC2 packages. When the testbed were used it's explicitly indicated.

ngtest

Check former test results here

arctest

CL-F12: job description support

  • Package: EMIX - (Update Y)

test XRSL

See former test results here

test JSDL

See former test results here

test JDL

See former test results here

CL-F13: brokering

  • test resource discovery, use a JDL that matches "everything"
  • test some of the brokering algorithms

CL-F14: data clis: access to EMI storage elements

  • arcls
  • arccp
  • arcrm
  • ngls
  • ngcp
  • ngrm

CL-F15: proxy manipulation

Check former test results here

Performance tests

CL-P1: load test of compute cli

submitting 1000 jobs in one client instance

CL-P2: reliability test of compute cli

having a client running with unstable environment: dying index services, computing elements stop/start

CL-P3: load test of data cli

Start with a clean SL5.5 x86_64 virtual machine, install from EMI repositories following ARC client installation instructions (http://www.nordugrid.org/documents/arc-client-install.html). 

yum install nordugrid-arc-client
yum install nordugrid-arc-plugins-globus
yum install ca_NorduGrid
  • upload/download 5GB file with arccp
    • using file://
    • using https://
    • using gsiftp://
  • upload/download 1000 files with arccp
    • using file://
    • using https://
    • using srm://
    • using gsiftp://

CL-P3.1: Download/upload 5GB file using file:// protocol

Description of the test:

Download/upload a 5GB file using file:// protocol as source and target

Testbed:

ARC client deployed on resource described.

Expected result:

The file specified as target shall exist and be of size 5GB.

Result: PASSED/FAILED

CL-P3.2: Upload 5GB file using https:// protocol

Description of the test:

Download/upload a 5GB file using https:// protocol as target

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The file specified as target shall exist and be of size 5GB.

Result:

CL-P3.3: Download 5GB file using https:// protocol

Description of the test:

Download/upload a 5GB file using https:// protocol as source

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The file specified as target shall exist and be of size 5GB.

Result:

CL-P3.4: Upload 5GB file using gsiftp:// protocol

Description of the test:

Download/upload a 5GB file using gsiftp:// protocol as target

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The file specified as target shall exist and be of size 5GB.

Result: PASSED/FAILED

CL-P3.5: Download 5GB file using gsiftp:// protocol

Description of the test:

Download/upload a 5GB file using gsiftp:// protocol as source

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The file specified as target shall exist and be of size 5GB.

Result: PASSED/FAILED

CL-P3.6: Download/upload many files using file:// protocol

Description of the test:

Download/upload 1000 1kB files using file:// protocol as source and target

Testbed:

ARC client deployed on resource described.

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result: PASSED/FAILED

CL-P3.7: Upload many files using https:// protocol

Description of the test:

Upload 1000 1kB files using https:// protocol as target

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result:

CL-P3.8: Download many files using https:// protocol

Description of the test:

Download 1000 1kB files using https:// protocol as source

Testbed:

ARC client deployed on resource described. https service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result:

CL-P3.9: Upload many files using srm:// protocol

Description of the test:

Upload 1000 1kB files using srm:// protocol as target

Testbed:

ARC client deployed on resource described. srm service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result: PASSED/FAILED

CL-P1.10: Download many files using srm:// protocol

Description of the test:

Download 1000 1kB files using srm:// protocol as source

Testbed:

ARC client deployed on resource described. srm service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result: PASSED/FAILED

CL-P3.11: Upload many files using gsiftp:// protocol

Description of the test:

Upload 1000 1kB files using gsiftp:// protocol as target

Testbed:

ARC client deployed on resource described. gridftp service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result: PASSED/FAILED

CL-P3.12: Download many files using gsiftp:// protocol

Description of the test:

Download 1000 1kB files using gsiftp:// protocol as source

Testbed:

ARC client deployed on resource described. gridftp service running on external machine

Expected result:

The 1000 files specified as target shall exist and be of size 1kB.

Result: PASSED/FAILED

CL-P4: reliability test of data cli

Not yet planned

Scalability tests

CL-S1: jobs submission ratio

  • test number of jobs submitted sequentially per minute. turn off brokering (direct submission). use some very simple xrsl. no input data staging
  • turn on data staging, turn on brokering, so on...
  • use batch job processing feature (one big xrsl/jsdl containing all the jobs)
  • specifying middleware plugin vs. not-specifying mw plugins


CL-S2: data transfer ratio

  • Tester: Jon
  • due: 1 April

moving lots of small files, or big files

Standard compliance/conformance tests

CL-STD1: JSDL for compute client

  • Note that JSDL compliance is also tested in the ARCJSDLParserTest unit test located in source tree at arc1/trunk/src/hed/acc/JobDescriptionParser/ARCJSDLParserTest.cpp.

CL-S2: data standards

SRM, all the DMCs but RLS...

CL-ICT Inter-component tests

ARC Clients integration tests defined at https://twiki.cern.ch/twiki/bin/view/EMI/EmiJra1TaskForceIntegrationTesting

Integration Test 8

Summary Submit and monitor jobs through EMI-ES

Integration Test 19

Summary ARC data clients to list/put/get files on LFC for each of the EMI SEs using VOMS proxy

Testbed resources: From EMI-1 production testbed: DPM, LFC, StoRM, dCache, ARC Clients

This test consists of uploading a local file to an EMI SE, registering it in an LFC, listing the file, copying it back, and deleting it from LFC and the SE. Several error conditions are also tested.

To run the test: 

Usage: ARC-Clients-IntegrationTest-19 <protocol> <SRM flavour>

The script can test SRM on its own (protocol srm) or with LFC registration (protocol lfc), or protocol can be set to file to test with local files. If protocol is srm or lfc the flavour of SRM must be specified (dpm, dcache or storm), eg 

> ARC-Clients-IntegrationTest-19 lfc dpm

A summary of the number of failures is output at the end of the test. On success the test should leave no trace of local or remote files.

Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=Testing/ARC-Clients&oldid=5793"