This wiki is obsolete, see the NorduGrid web pages for up to date information.

Testing/ARC-CE emi1 rc1

From NorduGrid
Jump to navigationJump to search

Test report

EMI_Test_Report_ARC-CE_EMI1_RC1.pdf

Component Description

ARC-CE components are responsible for accepting requests containing a description of generic computational jobs and executing them in the underlying computing resource.

Code analysis

  • Tester: Marek
  • due: for every RC
  • Sloccount:
cpp:         136167 (74.26%)
perl:         18590 (10.14%)
php:          13007 (7.09%)
python:       10545 (5.75%)
sh:            4199 (2.29%)
ansic:          798 (0.44%)
java:            63 (0.03%)

Total Physical Source Lines of Code (SLOC) = 183,369

The complete results of SLOC count entire ARC code can be found here

______________________________________________________________________

  • CCCC metrics:
Metric Tag Overall Per Module
Number of modules NOM 890
Lines of Code LOC 106028 119.133
McCabe's Cyclomatic Number MVG 30720 34.517
Lines of Comment COM 12674 14.240
LOC/COM L_C 8.366
MVG/COM M_C 2.424
Information Flow measure ( inclusive ) IF4 618184 694.589
Information Flow measure ( visible ) IF4v 618047 694.435
Information Flow measure ( concrete ) IF4c 3341 3.754
Lines of Code rejected by parser REJ 3197

The complete results for CCCC for entire ARC code can be found here

Later the results will be split between components.

Unit tests

  • Tester: Anders
  • due: for every RC

Results of unit test code coverage for entire ARC code can be found here:

http://hep.nbi.dk/~waananen/nordugrid/20781/

The text description of results will come later.

Later the results will be split between components.

Regression tests

  • Tester: Marek
  • due: for every tester

initially the verification of critical and blocker resolved bugs

R1

R2

Deployment tests

D1: clean installation

  • Tester: Marek
  • due: for every RC

D2: upgrade installation

  • Tester: Marek
  • due postponed

Functionality tests

F1: job management with invalid input (WS interface)

  • Tester NIIF
  • due: May 2011

For all functions/operations of the ARC-CE interface check the invalid input. Invalid output should throw an exception as documented in XXX?

F2: job management with invalid credentials

  • Tester: Samir
  • due: May 2011

For all functions/operations of the ARC-CE interface check the invalid, non-authorized credentials. Use ordinary and VOMS proxies. Invalid/non-authorized credentials should throw security related exceptions and return reasonable error messages. Both pre-WS and WS interface.

F3: simple job execution

  • Tester: Kosice
  • due: 25 March

Test the job submission for simple job. Test/use all supported (on the server side) job description languages. production interface.

F4: data stage-in job

  • Tester: Kosice
  • due: 1 April.

Test the job submission for simple job with input files being staged-in by both the client and the CE itself. Test cache functionality.

  • submission of job uploading one file
  • submission of job uploading many files
  • submission of job staging in one file from gsiftp SE
    • PASSED for JSDL and XRSL jobs, FAILED for JDL (Bug 2254)
  • submission of job staging in many files from gsiftp SE
  • submission of job staging in one file from http SE
  • submission of job staging in many files from http SE
  • submission of job staging in one file from ftp SE
  • submission of job staging in many files from ftp SE
  • submission of job staging in one file from srm SE
  • submission of job staging in many files from srm SE
  • submission of job staging in one file from lfc SE
  • submission of job staging in many files from lfc SE

cache functionality

  • caching of staged in/uploaded files
    • caching of staged in file from Unixacl PASSED

F5: data stage-out job

  • Tester: Kosice
  • due 1 April

Test the job submission for simple job with output files being staged-out both the client (download) and the CE itself (server uploads to SE). Test all kind of protocols and index service registrations.

  • job results retrieval (retrieve of job with one output file)
  • job results retrieval (retrieve of job with many output files)
  • submission of job staging out one file to gsiftp SE
    • PASSED for JSDL and XRSL jobs, FAILED for JDL (Bug 2254)
  • submission of job staging out many files to gsiftp SE
  • submission of job staging out one file to http SE
  • submission of job staging out many files to http SE
  • submission of job staging out one file to ftp SE
  • submission of job staging out many files to ftp SE
  • submission of job staging out one file to srm SE
  • submission of job staging out many files to srm SE
  • submission of job staging out one file to lfc SE
  • submission of job staging out many files to lfc SE


F6: job management via pre-WS

  • Tester: Kosice
  • due: 25 March

Check the main operations of the ARC-CE production interface: cancel, kill, so on... Use ordinary and VOMS proxies.

  • simple job submission
  • simple job migration
  • migration of job with input files
  • job status retrieval
  • job catenate retrieval
  • killing job
  • job cleaning
  • job results retrieval (retrieve of job with one output file)

F7: parallel job support

  • Tester: NIIF
  • due: 1 April

Check that more than one slots are request-able and allocated to a job when the corresponding job description element is used.

F9: job management through WS-interface

  • Tester: Kosice
  • due: May 2011
  • simple job submission
    • Submission of simple job described in JDL
arcsub -c ARC1:https://pgs03.grid.upjs.sk:50000/arex jdl_hostname.jdl

jdl_hostname.jdl
[
Executable = "/bin/hostname";
StdOutput = "std.out";
StdError = "std.err";
OutputSandbox = {"std.out","std.err"};
OutputSandboxDestURI = { "gsiftp://localhost/std.out", "gsiftp://localhost/std.err" };
]

FAILED - Bug 2254

    • Submission of simple job described in XRSL
arcsub -c ARC1:https://pgs03.grid.upjs.sk:50000/arex xrsl_hostname.xrsl
xrsl_hostname.xrsl
&(executable = "/bin/hostname")
(stdout = "stdout.txt")
(jobName= "hostname-test")

PASSED

    • Submission of simple job described in JSDL
arcsub -c ARC1:https://pgs03.grid.upjs.sk:50000/arex jsdl_hostname.xml
jsdl_hostname.xml
<?xml version="1.0" encoding="UTF-8"?>
<JobDefinition
 xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl"
 xmlns:posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix"
 xmlns:arc="http://www.nordugrid.org/ws/schemas/jsdl-arc">
 <JobDescription>
   <JobIdentification>
     <JobName>JSDL-TEST</JobName>
   </JobIdentification>
   <Application>
     <posix:POSIXApplication>
       <posix:Executable>/bin/hostname</posix:Executable>
       <posix:Output>out.txt</posix:Output>
       <posix:Error>err.txt</posix:Error>
     </posix:POSIXApplication>
   </Application>
   <DataStaging>
     <FileName>out.txt</FileName>
     <DeleteOnTermination>false</DeleteOnTermination>
     <DownloadToCache>false</DownloadToCache>
   </DataStaging>
   <DataStaging>
     <FileName>err.txt</FileName>
   </DataStaging>
 </JobDescription>
</JobDefinition>

PASSED

  • submission of job uploading one file
    • Submission of simple job described in XRSL
arcsub -c ARC1:https://pgs03.grid.upjs.sk:50000/arex xrsl_shell.xrsl
xrsl_shell.xrsl
&(executable = "shell.sh")
(inputFiles = ("shell.sh" ""))
(outputFiles =
("stdout.txt" "")
("stderr.txt" "")
)
(stdout = "stdout.txt")
(stderr = "stderr.txt")
(jobName= "shell-test")

PASSED

  • submission of job uploading many files
  • simple job migration - FAILED (Bug 2255)
  • migration of job with input files - FAILED (Bug 2255)
  • job status retrieval
    • PASSED for JDL, JSDL, XRSL
  • job catenate retrieval - FAILED (Bug 2257)
  • killing job
    • PASSED for JDL, JSDL, XRSL
  • job cleaning
    • PASSED for JDL, JSDL, XRSL
  • job results retrieval (retrieve of job with one output file)
    • PASSED for JDL, JSDL, XRSL
  • job results retrieval (retrieve of job with many output files)


F10: LRMS support

  • Tester: Adrian
  • due date: 1 April

Test pbs/maui, SGE, Condor, SLURM, fork, .....

  • correct job status identification
  • correct identification of running/pending/queueing jobs
  • correct CE information propagation (part of Glue2 tests)

F11: Janitor tests

  • Tester: Adrian
  • due May 2011
  • Static
  • Dynamic using Janitor component
    • ???? is there a plan to have janitor?
  • classic RTE tests
  • submission of jobs requiring different types of RTEs

F12: gridmapfile

  • Tester: Samir (Uppsala)
  • due 1 April
  • retrieval of proper DN lists
  • example authorization scenarios (vo groups)

F13: infopublishing: nordugrid schema

  • Tester: Florido
  • 1 April

Check that the CE properly publishes cluster, queue, user and job information according to nordugrid schema.


F14: infopublishing: glue1.2 schema

  • Tester: Florido
  • due: 1 April

Check that the CE properly publishes resource info according to glue-1.2 schema.

F15: infopublishing: glue2 LDAP schema

  • Tester: Florido
  • due: 1st April

Check that the CE properly publishes resource info according to the glue2 LDAP schema. Use EMI validator

F15: infopublishing: glue2 xml schema

  • Tester: Florido
  • due: 1st April

Check that the CE properly publishes resource info according to the glue2 xml schema. Use some validator

Performance tests

P1: service reliability

  • Kosice
  • due: 25 March

Services run by the component must maintain a good performance and reliability over long periods of time with normal operation. Long running unattended operation test measuring performance of the product. Service must not show performance degradation during a 3-day period.

example provided by: NSC medium-size resource: ~100 nodes running ~1000 jobs each 5 hours, which would give around < 5000 jobs in infosys that would give peak submission at 200 jobs per hour large-size resource: ~10k nodes running ~100k jobs each 5 hours, which would give around < 500k jobs in infosys that would give peak submission at 20 000 jobs per hour

P2: load test

  • Kosice
  • due 25 March

stress test the CE with

  • massive amount of synchronous job submission
  • massive amount of synchronous info query

P3: job submission failure rate

  • Jozef, Kosice
  • due 25 March

run a properly configured CE and exercise that with e.g. series of 100 job submissions. check the failure rate

Scalability tests

WILL NOT BE DONE FOR RC1

Standard compliance/conformance tests

see the XYZ functionality test for glue2

Inter-component tests

  • the staging in/out to/from third-party SE will be tested on these components: TBD: IDENTIFY component
  • VOMS proxies bla bla ....
Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=Testing/ARC-CE_emi1_rc1&oldid=3692"