This wiki is obsolete, see the NorduGrid web pages for up to date information.

Testing/ARC-Clients emi1 rc0

From NorduGrid
Jump to navigationJump to search

Component Description

brief description of what is being tested, version, modules, components

Code analysis

Very preliminary results of SLOC count entire ARC code can be found here:

http://testbed1.grid.upjs.sk/static_code_analysis/nordugrid-arc-1.0.0b1.sloccount

http://testbed1.grid.upjs.sk/static_code_analysis/nordugrid-arc-compat-1.0.0b1.sloccount

Later the results will be split between components.

Unit tests

Preliminary results of unit test code coverage for entire ARC code can be found here:

http://hep.nbi.dk/~waananen/nordugrid/20702/

Later the results will be split between components.

Regression tests

initially the verification of resolved bugs

Deployment tests

clean installation

There are issues with installing ARC from the RC0 repository. The executive summary is: Packages based on the lcg-dm-common sources needs fixing or we will have the same problem for RC1.

ETICS respects EMI packaging policy which is to follow among others the EPEL packaging policy. That policy states that AutoReqProv by default MUST be on. Packages SHOULD not use it since the defaults are fine. The source package: 

lcg-dm-common-1.8.0-1sec.sl5.src.rpm

produces the binary packages: 

lcgdm-devel-1.8.0-1sec.sl5.x86_64.rpm
lcgdm-libs-1.8.0-1sec.sl5.x86_64.rpm

which explicitly have AutoReqProv turned off. Therefore ARC and dependent packages which are packaged correctly will not work. This includes: 

nordugrid-arc-compat-1.0.0-0.b1.el5.src.rpm
nordugrid-arc-1.0.0-0.b1.el5.src.rpm
semsgplugins-1.0.0-1.src.rpm

The packages that apparently have the autoreqprov on and use direct package dependencies and thus will work are: 

lcg-dm-common-1.8.0-1sec.sl5.src.rpm
DPM-mysql-1.8.0-1sec.sl5.src.rpm
DPM-oracle-1.8.0-1sec.sl5.src.rpm
LFC-mysql-1.8.0-1sec.sl5.src.rpm
LFC-oracle-1.8.0-1sec.sl5.src.rpm

These packages are broken according to EMI policies. Known violations are: 

No sonames in libraries           https://savannah.cern.ch/bugs/?57528
Using static libraries            https://savannah.cern.ch/bugs/?57529
Turning off autoreqprov           <not reported in Savannah>
Same sources in several src.rpms  <not reported in Savannah>

The combination of the first 3 issues break ARC and one other package. Note that direct dependencies are allowed and needed when the AutoReqProv can not find dependencies/provides. Packages with plugins are an example.

The ARC packages in RC0 would work if the lcg-dm-common was fixed according to the above. However, since ARC was built against a broken lcg-dm-common there are some erroneous dependencies on lcgdm.so which require a rebuild of ARC to get fixed.

So the bottom line is that there are no known packaging bugs in the ARC packages and that they would work (in principle) without recompilation.

One can still test the ARC packages from RC0 but this requires installing RPMs with broken dependencies.

upgrade installation

Functionality tests

Job submission

  • ngsub
  • arcsub

Job migration

  • arcmigrate

job and cluster querying

  • ngstat
  • arcstat
  • arcinfo

retrieval of job results/output

  • ngget
  • arcget

view of stdout/stderr or server logs

  • ngcat
  • arccat

removal of jobs from target

  • ngclean
  • arcclean

kill running jobs

  • ngkill
  • arckill

proxy reneval

  • ngrenew
  • arcrenew

resumming of failed jobs

  • ngresume
  • arcresume

synchronization of job list

  • ngsync
  • arcsync

testing client and server setup

  • ngtest

acess to EMI storage elements

  • arcls
  • arccp
  • arcrm
  • ngls
  • ngcp
  • ngrm


arcproxy

  • ordinary RFC proxy
  • RFC proxy with VOMS extensions


command: arcproxy correct output: 

Your identity: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak 
Enter pass phrase for /home/cernak/.globus/userkey.pem: 
....++++++ 
.......++++++ 
Proxy generation succeeded 
Your proxy is valid until: 2011-03-03 00:10:52

The result of test: 

Your identity: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak 
Enter pass phrase for /home/cernak/.globus/userkey.pem: 
....++++++ 
.......++++++ 
Proxy generation succeeded 
Your proxy is valid until: 2011-03-03 00:10:52

PASSED

command arcproxy -d (FATAL, ERROR, WARNING, INFO, VERBOSE, DEBUG)

arcproxy -d VERBOSE 

INFO: Loading configuration (/etc/arc/client.conf) 
INFO: Configuration (/etc/arc/client.conf) loaded 
INFO: Loading configuration (/home/cernak/.arc/client.conf) 
INFO: Configuration (/home/cernak/.arc/client.conf) loaded 
Your identity: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak 
Enter pass phrase for /home/cernak/.globus/userkey.pem: 
VERBOSE: Certificate verification succeeded 
.++++++ 
.............++++++ 
INFO: Succeeded to sign the proxy certificate 
INFO: Succeeded to verify the signed certificate 
INFO: Output the proxy certificate 
VERBOSE: Certificate verification succeeded 
Proxy generation succeeded 
Your proxy is valid until: 2011-03-03 00:19:41

PASSED

command arcproxy -d INFO 

INFO: Loading configuration (/etc/arc/client.conf) 
INFO: Configuration (/etc/arc/client.conf) loaded 
INFO: Loading configuration (/home/cernak/.arc/client.conf) 
INFO: Configuration (/home/cernak/.arc/client.conf) loaded 
Your identity: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak 
Enter pass phrase for /home/cernak/.globus/userkey.pem: 
...............++++++ 
..++++++ 
INFO: Succeeded to sign the proxy certificate 
INFO: Succeeded to verify the signed certificate 
INFO: Output the proxy certificate 
Proxy generation succeeded 
Your proxy is valid until: 2011-03-03 00:21:00

PASSED

command: arcproxy -v 

arcproxy version 1.0.0b1

PASSED

command arcproxy -I 

Subject: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak/CN=448705739 
Identity: /C=SK/O=SlovakGrid/O=UPJS/CN=Jozef Cernak 
Time left for proxy: 11 hours 45 minutes 48 seconds 
Proxy path: /tmp/x509up_u500 
Proxy type: X.509 Proxy Certificate Profile RFC compliant restricted proxy

PASSED

Performance tests

Scenario

Initial setup server(production version or RC) client(RC). From client jobs are submitted on the server in loop. The number of submitted jobs is N, the number of really submitted jobs is N1. We investigate ration N1/N for various settings of client and for number of jobs 10, 100, 1000 or 10000.

Criteria

test is successful if N1/N = 1

many_jobs.sh 

#!/bin/bash
for i in `seq 1 100`
do
 arcsub  -f tasks.xrls -c ARC1:https://pgs03.grid.upjs.sk:50000/arex
# arcsub -f tasks.xrls -c ARC1:https://rc1.grid.upjs.sk:50000/arex
# arcsub -f tasks.xrls -c ARC0:ldap://pgs02.grid.upjs.sk:2135/nordugrid-cluster
# arcsub -f tasks.xrls -c ARC1:https://pgs03.grid.upjs.sk:50000/arex
# arcsub -f tasks.xrls -c ARC0:ldap://rc0.grid.upjs.sk:2135/nordugrid-cluster-n
# arcsub -f tasks.xrls -c ARC0:ldap://pgs02.grid.upjs.sk:2135/nordugrid-cluster
#arcsub -f tasks.xrls -c ARC1:https://lem.grid.niif.hu:60000/arex
# ngsub -f tasks.xrls -c pgs02.grid.upjs.sk
done

prog.c

#include<stdlib.h>
#include<stdio.h>
int main(int argc, char *argv[])
{
 int i;
  for (i=1;i < argc;i++) printf("%s\n",argv[i]);
  return 0;
}

start.sh

#!/bin/sh
gcc $1 -o prog
./prog $2 $3 $4
if test $? = 0
 then exit 0
 else exit 1
fi



tasks.xrls

&(executable="start.sh")
(arguments=   "prog.c" "arg1" "arg2" "arg3" )
(inputfiles=
("start.sh" "start.sh")
("prog.c" "prog.c")
)
(outputfiles=("/"  " ")
)
(stdout="out.txt")
(stderr="err.txt")
(jobName="prog")
(cpuTime="70")
(gmlog=".log")
(disk="10")


The results I submitted N=1000, N1=850

http://bugzilla.nordugrid.org/show_bug.cgi?id=2283


FAILED

Scalability tests

Load and stress tests

Standard compliance/conformance tests

can be a reference to a functionality test

Inter-component tests

clearly identify the third-party components involved in the test

Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=Testing/ARC-Clients_emi1_rc0&oldid=3540"