This wiki is obsolete, see the NorduGrid web pages for up to date information.
NOX/Client tutorial
Client Tutorial
This tutorial will show the basic job configuration a and job handling via the Command Line Tool (CLI)
Help
In order to see all options of a command there several possible help calls:
><command> -h ><command> -? ><command> --help
On linux systems one van also use
>man <command>
Certificate
In order to access grid resources a user has to have a certificate. Every country has its own certificate issuing authority (CA) which defines specidifc procedures. Please consult The International Grid Trust Federation site in order to locate your country's CA.
For residents of Nordic countries (Denmark, Finland, Norway, Iceland and Sweden) one has to install the necessary Globus packages and the NorduGrid CA configuration package, and then run a certificate request command:
grid-cert-request -int -ca
filling in the information which is being asked for. The certificate request (usercert_request.pem file) has to be sent by e-mail to ca@nordugrid.org. More information about certificates is available here:
Instant CA
For testing one can acquire an temporary certificate from the KnowARC instant Certification Authority portal:
https://vls.grid.upjs.sk/CA/index
Fill out the form and follow the instructions in the readme file.
Note: You have to ask resource owners to install your instant CA credenials
arcproxy
A grid session is started by creating a proxy
>arcproxy
Check the proxy information:
>arcproxy -I
To create a voms proxy use option -S:
>arcproxy -S <vo_name>
This is only the simplest option, one can also specify group an role. See the help function for more information.
To submit jobs to the CREAM service, gLite, a GSI proxy has to be created: >arcproxy -O -S <vo_name>
The KnowARC test CREAM service can be used by members of the knowarc.eu virtual organization. Contact
Create a job
The new and standard compliant way of specifying jobs is via the Job Submission Description Language (JSDL)
Copy the definition below into a file, example testjob.jsdl.
<JobDefinition
xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl"
xmlns:posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix"
>
<JobDescription>
<JobIdentification>
<JobName>test_1 job </JobName>
</JobIdentification>
<Application>
<posix:POSIXApplication>
<posix:Executable>/bin/sh</posix:Executable>
<posix:Argument>test.sh</posix:Argument>
<posix:Argument>Testing</posix:Argument>
<posix:Output>out.txt</posix:Output>
<posix:Error>err.txt</posix:Error>
</posix:POSIXApplication>
</Application>
<DataStaging>
<FileName>test.sh</FileName>
<DeleteOnTermination>false</DeleteOnTermination>
<Source><URI>http://www.fys.uio.no/~katarzp/test/test_1.sh</URI></Source>
</DataStaging>
<DataStaging>
<FileName>out.txt</FileName>
<DeleteOnTermination>false</DeleteOnTermination>
</DataStaging>
<DataStaging>
<FileName>err.txt</FileName>
<DeleteOnTermination>false</DeleteOnTermination>
</DataStaging>
</JobDescription>
</JobDefinition>
xrsl - still available
All though the JSDL is the mother tongue of the new ARC, one can still use the classical way of specifying a jobs in xrsl. The job specification will be translated before submission.
This is a simple job which downloads a program from a web location, compiles and runs it. The output is a list of prime numbers. Copy the code into a file, for example testjob.xrsl.
&("executable" = "run.sh" )
("arguments" = "2" )
("inputfiles" = ("run.sh" "http://www.fys.uio.no/~katarzp/test/run_1.sh" )
("Makefile" "http://www.fys.uio.no/~katarzp/test/Makefile" )
("prime.cpp" "http://www.fys.uio.no/~katarzp/test/prime.cpp" ) )
("stderr" = "primenumbers" )("outputfiles" = ("primenumbers" "" ))
("jobname" = "ARC primenumber test" )
("stdout" = "stdout" )
("gmlog" = "gmlog" )
("CPUTime" = "8" )
Submit job - arcsub
As default submission without any arguments will use the default services
>arcsub testjob.jsdl
A job can be directed to a certain computing service either using an alias or the
>arcsub -c ARC1:http://knowarc1.grid.niif.hu:50000/arex testjob.jsdl >arcsub -c areax1 testjob.jsdl
for xrsl and ARC0 services:
>arcsub -c arc0 testjobs.xrsl >arcsub -i ARC0:ldap://index1.nordugrid.org:2135/Mds-Vo-name=NorduGrid,o=grid testjob.xrsl
Once a job is submitted it is assigned a unique job ID for example http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Check status - arcstat
To see the status of all jobs use option -a of for a particular job, the job ID
> arcstst -a > arcstat http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
To get more detailed listing use option -l
> arcstat -l http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Read the output - arccat
As the jobs is executing one can ant any point have a look at the log files
> arccat http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Shows the standard output.
> arccat -l http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Shows the grid manager log file.
Get the results - arcget
Once the job is finished one can download the results using arcget
> arcget http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
As directory named after the session directory will be crated and it will contain all output files 1130412584325021155367992. Only files specified as output (xrsl) or in JSDL exuiped with the tag <DeleteOnTermination>false</DeleteOnTermination> will be kept for download. All other files only required for the execution of the job will be remove from the session directory when the job finishes.
Kill job - arckill
A job can be killed using the arckill command:
> arckill http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Jobs at a certain cluster can be killed using the -c/--cluster option:
> arckill -c arex1
Jobs with a certain status can be killed using the -s/--status option:
> arckill -s PREPARING
By default a job that is killed will be removed from the cluster. If one wants to download the output, for example to study error messages, one must use the -k/--keep option.
Clean jobs - arcclean
If jobs are not downloaded they remain for some time at the cluster. The jobs and it's session directory can be removed using the arcclean command.
Examples: clean single job, jobs with status FINISHED, jobs at cluster with alias arex1:
> arcclean http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992 > arcclean -s FINISHED > arcclean -c arex1
Data handling
ARC offers several different ways of datanadling and supports a number of protocols. This section show only some very basic examples.
List files - arcls
One can at any moment check the content of the session directory of a job:
> arcls http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992
Or to list the content of some storage location:
>arcls srm://grid.uio.no:8446/srm/managerv2?SFN=/johndoe/log2
detailed listing is obtained with the -l/--long option.
Copy files - arccp
Files stored on the grid can of course be copied to a local file system using the arccp command:
> arccp http://knowarc1.grid.niif.hu:50000/arex/1130412584325021155367992/remote.file local.file >arcls srm://grid.uio.no:8446/srm/managerv2?SFN=/johndoe/log2 log2.local
ARC offers also a lightweight distributed storage system - Chelonia. For a full manual see ARC Client manual
Monitor
A short while after the jobs are submitted they will appear in the monitor.
- Monitor for the test clusters in the example client.conf in client configuration instructions
- The full NorduGrid production monitor
