ARC1/Survey

NOTE: This page is out of date and is kept for purely historical reasons.

Introduction

For this survey we prepared two ways how the survey participants can access the ARC resources:

• preinstalled ARC clients on a machine publicly available (we granted access to a server cuda1.grid.upjs.sk).
• manual installation of ARC standalone client package (avaliable for Linux and MS Windows, for MAC and Solaris there is a posibility for classic code compilation).

Access to preinstalled ARC clients

• The ARC clients are preinstalled on cuda1.grid.upjs.sk. This access is recommended for small users groups up to 10 participants (for example students). In this case each user can choose from following logins:

login: knowarc1, knowarc2,..., knowarc10

which have identical passord:knowarc

To login you have to use secure clients (ssh). Clients for secure connection are available here: http://www.openssh.com/

Example:

(Linux): ssh -l knowarc5 cuda1.grid.upjs.sk

• Users which want to use own computer or notebook may install standalone clients depending on the platform. In this case they have to follow instructions in the next section.

Installation of ARC client

ARC clients and documentation are available on the address: http://www.knowarc.eu/nox/client_dw.html

Linux standalone

Users on many Linux platforms (see list) can make use of ARC even without root privileges. For that purpose standalone package can be used. These are the steps to follow:

-Download the package from the Nordugrid website ftp://download.nordugrid.org/test/standalone/ (for now only limited subset of platforms is supported)
-Untar, unzip and cd into it
-Run installation script 

After setting the env variables appropriately (PATH,MANPATH), the full set of ARC client tools is at your disposal and ready for testing.

Windows XP/Vista/7 installer

http://download.nordugrid.org/test/nordugrid-arc-nox-1.0.0-0.rc6-1.exe

Read README.txt after installation to finish installation.

Client configuration

The client configuration is located in client.conf file. Its most important variables are following:

[common]
defaultservices=computing:ARC1:https://interop.grid.niif.hu:2010/arex-x509 index:ARC0:ldap://vls.grid.upjs.sk:2135/Mds-Vo-name=Slovakia,o=grid
keypath=/home/<user>/.globus/userkey.pem
certificatepath=/home/<user>/.globus/usercert.pem
cacertificatesdirectory=/etc/grid-security/certificates

For MS Windows use this client.conf:

[common]
defaultservices=computing:ARC1:https://interop.grid.niif.hu:2010/arex-x509

Certificates

User demo key and certificate can be found at [1]. The password is knowarc. The demo CA certificate can be found at [2]

The standard location of user key and certificate is:

-(Linux) ~/.globus/
-(Windows)<home-directory>\.globus
-(MAC) ~/.globus/

The standard location of CA certificates is

-(Linux) /etc/grid-security/certificates
-(Windows) c:\Program files\NorduGridARC\etc\certificates
-(MAC) /etc/grid-security/certificates

But the users may use their own location. In that case the client.conf file has to be updated accordingly

Computational tasks

Grid has features of a universal computer http://en.wikipedia.org/wiki/Turing_machine thus users can prepare a program to solve required computational tasks.

We prepared two task examples:

1. simple task
2. parametric task

Each survey participant may use own computational task which has to be modified for grid submission (job submission script). We proposed the simple parametric task with job submission scripts to demonstrate a simple job submission.

Simple Task

We assume that binary of a required program is installed on remote computer. We use date which is installed on each Linux machine.

Task description file: simple.xrsl

&(executable="/bin/date")
(stdout="out.txt")
(stderr="err.txt")
(jobName="date")

Example:

arcsub simple.xrsl

Parametric task

This example shows all required files to submit program written in C language. Examples show the commands for both job submissions: OS Linux (local submission) and ARC middleware (remote job submission).

Program in C file: prog.c

#include<stdlib.h>
#include<stdio.h>
int main(int argc, char *argv[])
{
 int i;
 for (i=1;i < argc;i++) printf("%s\n",argv[i]);
 return 0; 
}

Program prog prints each argument in a new line.

Example: Passing two parameters into the program (OS Linux):

gcc prog.c -o prog
./prog arg1 arg2

Output:

arg1
arg2

Example: Passing of three parameters into the program. Script start.sh runs a compilation and runs a program prog with three parameters (arg1,arg2, arg3) file: start.sh

#!/bin/sh
gcc $1 -o prog
./prog $2 $3 $4
if test $? = 0
 then exit 0
 else exit 1
fi 

To run the script start.sh (OS Linux) we have type:

./start.sh  prog.c arg1 arg2 arg3  

Output:

arg1
arg2
arg3 

The same task submitted in the grid needs the task description file in XRSL http://www.nordugrid.org/manuals.html or JSDL language. In this example XRSL file task.xrsl is used.

&(executable="start.sh")
(arguments=   "prog.c" "arg1" "arg2" "arg3" )
(inputfiles=
("start.sh" "start.sh")
("prog.c" "prog.c")
)
(outputfiles=("/"  " ")
)
(stdout="out.txt")
(stderr="err.txt")
(jobName="prog")
(cpuTime="70")
(gmlog=".log")
(disk="10")

To submit the task in the grid we have to type:

arcsub task.xrsl

To submit the task (MS Windows) in the grid we have to type:

arcsub task.xrsl -j jobs.xml

-j option serves to create list of submitted jobs (jobIDs). Use the switch at any command which enables it (MS Windows), for example arcget.

Recommended commands

To fill the questionnaire we recommend to try a few commands, for example an initialization of grid-proxy, job submission, data manipulation and etc. The commands are for people which are not familiar with grid computing or ARC middleware.

Initialization of grid proxy

• To initialize proxy certificate type the command (if some of your credential are not placed on a standard location the command may differ):

 $ arcproxy 

You are asked to type the user key password (for example users knowar1-knowarc11 hove to type: knowarc)

Your identity: /DC=eu/DC=KnowARC/O=Usability/CN=knowarc1

Enter pass phrase for /home/knowarc1/.globus/userkey.pem:

Correct initialization:

Your identity: /DC=eu/DC=KnowARC/O=Usability/CN=knowarc1

Enter pass phrase for /home/knowarc1/.globus/userkey.pem:

...++++++

.........++++++

Proxy generation succeeded

Your proxy is valid until: 2009-11-19 09:33:33

• To check validity of proxy certificate type:

 $ arcproxy -I 

Correct information looks like:

Subject: /DC=eu/DC=KnowARC/O=Usability/CN=knowarc1/CN=1436201404

Identity: /DC=eu/DC=KnowARC/O=Usability/CN=knowarc1

Timeleft for proxy: 11 hours 57 minutes 59 seconds

Proxy path: /tmp/x509up_u506

Proxy type: X.509 Proxy Certificate Profile RFC compliant restricted proxy

Job submission

• To submit job, type (notice that before you submit the command below, you need to have your clint.conf correctly configured):

 arcsub -f task.xrsl 

Correct job submission have to end with job identification number:

Job submitted with jobid: https://interop.grid.niif.hu:2010/arex-x509/119461258579190381739298

Job control

• To check status of all jobs type:

arcstat -a

The correct result could like this:

Job: https://interop.grid.niif.hu:2010/arex-x509/1246012585760392060202524

Name: prog

State: Finished (FINISHED)

Exit Code: 0

Job: https://interop.grid.niif.hu:2010/arex-x509/119461258579190381739298

Name: prog

State: Running (INLRMS:R)

arcstat <jobID>

Example:

arcstat https://pgs03.grid.upjs.sk:60000/arex/289471258705052330830588

Job: https://pgs03.grid.upjs.sk:60000/arex/289471258705052330830588

Name: prog

State: Finished (FINISHED)

Exit Code: 0

• Download of session directory(-ies):

-To download all finished jobs type:

arcget -a 

-To download specific job type:

arcget jobid 

Example:

arcget https://pgs03.grid.upjs.sk:60000/arex/289471258705052330830588

(after the job is downloaded it is deleted on the server side)

• If you want to kill the job with jobID then you have to type:

 
arckill jobid

Example:

arckill https://interop.grid.niif.hu:2010/arex-x509/119461258579190381739298

Data manipulation

To check data in job directory or in storage elements ARC provides set of functions.

If a job with jobID is submitted then we can list the job directory using function:arcls

arcls jobid

Example:

arcls https://interop.grid.niif.hu:2010/arex-x509/3167312587014182021216071

prog

err.txt

prog.c

out.txt

start.sh

.log

• To copy the file file type:

arccp jobid/file file

arccp https://interop.grid.niif.hu:2010/arex-x509/12460125870632416314211/out.txt out.txt

• To read standard outpot file out.txt type:

arccat -o jobid

Example:

arccat -o gsiftp://vls.grid.upjs.sk:2811/jobs/1169512587037531753203459

stdout from job gsiftp://vls.grid.upjs.sk:2811/jobs/1169512587037531753203459

arg1

arg2

arg3