This wiki is obsolete, see the NorduGrid web pages for up to date information.

MS Windows

From NorduGrid
Jump to navigationJump to search

Warn.png

NOTE: This page is out of date and is kept for purely historical reasons.
Support for ARC on Windows has been reduced due to other priorities. The following instructions may or may not work with the latest ARC client version

ARC is working on Windows (XP, Vista)

Bugs

Reports on the testing of the latest and previous versions can be found on https://www.knowarc.eu/wiki/index.php/Final_ARC_Release/Client_tests#CLI_Windows.

  • #1739 userkey.pem may not have a password
  • #1740 arcget does not create directory

Packages

You need to install these packages:

Document

Deliverable 5.3-1 Native client interface on MS Windows and Mac OS-X

Environment setup

Control panel->System->Special->Environment variables->System variables

You need to set these variables:

  • X509_USER_CERT
  • X509_USER_KEY

Automatic testing

Test scrips for windows

Example client configuration

Here are some test services.

ARC0, ARC1, Unicore job submission

  • RFC proxy creation for these middlewares:
C:\>arcproxy
Your identity: /C=HU/O=NIIF CA/OU=GRID/OU=NIIF/CN=Gabor Roczei/emailAddress=roczei@niif.hu
...............................................++++++
...............................++++++
Proxy generation succeeded
Your proxy is valid until: Wed, 08 Jul 2009 21:37:09 GMT

C:\>
  • ARC0:
C:\>arcsub -z client.xml -c arc0 test.jsdl
Job submitted with jobid: gsiftp://knowarc1.grid.niif.hu:2811/jobs/318731247053967475390863
C:\>
  • ARC1:
C:\>arcsub -z client.xml -c arex2 test.jsdl
Job submitted with jobid: https://interop.grid.niif.hu:2010/arex-x509/15711247053729737964426
C:\>
  • Unicore
C:\>arcsub -z client.xml -c unicore test.jsdl
Job submitted with jobid: https://testbed5.grid.upjs.sk:8080/KnowARC-testbed/services/BESActivity?res=447c8dd6-5e2c-4be7-bbca-6a50ec86ec44
C:\>

CREAM job submission

VOMS file: 

[roczei@knowarc1:~] $cat /etc/vomses 
"gin.ggf.org" "kuiken.nikhef.nl" "15050" "/O=dutchgrid/O=hosts/OU=nikhef.nl/CN=kuiken.nikhef.nl" "gin.ggf.org"
"pamela" "voms.cnaf.infn.it" "15013" "/C=IT/O=INFN/OU=Host/L=CNAF/CN=voms.cnaf.infn.it" "pamela"
"atlas-old" "voms.cern.ch" "15001" "/C=CH/O=CERN/OU=GRID/CN=host/voms.cern.ch" "atlas-old"
"atlas-lcg" "lcg-voms.cern.ch" "15001" "/C=CH/O=CERN/OU=GRID/CN=host/lcg-voms.cern.ch" "atlas-lcg"
"knowarc.eu" "arthur.hep.lu.se" "15001" "/O=Grid/O=NorduGrid/CN=host/arthur.hep.lu.se" "knowarc.eu"
"atlas" "voms.cern.ch" "15001" "/DC=ch/DC=cern/OU=computers/CN=voms.cern.ch" "atlas"
"nordugrid.org" "voms.uninett.no" "15015" "/O=Grid/O=NorduGrid/CN=host/voms.ndgf.org" "nordugrid.org"
"playground.knowarc.eu" "arthur.hep.lu.se" "15002" "/O=Grid/O=NorduGrid/CN=host/arthur.hep.lu.se" "playground.knowarc.eu" 
"dteam" "lcg-voms.cern.ch" "15004" "/DC=ch/DC=cern/OU=computers/CN=lcg-voms.cern.ch" "dteam"
[roczei@knowarc1:~] $
  • Legacy proxy creation
C:\>arcproxy -O -S knowarc.eu
Your identity: /C=HU/O=NIIF CA/OU=GRID/OU=NIIF/CN=Gabor Roczei/emailAddress=roczei@niif.hu
......++++++
....++++++
Contacting VOMS server (named knowarc.eu ): arthur.hep.lu.se on port: 15001
Proxy generation succeeded
Your proxy is valid until: Wed, 08 Jul 2009 21:39:25 GMT

C:\>



C:\>arcsub -z client.xml -c cream job.jdl
Job submitted with jobid: https://cream.grid.upjs.sk:8443/ce-cream/services/CREAM2/CREAM534149033

C:\>

ARC1 win32 development environment install

Download

The size of this zip file is 240 MByte:

http://www.knowarc.eu/download/win32-arc-development.zip

The environment need about 1 GByte when you uncompress it.

These directories will be created (uncompress dir should be the "c:\"): 

c:\GTK
c:\Globus
c:\MinGW
c:\OpenSSL
c:\Python24
c:\msys

MSYS shell executable

c:\msys\1.0\msys.bat


Limitation

This python error has not fixed yet: 

test -z "C:\Python24\Lib\site-packages" || /bin/mkdir -p "C:\Python24\Lib\site-packages"
/bin/sh ../libtool --mode=install /bin/install -c  '_arc.la' 'C:\Python24\Lib\site-packages/_arc.la'
libtool: install: /bin/install -c .libs/_arc.dll.a C:\Python24\Lib\site-packages/_arc.dll.a
/bin/install: cannot create regular file `C:Python24Libsite-packages/_arc.dll.a': No such file or directory
make[2]: *** [install-pythonLTLIBRARIES] Error 1
make[2]: Leaving directory `/home/Rendszergazda/arc1/python'
make[1]: *** [install-am] Error 2
make[1]: Leaving directory `/home/Rendszergazda/arc1/python'
make: *** [install-recursive] Error 1

Resolution:

Copy these files to the install directory (C:\Python24\Lib\site-packages) because we have not found other solution yet

Example configure command

./configure CXXFLAGS="-g -O0 -I/usr/local/include -I/usr/local/include/python2.4" OPENSSL_CFLAGS="-I/c/OpenSSL/include" \
OPENSSL_LIBS="-L/c/OpenSSL -lssl32 -leay32"  --prefix=/c/arc1 LDFLAGS="-L/usr/local/bin -L/usr/local/lib -L/usr/local/BerkeleyDB.4.7/bin \
-L/usr/local/lib/python2.4"  --with-dbcxx-include="/usr/local/BerkeleyDB.4.7/include" --disable-java --enable-storage-service \
--enable-sched-service --enable-paul-service --enable-hopi-service --enable-compiler-service --enable-charon-service \
 --enable-isi-service --enable-python

SVN clients

GUI (TortoiseSVN)

http://sourceforge.net/projects/tortoisesvn/files/Application/

CLI (Subversion)

http://subversion.tigris.org/getting.html#binary-packages

Globus packages

The same set of Globus packages is available as for Fedora and Debian modulo the globus-rls-server (we don't need the server to run on Windows, and it has additional external dependencies) and globus-usage (it is only used to build globus-rls-server).

RPM repo (binary packages): 

 http://www.grid.tsl.uu.se/repos/globus-mingw/noarch/

RPM repo (source packages): 

 http://www.grid.tsl.uu.se/repos/globus-mingw/src/

Windows installer: 

 http://www.grid.tsl.uu.se/repos/globus-mingw/GlobusInstaller.exe

Build instructions (specfiles) for cross-compilation can be found under: 

 http://svn.nordugrid.org/repos/workarea/T5.2/fedora-distribution/

Test listing

Important: please set the GLOBUS_LOCATION variable 

 $ export HOMEDRIVE='Z:'
 $ export HOMEPATH='\home\ellert'
 $ export GLOBUS_LOCATION='Z:\usr\i686-pc-mingw32\sys-root\mingw'
 
 $ mkdir ~/.wine/drive_c/windows/system32/drivers/etc
 $ ln -s /etc/grid-security ~/.wine/drive_c/windows/system32/drivers/etc
 
 $ /usr/i686-pc-mingw32/sys-root/mingw/bin/grid-proxy-init.exe -pwstdin
 fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
 fixme:toolhelp:Heap32ListFirst : stub
  Your identity: /O=Grid/O=NorduGrid/OU=fysast.uu.se/CN=Mattias Ellert
 <passphrase was here>
 Creating proxy .................................... Done
 Your proxy is valid until: Mon May 11 10:48:36 2009
 
 $ /usr/i686-pc-mingw32/sys-root/mingw/bin/grid-proxy-info.exe 
 fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
 fixme:toolhelp:Heap32ListFirst : stub
 subject  : /O=Grid/O=NorduGrid/OU=fysast.uu.se/CN=Mattias Ellert/CN=735918324
 issuer   : /O=Grid/O=NorduGrid/OU=fysast.uu.se/CN=Mattias Ellert
 identity : /O=Grid/O=NorduGrid/OU=fysast.uu.se/CN=Mattias Ellert
 type     : RFC 3820 compliant impersonation proxy
 strength : 512 bits
 path     : Z:\home\ellert\x509up_uellert
 timeleft : 11:58:44
 
 $ /usr/i686-pc-mingw32/sys-root/mingw/bin/globus-url-copy.exe gsiftp://grid.tsl.uu.se/storage    /test/echo.sh echo.sh
 fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
 fixme:toolhelp:Heap32ListFirst : stub
 fixme:ntdll:NtQueryInformationFile Unsupported class (16)
 
 $ cat echo.sh 
 #!/bin/sh
 
 echo $*
 
 $ /usr/i686-pc-mingw32/sys-root/mingw/bin/globus-rls-cli.exe query lrc lfn __storage_service__     rls://grid.tsl.uu.se
 fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
 fixme:toolhelp:Heap32ListFirst : stub
   __storage_service__ se://alice.grid.upjs.sk:8000/sse
   __storage_service__ se://knowarc1.grid.niif.hu:8000/se
   __storage_service__ se://morpheus.dcgc.dk:8000/se
   __storage_service__ se://pgs02.grid.upjs.sk:8000/sse


The "fixme" lines will not show up if you are running on a real Windows. The -pwstdin flag for grid-proxy-init should not be necessary either (wine doesn't do _getch() very well).

Portability issues

  • Directory separator handling: MS Windows is using this “\” while Unix systems has “/”. The solution was to modify the source code to handle different directory separators using the generic macros G_DIR_SEPARATOR_S and G_DIR_SEPARATOR from glibmm, as these are operating system independent solutions.
  • Temporary directory location is different on MS Windows compared to Unix systems. The MS Windows temporary directory is located in the local user directory but on Unix it is put in a "central" place (“/tmp”). To solve this problem in a platform independent way, the source code was adapted to use the glibmm method named Glib::get_tmp_dir().
  • Duplicate name definitions
    • An internal ARC method named CopyFile conflicted with method provided by the MinGW framework. This was resolved by renaming the ARC method to ARCCopyFile.
    • The Arc namespace conflicted with components in the MS Windows GDI (Graphics Device Interface). The solution was to disable GDI as this is not used by the client interface. Initially target for Unix, parts of the source code relied on Unix specific password data structures, which does not exist on MS windows. To accommodate this, a new structure (“passwd”) was defined in the Arc specific win32.h header file.
  • Proxy creation
    • Proxy file creation using the arcproxy utility produced incorrect expiration times. The affected OpenSSL code in ARC was corrected.
    • For the CREAM ACC flavour, the created proxy time was inconsistent and resulted in a delegation failure. The solution was to shift the proxy time a few minutes.
  • Thread handling
    • Since the pthread library is not part of the MinGW/MSYS environment, a custom ARC Thread method, Arc::CreateThreadFunction, was created as a wrapper method for Glib thread creation. The LDAP DMC was modified appropriately to use this creation of LDAP binding threads.
    • ARC job submission caused a thread lock due to a blocking problem in ARC TargetGenerator method. It was corrected using Glib::Mutex::Lock.
  • An array overflow inside the ARC DMC caused the arccp program to enter in an infinite loop and consume all available memory. This problem was removed by increasing the array size appropriately.
  • The command line parsing required users to add an extra dash ('-') for any optional argument. This was solved by upgrading glibmm to version 2.6 on MS Windows.
Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=MS_Windows&oldid=6270"