This wiki is obsolete, see the NorduGrid web pages for up to date information.

NOX/MacOSX

From NorduGrid
< NOX
Jump to navigationJump to search

NOX-1.0.0 binaries for Mac OS X

These binaries are created with MacPorts. Currently no Globus support!

Some notes about client usage

The Mac OS X is in several ways similar to a linux OS, thus all the descriptions for linux most probably applies here as well.

After installing this package, the PATH should be set to location of the binaries, for example by adding this to the ~/.profile: 

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

The user's certificate and key file by default should be placed into ~/.globus as usercert.pem and userkey.pem, and the CA certificates should be placed by default into /etc/grid-security/certificates. If these files would be in a different directory, then their location should be indicated in the client.conf file.

The client.conf file goes by default into ~/.arc.

In order to use the chelonia CLI tool, it is needed to tell python where are the python bindingd for the ARC client libraries. This can be done with setting the PYTHONPATH variable. The Snow Leopard packages but those here: 

$ export PYTHONPATH=/opt/local/lib/python2.6/site-packages:$PYTHONPATH

The Leopard package install the libraries here: 

$ export PYTHONPATH=/Library/Python/2.5/site-packages:$PYTHONPATH

Example

If the certificate files are in their default directories, then we can have a client.conf like this: 

$ cat ~/.arc/client.conf
defaultservices=index:ARC1:https://knowarc2.grid.niif.hu:50000/isis

More information on client and voms configuration.

With the arcproxy command, we could create our proxy: 

$ arcproxy
Your identity: /DC=eu/DC=KnowARC/O=Usability/CN=knowarc11
Enter pass phrase for /Users/niif/.globus/userkey.pem:
........................................................................++++++
..++++++
Proxy generation succeeded
Your proxy is valid until: 2009-11-26 05:52:15

We can submit a basic job: 

$ cat simple.xrsl 
&(executable="/bin/date")
(stdout="out.txt")
(stderr="err.txt")
(jobName="date")

jim:~ niif$ arcsub simple.xrsl
   [some error messages could be here because of default services not currently online]
Job submitted with jobid:
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397

Getting the job's status and data: 

$ arcstat
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
Job: http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
Name: date
State: Running (INLRMS:EXECUTED)

$ arccat
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
stdout from job
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
Wed Nov 25 17:55:23 CET 2009


$ arcstat
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
Job: http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
Name: date
State: Finished (FINISHED)
Exit Code: 0

$ arcget
http://knowarc1.grid.niif.hu:50006/arex/119461259168082271239397
jim:~ niif$ 
jim:~ niif$ ls -l 119461259168082271239397/
total 8
-rw-------  1 niif  staff   0 Nov 25 17:58 err.txt
-rw-------  1 niif  staff  29 Nov 25 17:58 out.txt

$ cat 119461259168082271239397/out.txt 
Wed Nov 25 17:55:23 CET 2009

More examples in ARC client tutorial.

Full package (with server and client)

WARNING: Installing this package copies several libraries into /opt/local possibly overwriting the existing files inside, so if you ever used MacPorts, this could break things!

The Leopard version for some reason installs the python libraries and services to /Library/Python/2.5. Both versions put a file called a-rex into /Library/LaunchDaemons - I don't know what is this file.

These are the MPKG files created by MacPorts (in ZIP archive):

How to try the package

Installing this MPKG to a freshly installed Mac OS X, it would create the /opt/local directory with all these files (you need root access for this - maybe we could install these into the user's home directory, then we would not need root access to install the server part of ARC?). Then you can run arched. Let's try it with the C++ echo service, with this config: 

$ cat echo.xml 
<?xml version="1.0"?>
<cfg:ArcConfig xmlns="http://www.nordugrid.org/schemas/loader/2009/08"
	xmlns:cfg="http://www.nordugrid.org/schemas/arcconfig/2009/08"
	xmlns:tcp="http://www.nordugrid.org/schemas/tcp/2009/08"
	xmlns:tls="http://www.nordugrid.org/schemas/tls/2009/08"
	xmlns:echo="http://www.nordugrid.org/schemas/echo/2009/08">
    <cfg:Server>
        <cfg:PidFile>/tmp/arched.pid</cfg:PidFile>
        <cfg:Logger>
            <cfg:File>/tmp/arched.log</cfg:File>
            <cfg:Level>DEBUG</cfg:Level>
        </cfg:Logger>
    </cfg:Server>
    <ModuleManager>
        <Path>/opt/local/lib/arc/</Path>
    </ModuleManager>
    <Plugins>
        <Name>mcctls</Name>
        <Name>mcchttp</Name>
        <Name>mccsoap</Name>
        <Name>mcctcp</Name>
    </Plugins>
    <Chain>
        <Component name="tcp.service" id="tcp">
            <next id="tls"/>
            <tcp:Listen>
               <tcp:Interface>0.0.0.0</tcp:Interface>
               <tcp:Port>50000</tcp:Port>
               <tcp:Version>4</tcp:Version>
            </tcp:Listen>
        </Component>
        <Component name="tls.service" id="tls">
            <next id="http"/>
            <tls:KeyPath>/etc/grid-security/hostkey.pem</tls:KeyPath>
            <tls:CertificatePath>/etc/grid-security/hostcert.pem</tls:CertificatePath>
            <tls:CACertificatesDir>/etc/grid-security/certificates</tls:CACertificatesDir>
        </Component>
        <Component name="http.service" id="http">
            <next id="soap">POST</next>
            <next id="plexer">GET</next>
            <next id="plexer">PUT</next>
        </Component>
        <Component name="soap.service" id="soap">
            <next id="plexer"/>
        </Component>
        <Plexer name="plexer.service" id="plexer">
            <next id="echo">^/Echo$</next>
        </Plexer>
        <Service name="echo" id="echo">
            <echo:prefix>[</echo:prefix>
            <echo:suffix>]</echo:suffix>
        </Service>
    </Chain>
</cfg:ArcConfig>

Note the /opt/local/lib/arc part! Let's put some certificates in the proper directories, then run arched: 

$ /opt/local/sbin/arched -f -c echo.xml 
[2009-11-21 16:40:22] [Arc.MCC.TCP] [DEBUG] [313/4302374256] Trying to listen on TCP port 50000(IPv4)
[2009-11-21 16:40:22] [Arc.MCC.TCP] [INFO] [313/4302374256] Listening on TCP port 50000(IPv4)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded MCC tcp.service(tcp)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded MCC tls.service(tls)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded MCC http.service(http)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded MCC soap.service(soap)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded Plexer plexer.service(plexer)
[2009-11-21 16:40:22] [Arc.InfoSys] [DEBUG] [313/4302374256] InfoRegister created with config:
<?xml version="1.0"?>
<Service xmlns="http://www.nordugrid.org/schemas/loader/2009/08" xmlns:echo="http://www.nordugrid.org/schemas/echo/2009/08" name="echo" id="echo">
            <echo:prefix>[</echo:prefix>
            <echo:suffix>]</echo:suffix>
        </Service>

[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Loaded Service echo(echo)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC tcp.service(tcp) to MCC (tls) under (empty)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC tls.service(tls) to MCC (http) under (empty)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC http.service(http) to Plexer (plexer) under GET
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC http.service(http) to MCC (soap) under POST
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC http.service(http) to Plexer (plexer) under PUT
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking MCC soap.service(soap) to Plexer (plexer) under (empty)
[2009-11-21 16:40:22] [Arc.Loader] [INFO] [313/4302374256] Linking Plexer plexer to Service (echo) under ^/Echo$
[2009-11-21 16:40:22] [Arc] [INFO] [313/4302374256] Service side MCCs are loaded

Let's create some client.conf and put some client certificates in the proper places: 

$ cat .arc/client.conf 
keypath=/Users/niif/.arc/userkey.pem
certificatepath=/Users/niif/.arc/usercert.pem
cacertificatesdirectory=/etc/grid-security/certificates

Then use the echo_client.py to test it, which will as a side effect test the python bindings as well, so set the PYTHONPATH first (on Leopard this would be /Library/Python/2.5/site-packages) 

$ export PYTHONPATH=/opt/local/lib/python2.6/site-packages/
$ python echo_client.py https://localhost:50000/Echo
<soap-env:Envelope xmlns:echo="urn:echo" xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap-env:Body>
    <echo:echoResponse>
      <echo:hear>[hi!]</echo:hear>
    </echo:echoResponse>
  </soap-env:Body>
</soap-env:Envelope>

It seems working. I tried to run Chelonia with a centralized set of services, and it was working fine.

How was the package created

NOX 1.1rc2

File:Portfile-1.1rc2.txt

NOX 1.1rc1

File:Nordugrid-nox-1.1.0rc1-Portfile.txt

NOX 1.0.0

This is a basic Portfile for NOX-1.0.0: 

# $Id$

PortSystem          1.0

name                nordugrid-nox
version             1.0.0
categories          net
maintainers         Maintainers
description         The Nox 1.0.0 release of the Advanced Resource Connector
long_description    \
        The Advanced Resource Connector (ARC) middleware, introduced by \
        NorduGrid (www.nordugrid.org), is an open source software solution \
        enabling production quality computational and data Grids since May \
        2002. The latest production ARC release, the version 0.8.1 was \
        released on November 6, 2009. \
 \
        The Nox release of the ARC software collects and integrates several \
        innovative next generation services and client tools into a consistent \
        software release. Nox components have been developed by the KnowARC \
        project over the last three years and represent software services of \
        different levels of maturity. Some of them are already deployable in \
        production environments, while others should be considered only as \
        technology preview prototypes. \
 \
        Nox is not meant to be a replacement of the current line of ARC \
        production releases. The purpose of the release is to offer an early \
        possibility to try out the new Web Service (WS) based \
        standard-compliant components before they can appear in one of the \
        coming production ARC releases. Some of the Nox components are \
        actually already a part of the latest production ARC release. The Nox \
        release can be deployed simultaneously with a production release ARC \
        installation. \
 \
        The Nox release makes the ARC middleware available on additional \
        long-awaited popular platforms. ARC has long been known for its very \
        good portability and support for wide range of Linux versions. This \
        release of ARC moves one step further by initiating the inclusion of \
        Nox packages into the Linux distributions themselves. The design of \
        ARC and the careful choice of underlying dependencies takes the \
        portability of the code to a new level. Nox is now available on \
        Microsoft Windows, Mac OS X and Solaris. \
 \
        The release code name Nox originates from the date of the end of \
        KnowARC project: November 2009. N - for November and Ox for the \
        corresponding Lunar year. \
 \
        Like the rest of ARC, Nox is released under Apache 2.0 license.
homepage            http://www.nordugrid.org
platforms           darwin
master_sites        http://download.nordugrid.org/software/nordugrid-arc-nox/releases/1.0.0/src
distfiles           nordugrid-arc-nox-1.0.0.tar.gz
checksums           md5     da296f402ab87be6a3be4cf1cc758120 \
                    sha1    abea0c30e93ade85f86f0538ce6b2c220c0b470b \
                    rmd160  66cd9c40cddf647442dceca67e8148e1df1a29d0
depends_build       port:swig-python
depends_lib         port:glibmm \
                    port:libxml2 \
                    port:db46 \
                    port:xmlsec
worksrcdir          nordugrid-arc-nox-1.0.0
pre-configure       {
    system "cd ${worksrcpath} && ./autogen.sh"
    system "cd ${worksrcpath}/po && sed -i.msgmergehack 's/@MSGMERGE@ --update/@MSGMERGE@ --update --backup=off/' Makefile.in.in"
}
configure.args-append --disable-java --with-dbcxx-include="/opt/local/include/db46"
configure.ldflags-append "-L/opt/local/lib/db46"

The swig-python dependency is only needed for the build, but not for running, the others are library dependencies, will be packaged into the multipackage MPKG file.

With this, and after creating a local macports repository, you can issue the 

port mpkg nordugrid-nox

command, which will create a multi-package with lots of pkg files for all the dependencies, on Snow Leopard it looks like this: 

$ ls -l nordugrid-nox-1.0.0-snow-leopard.mpkg/Contents/Packages/
total 0
drwxr-xr-x  3 niif  staff  102 Nov 30 09:00 db46-4.6.21.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:01 expat-2.0.1.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:01 gettext-0.17.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:01 glib2-2.22.2.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 glibmm-2.22.1.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 libiconv-1.13.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 libsigcxx2-2.2.4.2.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 libxml2-2.7.6.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 libxslt-1.1.26.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 ncurses-5.7.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:02 ncursesw-5.7.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 nordugrid-nox-1.0.0.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 openssl-0.9.8l.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 perl5-5.8.9.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 python_select-0.3.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 xmlsec-1.2.9.pkg
drwxr-xr-x  3 niif  staff  102 Nov 30 09:03 zlib-1.2.3.pkg

On Leopard it contains an additional perl package for some reason: 

drwxr-xr-x  3 niif  staff  102 Nov 21 20:49 perl5.8-5.8.9.pkg

We can check where will be files installed with the lsbom command, let's grep for anything which will not end up in /opt (this is the Snow Leopard package: 

$ lsbom nordugrid-nox-1.0.0-snow-leopard.mpkg/Contents/Packages/*/*/*.bom | head
.	40755	501/20
./opt	40755	0/80
./opt/local	40755	0/80
./opt/local/bin	40755	0/80
./opt/local/bin/db46_archive	100555	0/80	13592	547096937
./opt/local/bin/db46_checkpoint	100555	0/80	14128	3659331835
./opt/local/bin/db46_codegen	100555	0/80	26392	4171954899
./opt/local/bin/db46_deadlock	100555	0/80	14176	3050266610
./opt/local/bin/db46_dump	100555	0/80	14016	4025004395
./opt/local/bin/db46_dump185	100555	0/80	9352	267811715
niifs-macbook:Packages niif$ lsbom */*/*.bom | grep -v opt
$ lsbom nordugrid-nox-1.0.0-snow-leopard.mpkg/Contents/Packages/*/*/*.bom | grep -v opt
  [...]
./Library	41775	0/80
./Library/LaunchDaemons	40755	0/0
./Library/LaunchDaemons/a-rex	100775	0/0	12090	633116243
  [...]

Client package

Currently I don't know how to create a standalone client package which would not need root privileges to install it, so better just install the full packages.

Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=NOX/MacOSX&oldid=2012"