This wiki is obsolete, see the NorduGrid web pages for up to date information.

Service Registration

From NorduGrid
Jump to navigationJump to search

Note: The information in this page is work in progress. Please also consult the older not fully up-to-date How to plug an ARC site into EGI for the moment being.

EGI maintains a central catalogue of deployed services, participating sites and countries/regions. Being part of EGI, i.e. offering services for EGI, requires proper registration records in the Grid Configuration Database (GOCDB). GOCDB (https://goc.egi.eu/) is the central system for recording site information, their hosted Service Endpoints and their operational status.

GOCDB organizes and stores information following its own hierarchical model of the Grid: The basic unit is the GOCDB Service Endpoint (SE), the service endpoint represents a deployed/hosted single service defined by the hosting machine (hostname), the actual service (of certain service type) and the endpoint URL necessary to utilize the service. In EGI a service (GOCDB SE) always must be associated with a site. A GOCDB site is an administrative concept, a grouping of multiple Service Endpoints administrated/managed together. Information about sites includes various location, contact, timezone-related information AND the contact URL of the site LDAP information service.

Since GOCDB does not provide a full service or site information, in addition to GOCDB another information discovery system is operated to store such data: The TOP/Site BDII network (http://gridinfo.web.cern.ch/). GOCDB itself relies on information stored in TOP/Site BDII.

ARC services should be registered in GOCDB as Service Endpoints under a properly configured GOCDB Site entry AND ARC services also should be made visible for the TOP/Site-BDII network. Below we present detailed instructions for number of deployment scenarios:

  1. A (dedicated) GOCDB site hosting only one ARC CE
  2. A set of (dedicated) GOCDB sites each hosting only one ARC CE
  3. A (distributed) GOCDB site grouping several ARC services
  4. A GOCDB site hosting a mixture of ARC and non-ARC services (e.g. ARC CE, dCache, ..)

We suggest the system administrator to have the following references open during configuration:

It is NOT needed to read the Top-BDII documentation http://gridinfo.web.cern.ch/ .

Dedicated GOCDB site with one ARC CE and no Site-BDII

This is the simplest deployment scenario. Only one ARC CE is deployed and to be registered to EGI in a Resource Centre. The Resource Center has no other services, including non-ARC services for EGI than the single ARC CE. The Resource Center has no Site-BDII deployed, and does not need to deploy any, as ARC CE will publish the needed information through GLUE2.

The suggested setup for such scenario is the following:

A dedicated site entry is configured as a GOCDB-Site accommodating the single ARC CE as a GOCDB-SE. The ARC CE is configured in such a way that it publishes the necessary site information in a compatible format with Top-BDII.

As a result the ARC CE will automatically appear in the Top-BDII as well because Top-BDII will pull information directly from it based on the information entered in GOCDB-Site record.

To achieve this, the following steps are needed:

1. Creation of GOCDB Entries

Prerequisites

  1. To have the right GOCDB permissions to be able to 'Add Site' and 'Add Service Endpoint'. The procedure to obtain such clearance level is beyond the scope of this tutorial. EGI website is probably the best place to start.
  2. Define a GOCDB-Site Name
  3. Collect the administrative information needed to define a GOCDB-Site, like Email, Phone contacts and Location.
  4. Collect the Hostname of the ARC CE and the port of the ARC CE LDAP Information System.

1.a GOCDB-Site entry

In order to create a GOCDB-Site entry go to https://goc.egi.eu/portal/index.php?Page_Type=New_Site and fill the necessary records.

  • (mandatory) A unique (short) name: This is the main identifier of the Site and needs to be chosen carefully. This name is used in all operation services (APEL, NAGIOS, BDII) and for the GIIS URL below.
  • (critical for ARC) The site's GIIS URL: This is the ldap URL where the ARC CE presents GLUE2 Site information. This URL must be constructed this way (case sensitive!):


ldap:// <ARC CE hostname> : <ARC CE LDAP Information System port> /GLUE2DomainID=urn:ad: <unique short name of the GOCDB-Site>,o=glue


Example:
  • unique short name of GOCDB-Site: NIIF_Testbed
  • ARC CE hostname: gemaster.grid.niif.hu
  • ARC CE LDAP Information System port: 2135 (this port is the one in the [infosys] block in arc.conf)

resulting GIIS URL: ldap://gemaster.grid.niif.hu:2135/GLUE2DomainID=urn:ad:NIIF_Testbed,o=glue


  • (mandatory) A domain name : the network domain hosting the services. Example: for arcce.myorganization.org domain will be myorganization.org
  • (mandatory) A contact email address and telephone number
  • Production Infrastructure: unless you understand the requirements for a production site, set it to Test . Once your site is Certified, this can be upgraded to production.
  • Is this site visible to EGI? : If a Site is marked as being visible to EGI then it will be exposed to the central operational tools. If you don't understand this, do NOT mark this checkbox.
  • and other dozen self-explanatory optional administrative attributes.

1.b GOCDB Service Endpoint entry

In order to create a Service Endpoint entry in the GOCDB describing the ARC CE go to https://goc.egi.eu/portal/index.php?Page_Type=New_Service_Endpoint and fill the necessary records:

  • (mandatory) Hosting Site: choose the unique short name of the Site defined in step 1.a
  • (mandatory) Service Type : For an ARC CE, choose org.nordugrid.arex
  • (mandatory) Endpoint URL (Alphanumeric and $-_.+!*'(),:) : for an ARC CE, specify the GLUE2 LDAP Information System URL to discovery all the enabled interfaces of the CE.


ldap:// <ARC CE hostname> : <ARC CE LDAP Information System port> /GLUE2GroupID=services,o=glue


Example:
  • ARC CE hostname: gemaster.grid.niif.hu
  • ARC CE LDAP Information System port: 2135 (this port is the one in the [infosys] block in arc.conf)

resulting GIIS URL: ldap://gemaster.grid.niif.hu:2135/GLUE2GroupID=services,o=glue


  • (mandatory) Host name (valid FQDN format) : the hostname of the ARC CE as in arc.conf
  • (optional) Host Operating System : the value of 'opsys' in the [cluster] or [queue] block in arc.conf, if not 'adotf'
  • (optional )Host Architecture : the value of 'architecture' in the [cluster] or [queue] block in arc.conf, if not 'adotf'
  • Is it a beta service (formerly PPS service)? Select N unless you understand what this is.
  • Is this service in production? Select N unless you understand that this means.
  • Is this service monitored? Select Y if you want the remote nagios probes to be run against the CE
  • Is this service endpoint visible to EGI? If a Site is marked as being visible to EGI then it will be exposed to the central operational tools. If you don't understand this, do NOT mark this checkbox.
  • other self-explanatory optional administrative attributes.

2. Configuration of the ARC CE

Prerequisites

  • ARC 13.02 or later releases (nordugrid-arc packages from version 3.x on)
  • Have an ARC CE with a functional LDAP-based local Information System

2.a Enable GLUE2 publication

In addition to the default information published as in the NorduGrid schema, enable the LDAP GLUE2 rendering as well:

Check that in the [infosys] block in arc.conf you have the following line: 

 infosys_glue2_ldap="enable"

2.b Configure admindomain

GLUE2 AdminDomain is a conceptual representation of a "Collection of actors that manage a number of services". This entity is used to represent all the information needed for the GOCDB-Site entry.

The following configuration block in arc.conf should be edited. If it's not there, create one after the [infosys] block. 

[infosys/admindomain]
name="ARC-TESTDOMAIN"
distributed="yes"
description="ARC test Domain"
www="http://www.nordugrid.org/"
owner="admin@nordugrid.org"
otherinfo="Test Other info"

The values to be inserted in such block are as follows:

  • (mandatory) name: the unique short name of the GOCDB-Site created in step 1.a
  • (mandatory) distributed: set it to "no" for this deployment scenario.
  • the other attributes are self-explanatory and optional.
Example:
  • unique short name of GOCDB-Site: NIIF_Testbed

Resulting admindomain block: 

[infosys/admindomain]
name="NIIF_Testbed"
distributed="no"


2.c Restart services and test local information system

Restart all ARC CE services.

Execute the following commands in the follwing order:

  1. /etc/init.d/a-rex restart
  2. /etc/init.d/nordugrid-arc-bdii stop
  3. /etc/init.d/nordugrid-arc-slapd stop
  4. /etc/init.d/nordugrid-arc-slapd start
  5. /etc/init.d/nordugrid-arc-bdii start

Wait for 5 minutes for all the infosys components to update the LDAP databases.

then check the publication is working fine with the following commands:

  • check site/AdminDomain publication:
ldapsearch -x -LLL -h <ARC CE hostname> -p <ARC CE LDAP infosys port> -b 'GLUE2DomainID=urn:ad:<unique short GOCDB-Site name>,o=glue'
the records should appear with no errors and presenting the data you entered in step 2.b
Example: 
# ldapsearch -x -LLL -h gemaster.grid.niif.hu -p 2135 -b 'GLUE2DomainID=urn:ad:NIIF_Testbed,o=glue'
dn: GLUE2DomainID=urn:ad:NIIF_Testbed,o=glue
GLUE2EntityValidity: 60
GLUE2AdminDomainDistributed: TRUE
objectClass: GLUE2Domain
objectClass: GLUE2AdminDomain
GLUE2DomainID: urn:ad:NIIF_Testbed
GLUE2EntityName: NIIF_Testbed
GLUE2EntityCreationTime: 2013-07-19T11:40:12Z



  • check services publication:
ldapsearch -x -LLL -h <ARC CE hostname> -p <ARC CE LDAP infosys port> -b 'GLUE2GroupID=services,o=glue'
the records about services in the CE should appear.
Example:

This is shortened because usually services will give out lots of input. 

# ldapsearch -x -LLL -h gemaster.grid.niif.hu -p 2135 -b 'GLUE2GroupID=services,o=glue'
dn: GLUE2GroupID=services,o=glue
objectClass: GLUE2Group
GLUE2GroupID: services

dn: GLUE2ServiceID=urn:ogf:ComputingService:gemaster:arex,GLUE2GroupID=service
 s,o=glue
GLUE2ServiceServiceForeignKey: urn:ogf:ComputingService:gemaster:arex
GLUE2EntityValidity: 60
GLUE2ServiceType: org.nordugrid.arex
GLUE2ServiceID: urn:ogf:ComputingService:gemaster:arex
GLUE2ComputingServiceSuspendedJobs: 0
objectClass: GLUE2Service
objectClass: GLUE2ComputingService
GLUE2ComputingServicePreLRMSWaitingJobs: 0
GLUE2ServiceQualityLevel: testing
GLUE2ComputingServiceWaitingJobs: 0
GLUE2ServiceComplexity: endpoint=11,share=1,resource=1
GLUE2ComputingServiceTotalJobs: 0
GLUE2ServiceCapability: data.transfer.cepush.srm
GLUE2ServiceCapability: executionmanagement.jobmanager
GLUE2ServiceCapability: information.discovery.resource
...


3. Top-Bdii/Site-Bdii registration

This is achieved automatically, only testing is necessary. The whole cycle is about GOCDB to update information entered, Top-BDII to retrieve all the ldap URLs, and then Top-BDII will contact the GOCDB-Site URL to get information about the ARC CE.

The whole process might take up to three hours. Therefore we suggest to have a walk and get some coffee, and then run the following commands on a top-bdii. In the following example we've choosen lcg-bdii.cern.ch:

1. Test that domain information is present in top-bdii (timestamps may differ): 

[root@piff tmp]# ldapsearch -x -LLL -h lcg-bdii.cern.ch -p 2170 -b 'GLUE2DomainID=urn:ad:NIIF_Testbed,GLUE2GroupID=grid,o=glue' -s base
dn: GLUE2DomainID=urn:ad:NIIF_Testbed,GLUE2GroupID=grid,o=glue
GLUE2EntityValidity: 60
GLUE2AdminDomainDistributed: TRUE
objectClass: GLUE2Domain
objectClass: GLUE2AdminDomain
GLUE2DomainID: urn:ad:NIIF_Testbed
GLUE2EntityName: NIIF_Testbed
GLUE2EntityCreationTime: 2013-07-19T11:40:12Z
check that the data is consistent with the one at step 2.b

2. Test that services information is present in top-bdii: 

[root@piff tmp]# ldapsearch -x -LLL -h lcg-bdii.cern.ch -p 2170 -b 'GLUE2GroupiID=services,GLUE2DomainID=urn:ad:NIIF_Testbed,GLUE2GroupID=grid,o=glue'
dn: GLUE2GroupID=services,GLUE2DomainID=urn:ad:NIIF_Testbed,GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: services

dn: GLUE2ServiceID=urn:ogf:ComputingService:gemaster:arex,GLUE2GroupiID=services,GLUE2DomainID=urn:ad:NIIF_Testbed,GLUE2GroupID=grid,o=glue
GLUE2ServiceServiceForeignKey: urn:ogf:ComputingService:gemaster:arex
GLUE2EntityValidity: 60
GLUE2ServiceType: org.nordugrid.arex
GLUE2ServiceID: urn:ogf:ComputingService:gemaster:arex
GLUE2ComputingServiceSuspendedJobs: 0
objectClass: GLUE2Service
objectClass: GLUE2ComputingService
GLUE2ComputingServicePreLRMSWaitingJobs: 0
GLUE2ServiceQualityLevel: testing
GLUE2ComputingServiceWaitingJobs: 0
GLUE2ServiceComplexity: endpoint=11,share=1,resource=1
GLUE2ComputingServiceTotalJobs: 0
GLUE2ServiceCapability: data.transfer.cepush.srm
GLUE2ServiceCapability: executionmanagement.jobmanager
GLUE2ServiceCapability: information.discovery.resource
...

Glossary

GLUE2
A conceptual information model for Grid Entities, developed by the OGF GLUE Working Group. It has been rendered into concrete data models in LDIF (for LDAP servers) XML and SQL. Work is ongoing for JSON rendering.
More info: http://www.ogf.org/documents/GFD.147.pdf
Official Open Grid Forum site: http://www.ogf.org/
GLUE2 AdminDomain
It is conceptual a representation of a "Collection of actors that manage a number of services". It can be seen as an extension on the concept of Site, since an AdminDomain can be geographically distributed, that is, multiple actors around the world can manage a number of distributed services.
More information can be found in http://www.ogf.org/documents/GFD.147.pdf and http://www.nordugrid.org/documents/arc-ce-sysadm-guide.pdf
Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=Service_Registration&oldid=6466"