This wiki is obsolete, see the NorduGrid web pages for up to date information.

Data brokering

From NorduGrid
Jump to navigationJump to search


NOTE: This page is out of date and is kept for purely historical reasons.

Currently data-driven brokering functionality in ARC is very limited. This page tries to collect information about data brokering in general, and ARC specific work which should be carried out in order to improve data brokering in ARC.

Current state

The pre-WS client comes with a data broker, which only considers data (input files) located at CE targets (either local or cached and targets being GM). Targets are then ranked according to the "effective" size of the requested input files.

In the WS client an attempt at making a data broker, however it only considers A-REX targets, and furthermore only cached data. Additionally source code of that broker requires clean-up, and the method used to fetch cached data information is a hack and instead the GLUE2 model should be used. Due to the modular nature of the libarcclient library the data broker should at least be able to consider data at all supported targets if applicable.

Information schemas

The following information schemas are used by various services, both computing and storage

Glue 1.3

The information schema glue 1.3, contains attributes applicable for storage resources. For data brokering purposes the Info.DefaultSE attribute in the Computing Element entity and the Bind entity (also know as the CE-SE-Bind CESEBind entity) is important.

Nordugrid schema

Glue 2.0

The Glue 2.0 schema, which is the replacement for the 1.3 version, also contains fields useful in respect to data brokering. In the ComputingShare entity the DefaultStorageService attribute exist which specifies the defualt storage element this computing share should use. Also the

Available information




The CREAM CE publish information in the Glue 1.3 format, accessible through BDII using the ldap protocol.


  • It should be confirmed whether or not the Info.DefaultSE attribute

and Bind entity can be set in CREAM.

  • Information about how to contact and communicate with the infoprovider

should also be gathered.