This wiki is obsolete, see the NorduGrid web pages for up to date information.
Data brokering
|
|
NOTE: This page is out of date and is kept for purely historical reasons. |
Currently data-driven brokering functionality in ARC is very limited. This page tries to collect information about data brokering in general, and ARC specific work which should be carried out in order to improve data brokering in ARC.
Current state
The pre-WS client comes with a data broker, which only considers data (input files) located at CE targets (either local or cached and targets being GM). Targets are then ranked according to the "effective" size of the requested input files.
In the WS client an attempt at making a data broker, however it only considers A-REX targets, and furthermore only cached data. Additionally source code of that broker requires clean-up, and the method used to fetch cached data information is a hack and instead the GLUE2 model should be used. Due to the modular nature of the libarcclient library the data broker should at least be able to consider data at all supported targets if applicable.
Information schemas
The following information schemas are used by various services, both computing and storage
Glue 1.3
The information schema glue 1.3, contains attributes applicable for storage resources. For data brokering purposes the Info.DefaultSE attribute in the Computing Element entity and the Bind entity (also know as the CE-SE-Bind CESEBind entity) is important.
Nordugrid schema
Glue 2.0
The Glue 2.0 schema, which is the replacement for the 1.3 version, also contains fields useful in respect to data brokering. In the ComputingShare entity the DefaultStorageService attribute exist which specifies the defualt storage element this computing share should use. Also the
Available information
Grid-Manager
A-REX
CREAM
The CREAM CE publish information in the Glue 1.3 format, accessible through BDII using the ldap protocol.
Actions:
- It should be confirmed whether or not the Info.DefaultSE attribute
and Bind entity can be set in CREAM.
- Information about how to contact and communicate with the infoprovider
should also be gathered.
