Minutes of the ARC1 Information System implementation F2F meeting

November 22

First day is for local components - those in services.

High Level Components

Information Indexing Sytem is considered by some people as bad name. Oxana proposes ISIS (Information Storage and Indexing Service). Meaning of name is not good. We use ISIS and longer name will be decide later. Oxana volunteers for finding good name.

ISIS stores and indexes Registration Entries(RE). Cloud of ISISes will form information system. This topic is for tomorrow. System is open for registrants and there is a way for partitioning information to form "VO view".

REs come from local information source (topic for today). Name for this component(s) is ALIS (A Local Information System). ALIS covers everything from LIDI to information backends.

ALIS is made of:

Security
Query
Registration
Information Collection
Caching
Local Interface
ALIS deals with Full Service Description (FSD)

Clients issue queries to ISIS and ALIS which are executed on services and return results.

Deployment

Every box and service comes with ALIS. Some functionality is per service and some other is per HED (daemon, box).

We need high responsiveness of ALIS (and ISIS). It is not clear how to measure responsiveness. Seconds are not suitable - response time depends on amount of information. Balazs suggests to measure time per number of entries in information system. Like 10 seconds for 1000 entries 100 attributes each should be enough for XPath query for 1 attribute from every entry. We are aiming for this number. Linaer scalability should be enough.

Scalability of resource discovery (client queries ISIS and then ALIS for suitable resource) - for tomorrow.

Security - to be more exact Access Control (Authorization). All operations will go through Access Control. Balazs says it must be implemented. Question is how granulary can we do that. Per information object? Balazs claims it is importatn to come with examples how cuurent security system can be used to express access control scenarios. Ferenc suggests Balazs write scenarios to Weizhong and ask for examples. Balazs has no scenarios. Oxana will invent 12/2 examples in plain english. Ferenc requires them to have reasons. We need authorization at data level at least in A-Rex.

Backend is bad name. We are using External Service Model. Oxana does not like word External. Proposed replacemets - Helper, Assistant, Executioner.

LIDI

Ferenc suggests to have LIDI as separate MCC and put it after Plexer before Service. LIDI will get information to serve from information cache. Another approach is for services to implement LIDI and to use Informatin Cache to store information.

Balazs wants some safety meassures against misbehaving services. It's not clear what is misbehaving?

Both solutions most probably can coexist. There are different possibilities how to configure LIDI MCC - either through availability of information in cache or directly in configuration. Solution - we will have LIDI MCC (called InfoMCC) which filters and processes information queries. We will have LIDI processing class in HED library (currently called InformationInterface) to be used by services directly.

Name for cache

Info(rmation) Cache. Provided by HED library.

Functionality - Set and Get information. It stores information and processes queries.

Registration

Problem: it is not clear which information should come from where. Either everyting comes from cache or partly from configuration.

Terms:

Registration Entry - information sent to ISIS
RE is made of Service Advertisement and Service Advertisement Metadata
SA is a subset of FSD
SAM is an additional information which defines how SA should be handled and presented.

Selection of attributes for SA:

Mandatory attributes are fixed in schema.

Oxana wants defintion of maximum attributes. It is known how to define that. Ferenc claims service developers are good and will take care of that.

How to choose attributes for SA? Will services provide filter for that? Will attributes be marked in cache? Will there be 2 entries for each service - one for FSD and another for SA?

Ferenc suggests to at first step to have registration in every service and then to switch to centralized registration process, sililar to LIDI.

Security constraints:

Do we identify services or hosts for registration to ISIS? We service certificates but we identify services by registered endpoints. Otherwise authorization is done in same way as defined previously. Registration may be done to multiple targets (ISIS) and each target may have different SAM.

Actions:

Define SA schema - based on Glue2. Johan will write description of registrant component.

Information collector

Ferenc suggests to make a pluggable interface to call information collectors instead of using Run class. Suggestion accepted and assigned to Ferenc.

Does Run class satisfy requirements put on it by backends/interface to LRMS? Ferenc suggests to start from test code - to extend run test currently in svn.

How to deal with plugins generating subtree (or multiple subtrees)? Discussion not finished yet.

How it fits into LRMS backends architecture? It goes almost inside information collectors plugins.

November 23

WSRF/WSRP support

Support for WSRP queries, other operations not supported. Faults are not properly used. Aleksandr needs to fix that. WSRT will need to be implemented in year from now.

Unicore uses WSRF - Gabor will contact Unicore to check their client against our WSRP enabled service.

Our WSRP client apinfo needs to be extended to support at least XPath queries. We need that to test the information cache.

FSD

In a week we can have a skeleton for SA for generic service, at least for A-Rex.

LIDI interface

Cache structure should be designed to satisfy INFO(LIDI) MCC. INFO MCC is not being designed till we have few services.

Information System Technical Document

Editor: Balazs
Parts: General overview, ALIS, ISIS
Assigned people:
- General oveview, ALIS (main part) - Balazs
- ALIS subparts:
  - Registrant - Johan
  - Information Collector - Ferenc
  - Security - Ferenc - postponed, will be described later.
  - LIDI - Aleksandr
  - INFO MCC -
  - Information Cache -

Information Cache

Ferenc presented possible infomation cache API.

Cache will store XML document per service.

The problem is how to get service id(entifier) which is uused to store service information in cache.

Ferenc concerned about performace of parsing big XML documents. It would be better to have it split into spearate files/chunks/... and do some preselection on them.

Possible solutions:

Treat separate components of FSD as subservices and address them through extended URL.
Add information to WS-Addressing header to preselect parts of FSD.

So far we choose using extended URLs.

Structure of cache

/service description - all non-activities
/activities
/activities/activity
 
 

<...>
<...> 

<...>
<...>

Cache API will allow to set and get parts of cache based on service id and path to cache entry. Ferenc will implement cache API.

Cache implmentation will not depend on VO. VO is part of schema.

Integration of batch system scripts with the HED.

See drawings.

BES vs WSRF

The recomended way of getting information for internal clients is through LIDI.

Security

All queries to be authenticatable. We require non-anonymous queries. We define anonymous clients which come without acceptable X509 authentication. Ferenc insists that this should be "recomended" way.

BES lacks possibility to manipulate access control of jobs - requires extension

Lowest granularity of access control we are going to provide is per XML element. ...

Deplyment-wise we will have authorization attached to selected elements of FSDi like activities.

Primary interface for reading, writing and modifying access control is through service specific interface. ACLs are still accessible through Information Interface.

It was impossible to agree on anything else. Different participants expressed different opinions. Also we failed to define how general system is going to be.

ISIS

Operations:

Register - a bit unsync, needs little fixing. Service ID is not same as service endpoint.
RemoveRegistration - should be removed by ID, not by EPR. Output should not be empty.
GetRegistrationStatuses - Statuses not defined - further work is needed. Schema requires cleaning. Balazs will do that.
GetIISList - backup solution if client does not trust results of other operations.

Registrations are obtained through LIDI. Ferenc wants separate operation for getting registraions. Agreed to have Get operation to retrieve all or specified registrations. Ferenc takes responsibility to implement ISIS by end of November - ISIS and LIDI interfaces, information stored in file.

general

Balazs creates place in subversion for Nordugrid techical document for InfoSystem. Thomas will upload his slides there.

ARC1/Infosys/LundF2F

Contents