HED

Abstract

This artcile presents HED - set of functional libraries used for development Grid-oriented middlewware called ARC []. The HED name appeared as abbreviation at early development stage and meant Hosting Environment Daemon. Initially HED was represented by the container of the functional components of ARC middleware. Later the scope of HED grew wider and now also covers set of libraries providing core functionality for developing flexible pluggable applications as well as minimal set of core plugins. The whole design of the HED is built around the idea of flexibility and modularity. This is why the HED mostly consists of pluggable modules with some glue among them.

The current set of plugins mostly provides modules for building SOAP based Web Services. But it would be a mistake to see the HED as yet another Web Services development framework. Its purpose is rather to merge Web Services with other functionality necessary for modern Grid middleware in seamless way avoiding and overcoming frequent incompatibility problems araising from usage of multiple third-party implementations of required features. On another hand purpose of HED is not to reimplement features but provide glue for binding existing implementations whenever possible.

Reasons

Modern complex Grid middlewares must cover wide scope of functionalities ranging from SOAP processing to efficient bulk data transfers using broad set of protocols. Wide scope leads to extremely broad complex and sometimes self-contradicting set of external dependencies. In practice that may limit applicability of final product to limited set of operational environments - limited range of supported versions and distributions of operating systems, platforms, etc. The HED tries to fight this issue by communicating external components mostly through pluggable modules hence allowing for flexibility in choice of external dependencies. On expense of provided functionality, of course.

Another positive outcome of using modules wrapping access to external software is that modules of same kind may provide same interface optimized for particular problem hence making it easier to access functionality from application code. In this way HED acts mostly as glue layer for bringing expandable functionality of external components together.

Base libraries

The HED code is written in C++. The choice was mostly influence by te fact that functionality provided by C++ code can be relatively easily interfaced from languages of interest, like Java[], Python[], Perl[], etc. On another hand C++ is can natively use C API provide by low level operating system interfaces like those defined by POSIX[].

Code uses glibmm[] for achieving idependence on execution platform. HED libraries expand capabilities of glibmm to achieve even better portability.

Other commonly used functionalities are provided by widely known libraries like libxml2 and OpenSSL.

Core functionality

Functionlity provided by core libraried of the HED is very limited.

Plugin concept

The approach of HED is to use pluggable modules for implementing various functionality.

Functionality pools

Functionality provided by HED is split into few fields. Those are described below.

1. Low level communication protcols handling
2. High level data management
3. Computational job management (client side)
4. Authentication and authorization

Protocol chains

This part provides low level communication protocols handling. It includes base library providing basic functionality, loading capabilities for corresponding plugins and patterns for building such plugins. Plugins are arranged in processing chains and operate on messages representing useful informational payload and some amount of associated attributes. Messages undergo various transformation while traveling through chains forth and back. Among implemented plugins are those handling TCP, HTTP and SOAP protocols. Implementation provides support per-message and per-session contexts and associated generic and security-specific attributes. The protocol chains may be implemented in two ways - for client and server side of communication. In case of server side chains end with special plugin implementing functionality of attached service.

Figure. Message propagation through protocol chain.

Data management

This part of the HED implements high level data management and provides interface for querying information about data objects - usually files or directories - and transfering their content. Base code provides capabilities for choosing, loading and calling methods provided by proper plugin. The set of implemented plugins provides support for protocols like LDAP, HTTP, LFC, RLS and others. Some of plugins use third party implementations of protocols. And some use protocol chains.

Figure. Data management library and plugins.

Computational task control

There is a set of plugins implementing functionality for managing submission, control and result retrieval of generic purpose computational tasks for various implementations of computational Grids. Those include support for both versions of ARC execution service [], UNICORE [], CREAM [] of gLite and generic BES [] service. There is also set of plugins implementing various brokering algorithms available.

Language bindings

Despite being C++ the HED uses SWIG to provide bindings for other programming languages. Currently there two functional sets of bindings implemented - for Java and Python. Those expose most of functionalities of HED libraries and make it possible to use most of functionality including all available plugins. There is also limited possibility for writing plugins themselves in languages other than C++. Currently it is possible to write services and computational taks brokers in Python.

Performance comparison

Protocol chains

hopi vs apache

Data management

arccp vs wget

Web services (SOAP processing)

echo vs tomcat