This wiki is obsolete, see the NorduGrid web pages for up to date information.

Documentation/Background

From NorduGrid
Jump to navigationJump to search

ARC documentation is quite extensive, but while some parts of code are covered too well, others are not documented at all. This page contains status overview and some wish lists.

Overview

In order to explain our work to various people, often to non-experts, we need human-readable narrative texts that:

  1. introduce every essential component, outlining its functionality and place in the architecture;
  2. describe important details about every essential component, including the underlying technology, interfaces if any, advantages and disadvantages;
  3. explain deployment and usage modes of each essential component, complete with examples, screenshots, templates - whatever is relevant;
  4. explain interactions between the components, what kinds of Grid infrastructures and facilities can be built of them, what services such facilities can provide, and how they are being set up, operated and utilized.
  5. provide build instructions
  6. provide AAA-related instructions

We must have different documents addressing different readers: end-users, CS students, IT professionals, code developers. A short list will thus include:

  • User manual (AAA requirements, client installation, utilities, use cases, examples, JSDL etc)
  • Administrator manual (build, installation, configuration, AAA requirements utilities, use cases, examples etc)
  • Developer's manual (an introduction to the API doc, e.g. explaining how to write services; description of interfaces)

In addition, it will be very useful to produce dedicated documents (research papers) about every essential component

We need an architecture overview document (research paper), that would say: we have components A,B,C,D. You can build system X from components A and D, system Y - from B,C,D, etc. This can also be a part of the administrator guide.

Existing documents are collected in: 

  http://www.nordugrid.org/manuals.html

Improvement ideas

Daniel's wishlist:

(see also Bug 1939)

  1. http://www.nordugrid.org/middleware/ - Packages change
  2. http://download.nordugrid.org/repos.html - New distributions become available
  3. http://www.nordugrid.org/middleware/build.html - Packages change

These files should be reviewed:

  1. README
  2. INSTALL
  3. doc/installation/*
  4. doc/userguide/userguide.tex

We also have obsolete documents, specifically

  1. doc/cvscheat.html
  2. doc/test/ngtest.txt


  • Globus documentation needs updating
    • Do we need documentation on how to build Globus?
  • Split documentation up into sections

Goal: Less documentation, less overlap of information, easier to keep updated, easier to find relevant information.

I want a structure like: 

components/gridftp
          /grid-manager
          /infosystem
          /arex
  
installation/build/dep/globus
                      /voms
                  /arc
            /packages/arc
  
userguide/sample-config-files
         /userguide

Basically you have 4 usecases for documents:

1) you want to install

-- Installation documents for dependencies and arc, build and packages.

2) you want to configure

-- Sample config files and userguide.

3) you want to use

-- Userguide.

4) you want to understand

-- Specific information about how each component works.

  • Common documentation for ARC0 and ARC1
  • www.nordugrid.org / download.nordugrid.org

Oxana's issues:

  1. What do we do with README files from arc0 and arc1 branches? Do they need to be synchronised? Currently both describe the ARC middleware.
  2. In arc0 branch, there is INSTALL file, which does not even mention arc1 packages
  3. Also in arc0 branch, there is Credits file that might be outdated
  4. License: Paul contains GPL-ed files fsusage.h, fsusage.c and /m4/fsusage.m4
  5. Need a separate repository for documents, independent of software tags and such

Documents collected by Katarina:

  • Release notes
  • CLI manual. Should it also include other instructions like chelonia (? are Chelonia client tools in the CLI package?), JSDL
  • JSDL documentation - instructions how to create a job description.
  • Admin and install manual
  • Documentation - technical documentation and manuals.
    • HED
      • NORDUGRID-TECH-19 (ARCHED_article.pdf) main documentation Contains an outline of the architecture, description of the implemented elements, developer instruction and some configuration info. Aleksandr:All is relevant, up to date but not complete.
      • Doxygen: HED (ARC1-API.pdf), HED Chain Components (RC1-ChainComponents.pdf), HED Services (ARC1-Services.pdf)
      • Python SWIG wrapper documentation for developers who would like to use Python and HED Kazinczy Tamás: at the moment the API is considered final, will make sure to update the documentation.
    • A-REX
      • NORDUGRID-TECH-14 (arex_tech_doc.pdf) Desctription and administration guide for system administrators who want to set up an A-REX service. Contains detailed description of the CE job managements, job cycle, configuration (setup, configuration schema), description of JSDL (includes ARC-specific extensions) (relevant for a user manual?). David: It may be missing information about the new xml configuration style. (Katarina: Yes, it seams only to have a section on arc.conf)
      • ARC Web Services Quick Usage Guide Installation and configuration, A-REX setup, testing and using [ Florido: this document has too many things inside. should be dropped 18:33, 13 May 2011 (CEST) ]
      • NORDUGRID-TECH-18 (Backends-arc1.pdf) Description and and configuration instructions for backend interface for GLUE2 schema. Useful for developers who want to add support for a new batch system into ARC.
    • Security
      • NORDUGRID-TECH-16 (arc-security-documentation.pdf) documantation and developers guide, some configuration instructions. Aleksandr: Has been updated. Currently we have following issues with that document:
        • 3 pictures are missing
        • 2 pictures need to be regenrated to use vector graphics
        • Most pictures need to be renegated to use better fonts
        • Text starting from "PDP service invoker configuration" and till end of document needs heavy cleaning and proof-reading.
    • ISIS/Infosys
    • Storage
      • NORDUGRID-MANUAL-10 (arc-storage-manual.pdf) Administrators manual. Contains a detailed building and installation instruction for Ubuntu (or similar). Prototype client tool and FUSE instruction.
      • NORDUGRID-TECH-17 (arc-storage-documentation.pdf) Documentation and developers guide.
      • Would be nice to have a user guide. Maybe some parts of the manual could be recycled.
        • Zsombor will put together a user manual for the chelonia interfaces CLI, FUSE module and the ARC DMC.
    • Jura
      • NORDUGRID-TECH-24 (jura-tech-doc.pdf) Technical description, configuration. Peter Dobe: The document is up-to-date. See revision 12970 (broken references fixed).
    • Janitor
      • Janitor.pdf installation and usage documentation. Under development. Steffen: It is being worked on and the plan is to finish it for the final release.
    • ECHO
      • Documentation in sys. admin guide, in preparation
    • Web service developers guide KnowARC Wiki: File:WS-tutorial.pdf tutorials in svn.
    • ARCLIB
      • NORDUGRID-TECH-20 (client_technical.pdf) Technical description, description of functionality and implementation. Seems to be aimed at developers adding plugins for support of other middlewares.
      • Application developer guide like the NORDUGRID-TECH-12 describing both the C++ classes and python binding would be very useful. The NORDUGRID-TECH-15 (WS-Based ARC Clients) contains a little bit of it, but it is outdated as it refers to the prototype.
        • Bjarte has a libarcclient developer guide on the to-do list
      • NORDUGRID-TECH-22 gLite gateway plugin for ARCLIB Technical description and user manual Peter Stefan: Conceptually it is still relevant, however the glite* commands have been integrated into arclib and normal arc* commands used for accessing glite sites. Katarina: The link is kept to have an overview.
    • Client
      • User interface it will for sure become a standard reference for all users. Oxana: Only a placeholder, on her to-do-list
      • KnowARC Wiki: Python client examples. Maybe this could be included in some manual and at the same time made available in the online public documentation. This could be moved to ARCLIB documentation.
      • KnowARC Wiki: list of all commands, but no explanation
      • Job description job_description.pdf and an overview of mapping between different middlewares job_description_mapping.odt It would be useful for a user to have an overview of the possible JSDL elements something like the xrsl reference manual NORDUGRID-MANUAL-4. The A-REX documentation contains a list NORDUGRID-TECH-14 it has also a useful description of a job cycle. Martin Skou Andersen: will develop the documents into documentation notes, possibly with examples. HTML exists ExecutionTargetMapping.html
    • Client for other platforms
  • Configuration documentation
Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=Documentation/Background&oldid=4149"