This wiki is obsolete, see the NorduGrid web pages for up to date information.

ARC server-side logging

From NorduGrid
Jump to navigationJump to search

This page is meant to evolve into a set of guidelines for writing logs that are useful for sysadmins as well as developers. For now it is just a place to collect the current state of logs in ARC and come with suggestions on how to improve the logs.

Overview of current server side logs in ARC


Log usefulness
log name sensibly formatted has useful information Is logging sufficient at default level? (%) Is loging lacking at default level? (%) Comments
cache-clean.log new default name|
grid-manager.log yes yes
job.helper.errors no yes Bugzilla #3486
Full overview
Is OK arc.conf.reference name Conf block Conf name Log level conf name Log levels Default location Default log level ARC Logger long format Has system log rotation Log format timestamp format Source
cache-clean.log grid-manager cachelogfile cacheloglevel 0-5 /var/log/arc/cache-clean.log 3 Y [time] who: level: msg YYYY-MM-DD hh:mm:ss cashe-clean
Y grid-manager.log grid-manager logfile debug 0-5 /var/log/arc/grid-manager.log 2 Y Y [time][who][level][pid/thread]msg YYYY-MM-DD hh:mm:ss arched
Y ws-interface.log grid-manager wslogfile debug 0-5 /var/log/arc/ws-interface.log 2 Y Y [time][who][level][pid/thread]msg YYYY-MM-DD hh:mm:ss arched
gm-jobs.log grid-manager joblog ? ? no ? time msg YYYY-MM-DD hh:mm:ss arched
jura.log grid-manager jobreport_logfile ? ? controldir/job.logger.errors ? level: msg YYYY-MM-DD hh:mm:ss jura
data-staging debug 0-5 controldir/job.[id].errors 2 [time][level][pid/thread] who msg YYYY-MM-DD hh:mm:ss backends, data, arched
dtrstate.log data-staging dtrlog ? ? controldir/dtrstate.log - msg -
gridftpd.log gridftpd logfile debug 0-5 /var/log/arc/gridftpd.log 3 Y Y [time][who][level][pid/thread]msg YYYY-MM-DD hh:mm:ss
infosys ? debug 0-1 ? 0?
infosys ? slapd_loglevel 0-? ? ?
inforegistration.log infosys registrationlog ? ? /var/log/arc/inforegistration.log ? Y ? Thu Mar 19 11:49:24 CET 2015
infoprovider.log infosys providerlog provider_loglevel 0-5 /var/log/arc/infoprovider.log 1? Y [time] who: level: msg YYYY-MM-DD hh:mm:ss
bdii-update.log infosys bdii_log_dir bdii_debug_level ERROR | WARNING | INFO | DEBUG /var/log/arc/bdii/bdii-update.log INFO, but InfosysHelper hardcoded to VERBOSE Y [time] who: level: msg YYYY-MM-DD hh:mm:ss bdi and infosyshelper
nordugridmap.log nordugridmap logfile debug 0-5 /var/log/arc/nordugridmap.log 2? msg -
acix-cache.log acix/cacheserver logfile ? ? /var/log/arc/acix-cache.log ? time [what] msg YYYY-MM-DD hh:mm:ss+tz00
gangliarc logfile? - 0-5 /var/log/arc/gangliarc.log 3 time [level] msg YYYY-MM-DD hh:mm:ss,mmm
controldir/job.helper.errors ? [time] who: msg YYYY-MM-DD hh:mm:ss

Changes to be made

  • one logfile per configuration block, config parameter name "logfile": this will require restructuring arc.conf (e.g. new blocks, subblocks for jura, cache)
  • WHO from the grid-manager logs should be documented on a wiki page
  • unify jobid-s in the grid-manager.log, ws-interface.log
  • change to
  • rename dtrstate.log dtr.state, conf parameter statefile instead of dtrlog
  • infosys/debug: remove
  • inforegistration.log: will dissappear together with the registration functionality, no need to fix this log
  • nordugridmap: sync with the agreed format, to be enabled by default
  • job.logger.errors is the jobreport_logfile default location in controldir: move to var/log/arc
  • job.helper.errors: move to var/log/arc, own confiuration parameter helperlog in gridmanager block

TODO: ARC_server-side_logging-TODO

Log levels

Ideally all log messages in ARC should fall into one of the following categories (in decreasing rank):

  • DEBUG: Detailed information, typically of interest only when diagnosing problems.
  • VERBOSE: Like DEBUG, but less verbose? (Do we need this level?)
  • INFO: Confirmation that things are working as expected.
  • WARNING: An indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected.
  • ERROR: Due to a more serious problem, the software has not been able to perform some function.
  • FATAL: A serious error, indicating that the program itself will be unable to continue running.

To configure the level of logging for a logfile either a number between 0 (FATAL) and 5 (DEBUG) or the name of the lowest category should be used.

Log configuration

All logs should be configurable and have a default value. All logs should have a configurable log level.

Log format

All log messages should at least have a timestamp. The preferred log format is the ARC::LongFormat



[2015-06-01 03:26:03] [Arc.JobDescriptionParserPlugin] [INFO] [25456/3] String successfully parsed as nordugrid:xrsl

Log rotation

All logs will grow in time. Sysadmins with disks running full because of big logs are not happy sysadmins. All ARC logs should have corresponding system logrotate rules by default.