This wiki is obsolete, see the NorduGrid web pages for up to date information.
ARC server-side logging
From NorduGrid
Jump to navigationJump to search
This page is meant to evolve into a set of guidelines for writing logs that are useful for sysadmins as well as developers. For now it is just a place to collect the current state of logs in ARC and come with suggestions on how to improve the logs.
Overview of current server side logs in ARC
Tables
| log name | sensibly formatted | has useful information | Is logging sufficient at default level? (%) | Is loging lacking at default level? (%) | Comments |
| cache-clean.log | new default name| | ||||
| grid-manager.log | yes | yes | |||
| ws-interface.log | |||||
| gm-jobs.log | |||||
| jura.log | |||||
| job.[id].errors | |||||
| gridftpd.log | |||||
| inforegistration.log | |||||
| infoprovider.log | |||||
| bdii-update.log | |||||
| nordugridmap.log | |||||
| acix-cache.log | |||||
| gangliarc.log | |||||
| job.logger.errors | |||||
| job.helper.errors | no | yes | Bugzilla #3486 |
| Is OK | arc.conf.reference name | Conf block | Conf name | Log level conf name | Log levels | Default location | Default log level | ARC Logger long format | Has system log rotation | Log format | timestamp format | Source | |
| cache-clean.log | grid-manager | cachelogfile | cacheloglevel | 0-5 | /var/log/arc/cache-clean.log | 3 | Y | [time] who: level: msg | YYYY-MM-DD hh:mm:ss | cashe-clean | |||
| Y | grid-manager.log | grid-manager | logfile | debug | 0-5 | /var/log/arc/grid-manager.log | 2 | Y | Y | [time][who][level][pid/thread]msg | YYYY-MM-DD hh:mm:ss | arched | |
| Y | ws-interface.log | grid-manager | wslogfile | debug | 0-5 | /var/log/arc/ws-interface.log | 2 | Y | Y | [time][who][level][pid/thread]msg | YYYY-MM-DD hh:mm:ss | arched | |
| gm-jobs.log | grid-manager | joblog | ? | ? | no | ? | time msg | YYYY-MM-DD hh:mm:ss | arched | ||||
| jura.log | grid-manager | jobreport_logfile | ? | ? | controldir/job.logger.errors | ? | level: msg | YYYY-MM-DD hh:mm:ss | jura | ||||
| data-staging | debug | 0-5 | controldir/job.[id].errors | 2 | [time][level][pid/thread] who msg | YYYY-MM-DD hh:mm:ss | backends, data, arched | ||||||
| dtrstate.log | data-staging | dtrlog | ? | ? | controldir/dtrstate.log | - | msg | - | |||||
| gridftpd.log | gridftpd | logfile | debug | 0-5 | /var/log/arc/gridftpd.log | 3 | Y | Y | [time][who][level][pid/thread]msg | YYYY-MM-DD hh:mm:ss | |||
| infosys | ? | debug | 0-1 | ? | 0? | ||||||||
| infosys | ? | slapd_loglevel | 0-? | ? | ? | ||||||||
| inforegistration.log | infosys | registrationlog | ? | ? | /var/log/arc/inforegistration.log | ? | Y | ? | Thu Mar 19 11:49:24 CET 2015 | ||||
| infoprovider.log | infosys | providerlog | provider_loglevel | 0-5 | /var/log/arc/infoprovider.log | 1? | Y | [time] who: level: msg | YYYY-MM-DD hh:mm:ss | ||||
| bdii-update.log | infosys | bdii_log_dir | bdii_debug_level | ERROR | WARNING | INFO | DEBUG | /var/log/arc/bdii/bdii-update.log | INFO, but InfosysHelper hardcoded to VERBOSE | Y | [time] who: level: msg | YYYY-MM-DD hh:mm:ss | bdi and infosyshelper | |||
| nordugridmap.log | nordugridmap | logfile | debug | 0-5 | /var/log/arc/nordugridmap.log | 2? | msg | - | |||||
| acix-cache.log | acix/cacheserver | logfile | ? | ? | /var/log/arc/acix-cache.log | ? | time [what] msg | YYYY-MM-DD hh:mm:ss+tz00 | |||||
| gangliarc | logfile? | - | 0-5 | /var/log/arc/gangliarc.log | 3 | time [level] msg | YYYY-MM-DD hh:mm:ss,mmm | ||||||
| controldir/job.helper.errors | ? | [time] who: msg | YYYY-MM-DD hh:mm:ss |
Changes to be made
- one logfile per configuration block, config parameter name "logfile": this will require restructuring arc.conf (e.g. new blocks, subblocks for jura, cache)
- WHO from the grid-manager logs should be documented on a wiki page
- unify jobid-s in the grid-manager.log, ws-interface.log
- change job.id.errors to job.id.log
- rename dtrstate.log dtr.state, conf parameter statefile instead of dtrlog
- infosys/debug: remove
- inforegistration.log: will dissappear together with the registration functionality, no need to fix this log
- nordugridmap: sync with the agreed format, to be enabled by default
- job.logger.errors is the jobreport_logfile default location in controldir: move to var/log/arc
- job.helper.errors: move to var/log/arc, own confiuration parameter helperlog in gridmanager block
TODO: ARC_server-side_logging-TODO
Log levels
Ideally all log messages in ARC should fall into one of the following categories (in decreasing rank):
- DEBUG: Detailed information, typically of interest only when diagnosing problems.
- VERBOSE: Like DEBUG, but less verbose? (Do we need this level?)
- INFO: Confirmation that things are working as expected.
- WARNING: An indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected.
- ERROR: Due to a more serious problem, the software has not been able to perform some function.
- FATAL: A serious error, indicating that the program itself will be unable to continue running.
To configure the level of logging for a logfile either a number between 0 (FATAL) and 5 (DEBUG) or the name of the lowest category should be used.
Log configuration
All logs should be configurable and have a default value. All logs should have a configurable log level.
Log format
All log messages should at least have a timestamp. The preferred log format is the ARC::LongFormat
[time][who][level][pid/thread]msg
Example:
[2015-06-01 03:26:03] [Arc.JobDescriptionParserPlugin] [INFO] [25456/3] String successfully parsed as nordugrid:xrsl
Log rotation
All logs will grow in time. Sysadmins with disks running full because of big logs are not happy sysadmins. All ARC logs should have corresponding system logrotate rules by default.
