Data Staging/Monitoring

To monitor the states of DTRs, look inside <controldir>/dtrstate.log. To get a summary of states edit and use this python script:

#!/usr/bin/env python

import sys

try:
    f = open('controldir/dtrstate.log')
except IOError:
    print 'Cannot open file controldir/dtrstate.log'
    sys.exit(1)
    
states = {}
for line in f:
    l = line.split()
    state = l[1]
    if state in states:
        states[state] = states[state] + 1
    else:
        states[state] = 1
        
f.close()
for state in states:
    print state, states[state]

These states can also be monitored using Gangliarc. ganglia is in widespread use in most cluster installations and Gangliarc adds ARC-related metrics to ganglia's data feed. The data staging metrics show the number of files in some of the most important data staging states over time as well as the total number of files in the data staging system.