Data Staging Minutes 020610

Went through internal logics of Generator, Processor, Delivery and Scheduler. Following decisions taken:

• Processor and Delivery are kept simple. All wisdom is in Scheduler.

• Processor is separate thread existing along with Scheduler.

• Result of meta-URL resolution in Processor may give meta-URL too. Scheduler will take care of repeating resolution. One Processor handles only one DTR at a time.

• Delivery is separate process started by Scheduler. One per physical location being trensfered.

• One Delivery handles only one physical location (DTR) at a time. Scheduler takes care of switching to another replica in case of failure.

• Scheduler handles only flattened priorities. Grouping is used only as hint. Generator handles job related parameters of DTR - flattens priorities and cancels other DTRs in group if one failed.

• No DTR persistency is needed - having per job persistency as it is now is enough. In case of restart Generator simply recreates DTRs.

• Communication infrastructure is undefined yet.

• Some features as presented on wiki are nice to have but will not happen in first approach.

• We have enough of design - going more into implementation now. David will provide DTR state transition schema. Dmytro starts writing Scheduler stub next week. After stub is ready everybody gets part of work.

Next meeting is going to happen next week same time - June 9, 2010, 12:00 CET