Presentation is loading. Please wait.

Presentation is loading. Please wait.

SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004.

Similar presentations


Presentation on theme: "SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004."— Presentation transcript:

1 SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004

2 Sam Services Experiment Specific Runs Luminosity Streams & Triggers Events (D0) Core Files Processes Cache/Resource Mgt (Stations) Job Handling Job Request (MC) Batch Processing General Support Tables Metadata Relational DB Metadata Query

3 Metadata Query Language (Dimensions) Is a metadata service. Associates keyword value pairs to their representations on the experiment independent databases. Allows definition of metadata within the query service. The constraints per dimension are needed to discover the relevant physics metadata. Is strongly enhanced by the dynamic parameter definition mechanisms associated with the job request services.

4 Files Are the heart and soul of Sam, they are the complete file metadata catalog. Also, maintains volume information for enstore (Should move to that specific SE). Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

5 Processes Process metadata stores –application, version –status of file processing (requested, delivered, crashed, ok) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

6 Stations Caching –All locations, sizes, for a file on all hardware –State: locked, available –Algorithm (policy) Resource Management –Admins (control resources) –Cache quota by group –Station rules (cache space, project limits) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

7 Production Job Requests (MC & Batch) Processes monte carlo & farm requests. Proposal to unify submission services. File metadata predefined by request metadata. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

8 General Support Authentication –Who am I? –What group am I in? –What can I do? –Grid Subjects Fabric Definition –Nodes –Operating systems –Hardware Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

9 Tracks run numbers from online (key to Experiment-specific Online metadata). Maintains association between events and files. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

10 Luminosity Streams & Triggers Two sections in sam dealing with luminosity, and streams & triggers. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

11 Events Tracking of events within a file Metadata for each event Huge volumes of data (D0: 17M evts/wk) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

12


Download ppt "SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004."

Similar presentations


Ads by Google