SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004

Sam Services Experiment Specific Runs Luminosity Streams & Triggers Events (D0) Core Files Processes Cache/Resource Mgt (Stations) Job Handling Job Request (MC) Batch Processing General Support Tables Metadata Relational DB Metadata Query

Metadata Query Language (Dimensions) Is a metadata service. Associates keyword value pairs to their representations on the experiment independent databases. Allows definition of metadata within the query service. The constraints per dimension are needed to discover the relevant physics metadata. Is strongly enhanced by the dynamic parameter definition mechanisms associated with the job request services.

Files Are the heart and soul of Sam, they are the complete file metadata catalog. Also, maintains volume information for enstore (Should move to that specific SE). Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Processes Process metadata stores –application, version –status of file processing (requested, delivered, crashed, ok) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Stations Caching –All locations, sizes, for a file on all hardware –State: locked, available –Algorithm (policy) Resource Management –Admins (control resources) –Cache quota by group –Station rules (cache space, project limits) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Production Job Requests (MC & Batch) Processes monte carlo & farm requests. Proposal to unify submission services. File metadata predefined by request metadata. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

General Support Authentication –Who am I? –What group am I in? –What can I do? –Grid Subjects Fabric Definition –Nodes –Operating systems –Hardware Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Tracks run numbers from online (key to Experiment-specific Online metadata). Maintains association between events and files. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Luminosity Streams & Triggers Two sections in sam dealing with luminosity, and streams & triggers. Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs

Events Tracking of events within a file Metadata for each event Huge volumes of data (D0: 17M evts/wk) Files Job Request Luminosity Stream& Triggers Events Batch Processing Processes Stations General Support Runs