OO Software and Data Handling in AMS Computing in High Energy and Nuclear Physics Beijing, September 3-7, 2001 Vitali Choutko, Alexei Klimentov MIT, ETHZ

A.Klimentov AMS software and data handling CHEP012 Outline AMS – particle physics experiment on the international space station : Data flow and AMS ground centers Software development Conditions and Tag Database Data Processing AMS Detector STS91 precursor flight AMS ISS mission

A.Klimentov AMS software and data handling CHEP013 AMS : a particle physics experiment in space PHYSICS GOALS : Accurate, high statistics measurements of charged, cosmic ray spectra in space > 0.1GV Nuclei and e- spectra measurement The study of dark matter (90% ?) Determination of the existence or absence of antimatter in the Universe Look for negative nuclei The study of the origin and composition of cosmic rays Measure isotopes D, He, Li, Be… +

A.Klimentov AMS software and data handling CHEP014

A.Klimentov AMS software and data handling CHEP015 Magnet : Nd 2 Fe 14 B TOF : trigger, velocity and Z Si Tracker : charge sign, rigidity, Z Aerogel Threshold Cerenkov : velocity Anticounters : reject multi particle events Results :  Anti-matter search : He / He = 1.1x 10  Charged Cosmic Ray spectra Pr, D, e-, He, N  Geomagnetic effects on CR under/over geomagnetic cutoff components 10 events recorded Trigger rates 0.1-1kHz DAQ lifetime 90% 8 -6 _ Precursor flight : +

A.Klimentov AMS software and data handling CHEP016

A.Klimentov AMS software and data handling CHEP017 AMS on ISS, 3 years in space Separate e- from p,p + _ up to 300 GeV He, He, B, C… 3 4 e-,   up to 1000 GeV +

A.Klimentov AMS software and data handling CHEP018

9 Monitoring & science data Stored data Real-time Data H&S Real-time & “Dump” data Real-time, “Dump”, & White Sand’s LOR playback AMS ACOP High Rate Frame MUX White Sand, NM facility MSFC, Al Payload Data Service system Telescience centers External Communications GSE Long Term Short Term Payload Operations Control Center Science Operations Center ISS to Remote AMS Centers Data Flow ISS NASA Ground Infrastructure Remote AMS Sites H&S Monitoring Science Flight ancillary data Real-time & “dump” NearReal-time File transfer playback

A.KlimentovCHEP0110 AMS Ground Centers Science Operations Center POCC AL AMS Remote center RT data Commanding Monitoring NRT Analysis NRT Data Processing Primary storage Archiving Distribution Science Analysis MC production Data mirror archiving External Communications Science Operations Center XTerm HOSC Web Server and xterm TReK WS commands Monitoring, H&S data Flight Ancillary data AMS science data (selected) TReK WS “ voice”loop Video distribution Production Farm Analysis Facilities PC Farm Data Server Analysis Facilities GSE D S A e T r A v e r GSE Buffer data Retransmit To SOC AMS Station AMS Station AMS Station GSE MC production cmds archive AMS Data, NASA data, metadata

A.Klimentov AMS software and data handling CHEP0111 AMS SW development Been started mid 1996 : basic decisions : new code C++ only (though we had a large part of legacy SW written on Fortran) Existing libraries (CERNLIB, Geant, etc) incorporated via C/Fortran interface (R.Burow) transient and persistent classes are separated with implementing of copy member functions Decide to use Root and HBOOK for histogramming and data visualization

A.Klimentov AMS software and data handling CHEP0112 AMS SW development (cont’d) Use different persistency solutions for various type of data :  Flat files for the raw data  Ntuples and Root files for ESD  Relational Database (Oracle) tables for file catalogues  Relational Database (Oracle) [Objectivity up to Sep 1998] o Event Tags o Calibration data o Slow control data o NASA ancillary data o Various catalogues (processing history, etc )

A.Klimentov AMS software and data handling CHEP0113 Tag Storage with Oracle RDBMS Tag is an unsigned 32 bit integer containing 16, 1 to 5 bit long parameters such as charge, momentum sign, ß,… Model : Query : retrieve tags with 3 parameters satisfied to the given limits (query taken from the “real analysis chain”) Data stored on Raid array connected to AS4100 (quad-CPU rated at 600MHz, 2GB RAM) Flat files – 2400 files, one file per DAQ run, tags are stored as an array of unsigned int. RootN - 10 files, each file with ~240 trees, one tree per DAQ run with single branch (tag) per tree RootS - 10 files, each file with ~240 trees, one tree per DAQ run, having 16 branches, every parameter stored in a dedicated branch OracleN - table with 10 partitions and 1 column, mapping tag to a column OracleI - table with 10 partitions and 1 column with 16 bitmap indices, mapping tag to a column OracleS – table with 10 partitions and 16 columns, every parameter mapping to a column

A.Klimentov AMS software and data handling CHEP0114 Oracle RDBMS to store AMS tags Method Size Gbyte Query Time (sec) Total write time (sec) Record write time (  sec) Flat Files RootN RootS OracleN OracleS OracleI ) 1) 500 sec to build indices for 100M tags

A.Klimentov AMS software and data handling CHEP0115 Design of the Conditions Database  Collection of Time Dependent Values (TDVs)  Primary access keys : name, id, validity interval  Secondary key : insert time  Major Components : table of names and ids, default TDVs, TDVs  Applications : Loading data into database Fetching conditions during event reconstruction Management utilities (TDV browser) Name, id Validity begin, validity end time Insert time Array of unsigned integers (size 100 byte – 8 Mbyte)

A.Klimentov AMS software and data handling CHEP0116 AMS Conditions Database Initially Objectivity, then flat files, now Oracle Performance test for TDV Total Records Flat file size (Mbyte) Oracle table size (Mbyte) Record write time (msec) TOF Temperature Tracker pedestals (a) Tracker pedestals (b) (a) BLOB array is stored inside the table, (b) - outside  TOF temperature (many short records)  Tracker pedestals (small amount of large records)

A.Klimentov AMS software and data handling CHEP0117 Oracle RDBMS to store Tags and TDVs Currently 8 Gbyte is stored in the Conditions DB (115 different TDV types) 100 million event tags are stored in Tag DB Oracle RDBMS performance and functionality satisfy AMS requirements. Using of bitmap indices for tags improves query time dramatically. The current implementation works with distributed CORBA technology. It allowes to reduce the number of database clients and machine loading.

A.Klimentov AMS software and data handling CHEP0118 producers Raw data server Oracle RDBMS Conditions DB Tag DB Active Tables : Hosts, Interfaces, Producers, Servers Catalogues server Nominal Tables Hosts, Interfaces Producers, Servers… ESD server Raw data {I} {II} {III} {IV} {V} {VI} {I} submit 1 st server {II} “cold” start {III} read “active” tables (available hosts, number of servers, producers, jobs/host) {IV} submit servers {V} get “run”info (runs to be processed, ESD output path) {VI} submit producers (LILO, LIRO,RIRO…) Notify servers AMS Production

A.Klimentov AMS software and data handling CHEP0119 AMS Production Highlights Stable running for more than 1 month Average efficiency 95% (98% without Oracle) Processes communication and control via Corba LSF for process submission Run Oracle server on AS4100 Alpha and Oracle clients on Linux. Oracle RDBMS  Tag DB with 100M entries  Conditions DB with 100K entries  Bookkeeping  Production status  Runs history  File catalogues