Data Grids for Next Generation Experiments Harvey B Newman California Institute of Technology ACAT2000 Fermilab, October 19,

Physics and Technical Goals u The extraction of small or subtle new “discovery” signals from large and potentially overwhelming backgrounds; or “precision” analysis of large samples u Providing rapid access to event samples and subsets from massive data stores, from ~300 Terabytes in 2001 Petabytes by ~2003, ~10 Petabytes by 2006, to ~100 Petabytes by ~2010. u Providing analyzed results with rapid turnaround, by coordinating and managing the LIMITED computing, data handling and network resources effectively u Enabling rapid access to the data and the collaboration, across an ensemble of networks of varying capability, using heterogeneous resources.

Four LHC Experiments: The Petabyte to Exabyte Challenge ATLAS, CMS, ALICE, LHCB Higgs + New particles; Quark-Gluon Plasma; CP Violation Data written to tape ~25 Petabytes/Year and UP (CPU: 6 MSi95 and UP) Data written to tape ~25 Petabytes/Year and UP (CPU: 6 MSi95 and UP) 0.1 to 1 Exabyte (1 EB = Bytes) (~2010) (~2020 ?) Total for the LHC Experiments 0.1 to 1 Exabyte (1 EB = Bytes) (~2010) (~2020 ?) Total for the LHC Experiments

LHC Vision: Data Grid Hierarchy Tier 1 Tier2 Center Online System Offline Farm, CERN Computer Ctr > 30 TIPS FranceCenter FNAL Center Italy Center UK Center Institute Institute ~0.25TIPS Workstations ~100 MBytes/sec ~2.5 Gbits/sec Mbits/sec 1 Bunch crossing; ~17 interactions per 25 nsecs; 100 triggers per second. Event is ~1 MByte in size Physicists work on analysis “channels” Each institute has ~10 physicists working on one or more channels Physics data cache ~PBytes/sec ~ Gbits/sec Tier2 Center ~622 Mbits/sec Tier 0 +1 Tier 3 Tier 4 Tier2 Center Tier 2 Experiment

Why Worldwide Computing? Regional Center Concept: Advantages u Managed, fair-shared access for Physicists everywhere u Maximize total funding resources while meeting the total computing and data handling needs u Balance between proximity of datasets to appropriate resources, and to the users è Tier-N Model u Efficient use of network: higher throughput è Per Flow: Local > regional > national > international u Utilizing all intellectual resources, in several time zones è CERN, national labs, universities, remote sites è Involving physicists and students at their home institutions u Greater flexibility to pursue different physics interests, priorities, and resource allocation strategies by region è And/or by Common Interests (physics topics, subdetectors,…) u Manage the System’s Complexity è Partitioning facility tasks, to manage and focus resources

SDSS Data Grid (In GriPhyN): A Shared Vision Three main functions: Raw data processing on a Grid (FNAL) Raw data processing on a Grid (FNAL) è Rapid turnaround with TBs of data è Accessible storage of all image data Fast science analysis environment (JHU) Fast science analysis environment (JHU) è Combined data access + analysis of calibrated data è Distributed I/O layer and processing layer; shared by whole collaboration Public data access Public data access è SDSS data browsing for astronomers, and students è Complex query engine for the public

US-CERN BW Requirements Projection (PRELIMINARY) [#] Includes ~1.5 Gbps Each for ATLAS and CMS, Plus Babar, Run2 and Other [*] D0 and CDF at Run2: Needs Presumed to Be to be Comparable to BaBar

Daily, Weekly, Monthly and Yearly Statistics on the 45 Mbps US-CERN Link

Regional Center Architecture (I. Gaines) Tapes Network from CERN Network from Tier 2 centers Tape Mass Storage & Disk Servers Database Servers Physics Software Development R&D Systems and Testbeds Info servers Code servers Web Servers Telepresence Servers Training Consulting Help Desk Production Reconstruction Raw/Sim  ESD Scheduled, predictable experiment/ physics groups Production Analysis ESD  AOD AOD  DPD Scheduled Physics groups Individual Analysis AOD  DPD and plots Chaotic Physicists Desktops Tier 2 Local institutes CERN Tapes Support Services

[*] See MONARC Architectures WG: Regional Centre Services Required MONARC Architectures WG: Regional Centre Services Required All data and technical services required to do physics analysis [*] All data and technical services required to do physics analysis [*] è All Physics Objects, Tags and Calibration data è Significant fraction of raw data è Excellent network connectivity to CERN and the region’s users è A fair share of post- and re-reconstruction processing è Manpower to share in the development of common validation and production software è Manpower to share in ongoing work on Common (Grid and Other) R&D Projects è Excellent support services for training, documentation, troubleshooting at the Centre or remote sites served by it è Service to members of other regions Long Term Commitment: staffing, hardware evolution, support Long Term Commitment: staffing, hardware evolution, support

LHC Tier 2 Center In 2001 OC-12 Tier2 Prototype (CMS) uDistributed: Caltech/UCSD, Over CALREN (+NTON) u UC Davis, Riverside, UCLA Clients uUniversity (UC) Fund Sharing u2 X 40 Dual Nodes: 160 CPUs Rackmounted 2U u ~2 TB RAID Array uMulti-Scheduler uGDMP Testbed uStartup By End October 2000 (CMS HLT Production)

u RD45, GIODNetworked Object Databases u Clipper/GC High speed access to Objects or File data FNAL/SAM for processing and analysis u SLAC/OOFS Distributed File System + Objectivity Interface u NILE, Condor:Fault Tolerant Distributed Computing u MONARCLHC Computing Models: Architecture, Simulation, Strategy, Politics u ALDAPOO Database Structures & Access Methods for Astrophysics and HENP Data u PPDGFirst Distributed Data Services and Data Grid System Prototype u GriPhyN Production-Scale Data Grids u EU Data Grid Roles of Projects for HENP Distributed Analysis

Grid Services Architecture [*] GridFabric GridServices ApplnToolkits Applns Data stores, networks, computers, display devices,… ; associated local services Protocols, authentication, policy, resource management, instrumentation, discovery,etc.... RemoteviztoolkitRemotecomp.toolkitRemotedatatoolkitRemotesensorstoolkitRemotecollab.toolkit A Rich Set of HEP Data-Analysis Related Applications [*] Adapted from Ian Foster: there are computing grids, access (collaborative) grids, data grids,...

The Particle Physics Data Grid (PPDG) u First Round Goal: Optimized cached read access to Gbytes drawn from a total data set of 0.1 to ~1 Petabyte PRIMARY SITE Data Acquisition, CPU, Disk, Tape Robot SECONDARY SITE CPU, Disk, Tape Robot Site to Site Data Replication Service 100 Mbytes/sec ANL, BNL, Caltech, FNAL, JLAB, LBNL, SDSC, SLAC, U.Wisc/CS Multi-Site Cached File Access Service University CPU, Disk, Users PRIMARY SITE DAQ, Tape, CPU, Disk, Robot Satellite Site Tape, CPU, Disk, Robot University CPU, Disk, Users University Users University Users University Users Satellite Site Tape, CPU, Disk, Robot u Matchmaking, Co-Scheduling: SRB, Condor, Globus services; HRM, NWS

PPDG WG1: Request Manager tape system HRM Replica catalog Network Weather Service Physical file transfer requests GRID Request Interpreter Disk Cache Event-file Index DRM Disk Cache Request Executor Logical Set of Files Request Planner (Matchmaking) DRM Disk Cache CLIENT Logical Request REQUEST MANAGER

LLNL Earth Grid System Prototype Inter-communication Diagram Disk Client Request Manager ISI GSI- wuftpd Disk SDSC GSI- pftpd HPSSHPSS LBNL GSI- wuftpd Disk ANL GSI- wuftpd Disk NCAR GSI- wuftpd Disk LBNL Disk on Clipper HPSSHPSS HRM ANL Replica Catalog GIS with NWS GSI-ncftp LDAP Script LDAP C API or Script GSI-ncftp CORBA

Grid Data Management Prototype (GDMP) Distributed Job Execution and Data Handling: Goals è Transparency è Performance è Security è Fault Tolerance è Automation Submit job Replicate data Replicate data Site A Site B Site C r Jobs are executed locally or remotely r Data is always written locally r Data is replicated to remote sites Job writes data locally GDMP V1.1: Caltech + EU DataGrid WP2 Tests by CALTECH, CERN, FNAL, Pisa for CMS “HLT” Production 10/2000; Integration with ENSTORE, HPSS, Castor

        EU-Grid Project Work Packages

GriPhyN: PetaScale Virtual Data Grids Build the Foundation for Petascale Virtual Data Grids Build the Foundation for Petascale Virtual Data Grids Virtual Data Tools Request Planning & Scheduling Tools Request Execution & Management Tools Transforms Distributed resources (code, storage, computers, and network ) è Resource è Management è Services Resource Management Services è Security and è Policy è Services Security and Policy Services è Other Grid è Services Other Grid Services Interactive User Tools Production Team Individual Investigator Workgroups Raw data source

Data Grids: Better Global Resource Use and Faster Turnaround u Build Information and Security Infrastructures è Across Several World Regions è Authentication: Prioritization, Resource Allocation u Coordinated use of computing, data handling and network resources through: è Data caching, query estimation, co-scheduling è Network and site “instrumentation”: performance tracking, monitoring, problem trapping and handling è Robust Transactions k Agent Based: Autonomous, Adaptive, Network Efficient, Resilient k Heuristic, Adaptive Load-Balancing E.g. Self-Organzing Neural Nets (Legrand)

GRIDs In 2000: Summary u Grids will change the way we do science and engineering: computation to large scale data u Key services and concepts have been identified, and development has started u Major IT challenges remain è An Opportunity & Obligation for HEP/CS Collaboration u Transition of services and applications to production use is starting to occur u In future more sophisticated integrated services and toolsets (Inter- and IntraGrids+) could drive advances in many fields of science & engineering u HENP, facing the need for Petascale Virtual Data, is both an early adopter, and a leading developer of Data Grid technology