1. Thoughts on Applications Area Involvement in ARDA
Torre Wenaus, BNL/CERN
LCG Applications Area Manager
ARDA Workshop
Jan 22, 2004

2. The main inputs so far… ARDA AA Meeting Nov 27
Small first meeting with the AA LCG people we expect to be involved with a substantial amount of their time
Derek Feichtinger, Juha Herrala, Kuba Moscicki, Frederick Orellana
Plus Frederic Hemmer, Predrag Buncic, Dirk Duellmann, Alberto Aimar and myself
Had an overview of ARDA from Predrag, then discussed possible areas of AA activity
Frederic’s initial ARDA middleware meeting in early Dec
TW attended (some of it) as AA rep
PEB and SC2 presentations of (almost) this talk in Dec
Workshop yesterday
Didn’t lead me to change anything substantive in this talk from the version presented Dec 9 (not necessarily a good thing)

3. General ARDA AA Objectives
Common software above the middleware layer
Adapting, extending, interfacing AA software for ARDA
Participating in ARDA interface definition; ensuring AA requirements met
Applying lower level middleware services in specialized higher level services directed at HEP and analysis
Early PEB agreement on ARDA: Middleware covers as much as possible; remaining higher levels covered by AA (if common) or experiments (if not)
Integration and validation
Integrating ARDA middleware services and analysis application level services into end-to-end distributed analysis prototype
Assisting integration of distributed analysis prototype or components thereof into experiment environments
Validation of the prototype [and feedback to middleware providers]
Proposal to use the GAG as the principal feedback channel seems a very good one to me

4. We expect ARDA will use SPI services and policies
Possible work areas
Event data management and access
Framework integration services
Interactive analysis tools
Analysis environment integration and validation
… and with thoughts on work package organization
We expect ARDA will use SPI services and policies

5. Event data management and access
Event collections, physics-level datasets, physics queries
Efficient sparse data access
Data access below file level (event objects)
Splitting at physics dataset level
A mix of interface development, POOL work, ROOT work
Collections work currently going on in a POOL WP, but this work needs an ‘analysis’ perspective and not just a ‘persistency’ perspective – ARDA can provide that
Make this ARDA work package a joint work package with POOL Collection WP
Will be addressed by Dirk in the next talk

6. Framework integration services
Interfacing/integrating framework-level distributed services
Distributed messaging, error handling, logging, …
Interactive interface; Python, ROOT bindings
Framework access to more sophisticated middleware services?
Workflow management, replication, …
Probably mostly a very ‘thin’ activity
not developing services, or even the interfaces
But contributing to interface definition
just packaging/integrating them for the AA architecture
Maybe participation in some specialization of generic services closely connected to the framework? (e.g. managing provenance data)
The long-empty ‘grid based services’ box in SEAL
Joint ARDA/SEAL WP

7. Interactive Analysis Tools
Interfacing to tools supporting interactive (low-latency, rapid-response) analysis
ROOT, PROOF integration
User interfaces to tools supporting analysis workload management
User level management/monitoring
User level reservations (‘what’ and ‘when’)
Interfacing to tools supporting dynamic job interaction/control
AIDA integration
Needs will vary from experiment to experiment; probably mostly experiment-specific integration
If there is an AA element, fold into the next WP…

8. Analysis Environment Integration & Validation
ARDA integration as an analysis system in experiment environments
Integrating experiment specific front end and service components with ARDA components into an end-to-end system
Early priority: users in experiments testing detailed use cases using experiment-integrated ARDA
Get ARDA in the hands of physicists doing analysis as soon as possible (as soon as there is a tool of interest to attract them – experiment ARDA teams need to sell the product)
The key work package
Support and assist four distinct but collaborative ARDA integration efforts in the experiments
Coordination for the higher level elements of ARDA not coming from the middleware
Coordinate gathering of feedback from experiment ARDA teams/users
Use GAG for middleware feedback channel
Provide overall coordination/coherence for ARDA end-to-end analysis systems

9. Summarizing My Current Thoughts on WPs
Integration and Validation
Primarily providing coordination, communication, coherence for efforts residing in the experiments and projects
Some similarity to Physics Validation in the simulation project
Though the (majority of the) work will go on in the experiments and projects, a common focal point is needed if it is to be a common effort
Event data management
Physics-driven event collections
Joint WP with POOL Collections
Framework integration
‘Thin’ adaptation of middleware services to whatever is required for integration in experiment analysis frameworks
Joint WP with SEAL

10. Slide Title : Overview of Main Activities and Deliverables
Experiment Prototype
Applications
ALIEN
ADA
GAE
DIRAC
physicists
with real requirements Al At C L
Other
Common
project
ARDA Services
Grid middleware
POOL’, SEAL’,
HEP-specific
Operational
experience Al At C L
HEP-common Service
SEAL
ARDA
coordination
integration, validation
development specific
services
GAG
SEAL’ Services
Consolidated
Requirements
POOL’ Services
POOL
Slide Title : Overview of Main Activities and Deliverables
Slide shows how ARDA project could relate to those existing activities that have a relevant role to play (circles). It also shows the main deliverables (boxes)
Shows scope of ARDA in green (indicates added value of project and in what context ARDA effort is working)
ARDA should provide a consolidated set of middleware services to the experiments that then use them to produce rapid prototypes of realistic applications, gaining operational experience which is fed back to the project.
Consolidated sets of requirements and use cases are provided by a cross experiment forum of appropriate experts (GAG – already exists and could assume this role with appropriate membership)
The basic (non-HEP-specific) services are developed by a joint EGEE/VDT effort:
delivery of an early prototype and further releases under short evolutionary cycles
uses experience and technology from existing grid development projects (e.g. AliEn, EDG, NorduGrid, VDT ..)
re-engineering according to agreed architectural models (OGSI) and implementation technology choices
HEP-specific services may also be identified and developed in an appropriate context, e.g.
POOL for data management related services
SEAL for common services involving integration with other standard software components (UI, Job control, etc.)
Main role of ARDA itself is one of coordination, integration, validation, development of other specific services
Callout emphasises importance of involvement of physicists doing real distributed analysis, having real requirements
ARDA itself may comprise people from the project and from the experiments. Project members will all have an experiment label (AACL) but will work together as a team.
Experiment Requirements
and Use Cases
Grid Middleware
Services
Experiment
AliEn
grid technology and experience
EGEE/ VDT
LCG
EDG
Grid MW
Nordu . . . .
ARDA
VDT