1. CHEP 2000 Session F: Data Analysis:Technology and Presentation Convenors: Francois Etienne - IN2P3, Atlas, Ruth Pordes, FNAL Thank You to all the Speakers for Providing me with their Transparencies to use. 29 Oral presentations; 1 Poster

2. Outline Quick Review of Topics/Titles More Detail for Each Category Summary

3. Numerical Library Software - 2 papers Development of Numerical Library Software in Java – KEK – Initial Work Scientific Toolkit for Java – Phenix – Really an Analysis Framework (poster paper not in this session on COLT - Cern/IT) All in Java

4. Electronic Document Management systems - 4 papers from 2 systems Java-based User Interfaces for CERN's Engineering Data Management System Integrated Information Management for TESLA Asset Management for Cavities and Computers at DESY Experience with Document Management at DESY

5. Singletons: Bringing Mathematical Formatting to the Web - An Overview of the Technology and SLAC's Experience Multi-threaded, Discrete Event, Simulation of Distributed Computing Systems – CMS CORBA Scripting in HEP and Beyond. ATLAS DAQ back-end Software  Integrated toolset, mainly Open Source  Second use of this will be an interesting test.  Defining deliverables of “pre-design” phase.  How to move to an Open Source development environment for critical piece of Experiment architecture?

6. Event Displays - 3 talks Atlantis an Event Display for ATLAS: When displaying 6000 hits recorded by only 7 layers, can we recognize tracks? WIRED - World-Wide Web Interactive Remote Event Display HepRep: a Generic Interface Definition for HEP Event Display Representables

7. General Analysis Packages/Frameworks Java Analysis Studio ROOT at RUN II Open Scientist A Comparison of Data Analysis Packages (Root and IDL)

8. Experiment Analysis Packages A Graphics Sampler for the D0 Experiment The CMS IGUANA Project Object Oriented Data Analysis in the DELPHI Experiment Distributed BELLE Analysis Framework New Data Analysis Environment in H1

9. General Analysis Components Abstract Interfaces for Data Analysis – AIDA – Cern/IT A Histogramming package in Java - KEK Lightweight Histogram Interface Layer – from NERSC/LBL

10. Software Process and Configuration Management - 5 talks all from Europe.. An Evaluation of Tools for Static Checking of C++ Code Impact of Software Review and Inspection C++ Code Analysis: an Open Architecture for the Verification of Coding Rules CMT: a Software Configuration Management Tool Use of a Configuration Management Tool in LHCb Software

11. What there was Not Software Development Methodologies and Case Tools. 3-D and Virtual Reality. Overall Impression Continuation of directions and initiatives from last Chep.

12. Situation today (finally):  method for defining a mathematical expression in terms of its presentation and semantics  MathML is a standardized instance of XML Used “IBM TechExplorer Hypermedia Browser”  A Web browser plug-in  Dynamically formats and displays documents coded with T E X/L A T E X and/or MathML  Beta version applied to existing Spires TeX files works. Try it and feed back comments... F19 MathML Interface for Abstracts in SLAC Spires

13. F19 - Tool for Simulation of Complex Distributed Data Handling Systems - Technology – in house developed JAVA toolkit.  Process driven not event driven architecture  Full presentation and easy editing of parameters  encourage many different models and users  Immediate publication of results on the web  Enable physicists to read, analyse and feedback. Alternative architectures and designs can be commented with quantitative and analytic input Technology is ready to be tested for “Simulator Friendliness” …  Volunteers to model the existing Babar and Run II systems would indeed be useful.

15. F359, F65, F66, - DESY Integrated Information Management Reminded us of the importance of need for and time required to  Peoples Use and Access Patterns change after they have experience with a working system  Current and Future Business Processes significantly affect design  Users expect latest Technology (currently video, mobile computing etc) Demonstrated Success of collaborative development of vendor tool for HEP needs

16. Current Tuovi Status & Statistic 20 projects with: - 6 GB data vault - 2400 project nodes - 18600 files - 3300 registrated user

17. Cern EDMS Extensions Import/Export Interface - define an exchange data format in XML.  Conversion of metadata to and from various formats e.g. Excel, Html, Databases, Text Processors etc. in use by several LHC groups:  LHC Baseline and ABS  ATLAS: Import/Export of ABS Structures  CMS: ECAL ABS  ALICE: Documents and Files check-in  Documents Management in the LHC Design office (in progress)

18. Histogramming Packages Histogrammer in JAVA - Successful interfacing of Java and C++ objects for in a simple Histogrammer application Lightweight C++ Histogram Interface - Break between Statistical Data Gathering, from analysis and visualization tasks. Define Standard API and Support Object Structures.  How does it relate to transient Cern/HTL implementation

19. Analysis and Event Display Toolkits and Interfaces WIRED HepRep AIDA Java Analysis Studio Scientific Toolkit for Java Open Scientist ROOT IDL - through comparison with ROOT

21. Wired Rewrite since Last CHEP In Use by CHORUS, BaBar, DELPHI etc. Lacks a Scripting Language. BaBar Use shows up Need to Think before Defining what Data gets Transported between User Client and Data Server – don’t want a “Packet per Point”

22. HepRep Experience with Wired at BaBar led to need to layer interface between Client and Display so as to  reduce granularity in getting remote information  increase ability to for Client/Users to easily add objects and views HepRep defines a general API for serving Representables, not Physics Objects “Serving Representables keeps the detailed reconstruction code, swimmers and detector models on the server side where they belong. Spatial information is assembled and shipped in an efficient manner, avoiding the overhead of too many individual method calls.”

23. General Comments Transport the Physics Analysis Code to the Data - JAS – No Answer Yet on Whether this will Work Well. WIRED and JAS both Talking about Scripting Languages – no decision yet. Once Package in Use and Responding to User Use Needs, More Core Development More Difficult Integration with C++ Analysis Algorithms and Data Adaptors Not Here Yet.

24. AIDA Part of Next LHC++ Analysis Toolkit – the Start. HepVis ’99 established Collaboration for Definition of Abstract Interface for Data Analysis Interface Architected to Support Scripting and GUI Equivalently. Goal of Adaptors/Interfaces to HepTuple, JAS, Openscientist, Root etc.

25. JAS Extensions since last CHEP:  JEdit Editor Bean  Using Swing JEditorPane JAS now supports HTML display  XML support - JASHist bean directly supports reading/writing XML  Servlet Support Continued use in BaBar Online and LCD Simulations Packaged and made Available in OpenSource Paradigm - Contributions starting e.g. 3D Classes from Fermilab.

26. Scientific Toolkit for Java Explained was a Separate Project to “Learn Technology”.  Use of JavaHelp  Based onJava 2D (JDK 1.2) Generic Swing Console for those Interpretive Languages not supported by Swing. Anticipate Discussing Collaboration with / Contributions to JAS. Starting New Development Allowed use of Latest Generation Technologies

27. Open Scientist Work Continuing for Experiments – LHCb, Virgo, Auger etc. Concept of “Hub” as Integration Layer. What is boundary/layering between Frameworks and Utilities - Do we need More Concepts Here? Worklist includes Integration of CINT and Extensions to Free Open Inventor Implementation.

28. ROOT - Run II Perspective Progress since Run II Adoption of ROOT with Caveats. Priorities from 1999 Workshop been in Large Part Addressed – Robustness, CINT Extensions, Pre- compilation of user classes. Tutorials and Education Popular with Run II Users. To meet Local Needs currently Building 23 Flavors. Collaboration Works with Technical Commitment Second ROOT Workshop last week at Cern resulted in New Worklist - now posted – main extension Self Describing Data format.

29. Comparison of ROOT and IDL: IDL command language  Concatenate Sets of Files of ntuples - chain command.  Read in and Use Variables from an “ntuple”File - Local Extension.  Event Selection Cuts on Several Variables  Plot - Static or “Liveplot”. Large ntuple problem addressed by “mapped files”. Needs test drive for “large scale analysis”.

30. Experiment Specific Event Displays and Analysis

31. Mature Experiments Retooling Technology for Analysis – Delphi and H1 reported Issues include:  Maintenance – Technical and Resources;  Support of Multiple Generations of Technology. OO, ROOT, C++ - Speed and Usability Benefits Encourage New Collaborators Allow and Provide for Curation of the Data

32. Atlas Event Display - Atlantis Applying Hans Drevermanns Algorithms to New/Future Experiment  “V-plot”  Z finder  Hit filtering Plan Recode from Fortran to Java.

36. Experiment Event Displays – D0 Suite of Different Tools based on Legacy and New Packages. New Tools based on Open Inventor. Will Evolve. Finding very useful for algorithm development and starting for analysis of processed Monte Carlo data

37. Open Inventor HEPVis Mesa D  Graphics OpenGL Rest of D0 software Exceed 3D Xi Graphics D  Graphics Sampler

38. CMS - IGUANA On the way to being a Fully Featured Data Presentation and Manipulation Framework. Will be used in CMS Simulation and Test Beam Analysis this Year. Pragmatic Approach to getting Prototypes and Early Production Releases. Based on Open Toolkits and Open Inventor. “Sharing experience may be more important than sharing code”- aligned with LHC++ strategy.

39. IGUANA Objectivity tag analyser (tag is similar to ntuple) 6(d) Qwt & Qt 6 (b) HEPInventor & OpenInventor, Motif IGUANA Prototypes: Objectivity Browsers and Persistent Histograms and Tags Plotters 6(c) SciPlot (Xt / Qt) 6(a) HepExplorer & IRIS Explorer IGUANA Objectivity browsing

40. BELLE – BASF Analysis Framework Currently supports Single SMP Parallel Processing of Events for Users. dBASF Extension being Designed to Support Network Connected Clusters of PCs. Package of Daemons for  Job Management  Data I/O  Resource Management  Monitoring. Resource Managers Dynamically Reconfigures – Increase, Decrease - Number of Computing Servers Running Jobs.

41. HEP Configuration Management Tools Report from 2 users of CMT – Build and Release System. Poster papers on SRT Rewrite, NT Extensions From Marjorie’s Plenary Talk: “Everyone now using them, can’t we use the same one…”

42. jcmt browser editor monitor Package areas Packages & versions Free action onto the package Requirements editor Configuration monitor

44. F63 - C++ Code Rules and Checking C++ Coding Standard – 180 Rules for Naming, Coding and Style of Varying Severity for Each Experiment. Evaluation of Commercial tools for Code Rule Checking. Tested on Geant4 and Atlas Libraries. Out of 5 tools considered 2 Meet the Basic Needs – about 75% of total: CodeWizard and QA C++. Now Traditional Questions on: “What does it take to Decide?” “Will IT/CD Support It?”

45. F361 - C++ Code Rules and Checking - Alice Development of Code Analysis Tools - Parser, Formatter, Checker Limitations in Checking Complete C++ syntax e.g. Templates, Namespace Successfully used on ALICE code

46. F331 - Atlas Software Review and Inspections Movement in HEP from Fagan Inspections (Fermilab SDSS/DART/Run II) to Gilb Inspection Method Bodes Well that this is Part of Formal Development of Proposed OpenSource Project..

48. F119 - Applications using CORBA Script – continuation of work reported at Chep ‘98. Interpreted Scripting Language which can interact with CORBA objects through IIOP and has dynamic access to any CORBA object. OMG Standard Interface Definitions Stored in Repository Applications in Babar Data Handling System

49. Summary – Technology Java, XML, CORBA JAVA  Mainstream for New Presentation Applications.  Not Mainstream for Data Analysis and Reconstruction.  Work Not yet Done to Integrate with C++ Algorithm Modules and Frameworks.  Work only just Beginning on HENP Utility Libraries.

50. Summary – Physics Analysis Packages Not much development using Commercial Analysis Tools. ROOT most Generally Used; Extensions still being requested to meet Experiment Long Term Needs. Open Source Collaboration without Open Source licence? JAS, OpenScientist, WIRED, progressing steadily. Question still Open as to How or Whether any of the Existing Analysis Technologies will Successfully meet Needs of >500 Physicists doing Analysis of Babar and Run II over the next 10 Years.

51. Summary – Development Environment Technology Experiments looking for more and better tools for Configuration Management but  no widespread adoption yet of single solution  no work yet on significant extensions to the services provided  no new work to look at commercial tools Attendees very interested in Code Quality and Checking Tools and Process – but if the number of talks is an indication of the relative investment..

52. Summary - misc Mature Experiments Taking Time to Upgrade Analyses to New Technologies to Support Ongoing Analysis and Provide Tools for Newcomers –  Reminder this will Happen for Current and Future Experiments. (Where did Virtual Reality go?)