1. Feb. 26, 2001L. Dennis, FSU The Search for Exotic Mesons – The Critical Role of Computing in Hall D

2. Hall D Collaboration Map

3. Production of Mesons and Gluonic Excitations Using 6-12 GeV Photons Fundamental Physics Role of “glue” in strong QCD Experimental Goal Unambiguous identification of gluonic excitations starting with exotic hybrids Experimental Requirements Hybrids are expected to exist precisely where we have Almost no experimental information – photoproduction Requires 6 – 12 GeV photon beam energies

4. Formation of Flux Tubes

5. Hybrids

6. Looking for Hybrids We should observe exotic hybrids precisely where we have no data: PHOTOPRODUCTION S = 0 – For pion and kaon probes where most of our data exist S = 1 – Use a probe with quark spins aligned - the photon where we have essentially no data

7. Predicted Meson Spectrum Predictions for exotic mesons come from: Lattice QCD Flux Tube Models In flux tube picture, gluons in hadrons are confined to flux tubes. Conventional mesons arise when the flux tube is in its ground state. Hybrid mesons arise when the flux tube is in an excited state. Meson Map

8. Hall D Online Data Acquisition CEBAF provides us with a tremendous scientific opportunity for understanding one of the fundamental forces of nature. 75 MB/s 900 MB/s

9. Critical Role for Computing in Hall D The quality of Hall D science depends critically upon the collaboration’s ability to conduct it’s computing tasks.

10. The Challenge Minimize the effort required to perform computing Data Intensive Application Compute Intensive Applications Information Intensive Analysis Research Application – methods and algorithms are not fully defined.

11. Trigger Rates for Hall D Detector 180 kev/s Trigger 15 kev/s 5 kB/ev 75 MB/s Trigger requires ~100 CPU’s* * Assume a factor of 10 improvement over existing CPU’s 5 CPU-ms/evFull Reconstruction (CLAS) 50 ms/ev today. 100 CPU-ms/evFull Simulation (CLAS) 1-3 s/ev today. 1/3Assumed detector & accelerator efficiency.

12. Required Sustained Reconstruction Rate [15 kev/s] * [1/3] * [2] = 10 kev/s Equipment Duty Factor Raw Rate Duplication Factor 10 kev/s * 5 CPU-ms/ev = 50 CPU’s

13. Required Sustained Simulation Rate 5 kev/s * 100 CPU-ms/ev = 500 CPU’s [15 kev/s] * [1/3] * [10] * [1/10] = 5 kev/s Equipment Duty Factor Raw Rate Systematics Studies Good Event Fraction PWA error is determined by one’s knowledge of systematic errors. This requires extensive simulations, but not all events simulated are accepted events.

14. Annual Date Rate to Archive Raw Data 75 MB/sec * (3 *10 7 s/yr) * (1/3)= 0.75 PB/yr Simulation Data 25 MB/sec * (3 *10 7 s/yr) = 0.75 PB/yr Reconstructed Data 50 MB/sec * (3 *10 7 s/yr) = 1.50 PB/yr Total Rate to Archive ~ 3 PB/yr

15. Requirements Summary

16. Annual Data Rates

17. CPU Requirements

18. Hall D Computing Tasks First Pass Analysis Data Mining Physics Analysis Partial Wave Analysis Physics Analysis Acquisition Monitoring Slow Controls Data Archival Planning Simulation Publication Calibrations

19. Initial Estimate of Software Tasks & Timeline

20. Meeting the Hall D Computational Challenges Moore’s law: Computer performance increases by a factor of 2 every 18 months. Gilder’s Law: Network bandwidth triples every 12 months. Solving the information management problems requires people working on the software and developing a workable computing environment. Dennis’ Law: Neither Moore’s Law nor Gilder’s Law will solve our computing problems.

21. “Chaos of Analysis” Problem: It is impossible to efficiently complete our computing in a single large, common, democratic computer facility. Solution: Provide several sites with the resources required to complete specific tasks. Choose those sites which seek to become lead institutions in specific efforts, such as simulations, calibrations or partial wave analysis.

22. Hall D Grid

23. Common access for Physicists everywhere. Common access for Physicists everywhere. Utilizing all intellectual resources Utilizing all intellectual resources  JLab, universities, remote sites  Scientists, students Maximize total funding resources while meeting the total computing need. Maximize total funding resources while meeting the total computing need. Reduce Systems’ complexity Reduce Systems’ complexity  Partitioning of facility tasks, to manage and focus resources. Optimization of computing resources to solve the problem. Optimization of computing resources to solve the problem.  Tier-n or “Grid” Model. Reduce long-term computational management problems. Reduce long-term computational management problems. Grid Computing Advantages

24. Hall D Offline Data Flow

25. Digital Hall D Ground Rules Distributed Objects Define all programs and data as objects. Define “or wrap” everything in XML. Implement in Object Model de jour (CORBA, Java, COM, SOAP …) Does not require that we use an Object Database or that we use relational databases inappropriately. Move and query metadata rather than data whenever possible. Move the applications to the data. Assume everybody has wireless access to the “Digital Hall D” through hand-held and conventional computers.

26. Digital Hall D Technologies  HallD Grid Globus provides infrastructure to access computer resources around the world  HallD Grid. Structure access to Digital Hall D as a Portal –  myHallD.org Use a multi-tier software architecture separating resources, servers/brokers, display engines, display devices. Do not write any HTML – use XML and convert. Program in C++ or Java.

27. Hall D Grid

28. Vision for Grid Environment Work toward a Grid-based Operating System. Standard toolkit for manipulating objects. For example: copy, find, create, delete,… Standards for developing additional complex Grid based tools. For example: A tool that builds an acceptance function from available GEANT simulations, whose results are stored in several locations. Tools to share intermediate results of large computations. Many of these tools exist, it is remain to selecting the appropriate ones and wrap them in standardized interfaces so they can work with Hall D objects.

29. Foundations for Grid Sites Grid Services Data Services Compute Services Information Services Interactive Services Batch Services Needs Very Reliable Hardware & Software at Remote Sites Needs Very Reliable, Easy to Install Software at Remote Sites

30. Hall D Grid

31. Logout Select Configure

32. ……... Hierarchy of Portals and Their Technology Portal Building Tools and Frameworks (XUL, Ninja, iPlanet, E-Speak, Portlets, WebSphere, www.desktop.com) Enterprise Portals Generic Portals Education & Training Portals Science Portals K-12University Biology Chem Eng Collaboration Universal Access Security ……. Databases ……. User customization, component libraries, fixed channels Education ServicesCompute Services Information Services Generic Services

33. Collaborative Objects Digital objects shared by more than one person. Asynchronous sharing: You create/modify an object. Others access/modify it at a later time. Synchronous Collaboration: Real-time access/modification of objects by several people in distributed locations.

34. Virtual Experimental Control Room Could be a big win as (unexpected) real-time decisions need “experts-on-demand.” Model being considered by NASA for remote spacecraft mission control and real-time scientific analysis of earthquakes. Need collaborative decision making (vote?) and planning tools. Needs shared streaming data and shared read-outs of experimental monitors (output of all devices must be distributed objects which can be shared). Needs to support experts caught on the beach with poor connectivity or in their car with just a cell phone and a PDA.

35. Building Computer Science & Physics Teams for Computing System Development Physicists Computer Scientists Computing environment we need to be successful $’s Prestige Tradition $’s Prestige Tradition

36. Conclusions Hall D provides tremendous opportunities for new physics. Requires unprecedented computing. Grid and portal technology provide a unique new method of involving distributed intellectual resources in this important problem. The resources required to create those solutions are not yet in place.

37. Collaboration Computing Organization Attracting physicists to work on software is difficult. Perceived importance is based on capital “$’s” spent. Accelerator  Detector  Computing. Once it works, they have nothing they can show to their dean and say, “I built that!” “Everyone” thinks it is easy. One good way to have a really positive impact on the science. Helps train and attract students for a variety of careers.

38. Collaboration Computing Organization Attracting computer scientists to work on physics software is difficult. Perceived importance is based on computer science research, not computer science applications. Physics publications don’t help computer scientists get tenure. “Everyone” thinks it is easy. A good way to actually test computer science theory. Science requires experimental testing to progress. Real world training ground for students.

39. 2 Tape Drives 4/1 ratio of processing to I/O per tape 1.2 TBytes of Disk Required e3e4 b1b2b3b4 a1a2a3a4 b5c1c2c3 b1b2b3b5b4a1a2a3a4a5 c4 a1 a2 a3 a4 a5 d1 b1 b2 b3 b4 b5 c1 c2 c1c2c3d1c4 c3 c4 d1 d2 d3d4e1e2 d3 d4 e1 e2 e3 e4 e3e4 d2d3d4e2e1 Start-up Equilibrium f1f2f3f4 f1 f2 f1f2f3f4 f3 f4 g1g2 Shut-down x1x2x3y1x4 x3 x4 y1 y2 y3 y4 z1 z2 z1z2z3z4 y2y3y4z2z1 x2 x1 w4 z3 z4 z3w4w3 Obtaining Optimum System Performance

40. Estimated System Efficiency

41. Efficient Information Access is Key to Using the HallD Grid Data Acquisition  Raw Data,  Experimental Conditions Calibrations Simulations Data Reduction Physics Analysis PWA Information From Researchers Hall D Experimental Information

42. Focus  Accurate, Timely Analysis Provide people with the information and resources they need to conduct their analysis Provide it reliably Provide it in the way scientists need it Provide it efficiently (speed, effort) Provide flexibility for other applications

43. Hall D Portal: MyHallD What’s Involved in MyHallD ? Probably needs some money, but < $30.9442 M, Commitment to use the “HallD Digital Object Framework”. Basic functions are available in existing commercial systems. Start to use these. Prototype some of the special capabilities needed. What is involved in making HallD objects collaborative? First use objects! Then we have choices – which vary in ease of use and functionality.

44. MyHallD: The Portal Door to: Experiment Control Room Simulation Farms & Data Calibration Farm & Data Reconstruction Farm Analysis Farms & Data Board Room & Archive Personalized Electronic Logbook Hall D Education and Outreach Area

45. Collaborative Computing Organization Clearly establishes responsibility for software subsystems. Gives University groups working on software something to show for their efforts. Helps to attract people and resources to the computing efforts. Can leverage other University and National resources. Infrastructure, personnel, funding, NSF & DOE ITR initiatives. Eases the creation of customized (Grid) computing systems. Establishes new capabilities within the JLab/NP community. These capabilities allow JLab to take advantage of new opportunities.

46. Critical Software Issues Early creation of a “core group” of software developers. Creation of key design elements. Commitment to key design goals. Key Software Problems. Simulations. Software organization and management. Data formats for raw and derived data. Software for defining and accessing raw and derived data. Event visualization. Using available software. Developing & maintaining high-quality software.

47. Computing Organization Issues Recommendations. Online database – rely totally on automated methods. Offline database – rely totally on automated methods. Integrated online/offline/simulation database. Event Analysis – do it at Jefferson Lab. Calibrations – possible to do elsewhere. Physics Analysis – possible to do elsewhere. Simulations – possible to do elsewhere. PWA – possible to do elsewhere.

48. Computing Organization Issues (continued) Recommendations. Develop infrastructure to easily share computing resources and information. Develop customized computing approach to Hall D computing. Provides clear lines of responsibility for software and computing tasks. These are social decisions – not technical or financial decisions.

49. Collaboration Computing Organization The job is too big to be managed without databases. Provides wider access to experimental information. Databases are optimized for managing large data sets. We will create 5 – 10 M files every year. Database use can be organized to minimize it’s impact on time critical applications.

50. Experiments Database RunDetector Config. Analysis SimulationCalibration 1/M M/M

51. Online & Offline Analysis Integrated online & offline analysis systems. Pros: Common system requires less effort. Encourages cooperation between online & offline. Potentially higher reliability. Challenges: Broad contributions to offline analysis require standards and convenience  performance overhead. Level 3 trigger performance must be acceptable. No working Level 3 trigger system at JLab. No “suitable” memory management system for CODA events.

52. Online, Offline & Simulation Database Automated Experiments Database. Pros: Common system requires less effort. Encourages cooperation between different computing groups. Better organization of needed information. Higher reliability and better access. Challenges: Anyone software developer in the information chain can break it. Distributed simulations require modern organization of the database.

53. Where to Perform 1 st Pass Analysis? 1 st Pass Analysis at JLab. Pros: Don’t need to transport the data. Computer system support is in place. Detector experts on site. Challenges: Oversubscribed computer system. Obtaining efficient tape access, system throughput is unlikely in a heterogeneous computing environment.

54. Where to Perform Physics Analysis? Physics analysis is done where the researcher live. Pros: Not competing with major analysis & simulation efforts. Easier to involve more people. Challenges: Requires a portable analysis code. Requires a good system for quality control of results.

55. Where to Perform Simulations? Simulations done at a few institutions. Pros: Get more groups invested in simulation effort. Probably don’t need to transport the data. Easy to do remotely. Challenges: Need computer infrastructure in place. Need software infrastructure in place.

56. Key Differences Between Halls B and D More uniform physics goals in Hall D. Jefferson Lab computing infrastructure is in place. Hall B computing personnel hired late in the process. Fundamentally changed the direction of the software and organizational approach to the problems. Many things had to wait until the very last minute.

57. Related Computing Trends We depend on commodity computing Clusters Networks Storage Media (disks & tapes) Intel’s Merced processors (Itainium) 500 MHz, 64 bits, 4-way processor A year late File Size Currently 2 GB software limit 2 GB going to 2 32 * 2 GB (effectively infinite for us) What determines the optimum file size?

58. Related Computing Trends (Continued) Grid Computing High speed networks Distributed “service” or “data” centers GLOBUS, Legion, home-grown XML – not just a better HTML Standard method for creating self-describing data Many tools available (B2B) Mobile Computing, Portal Technology Customized access to computing resources via data starved devices Customized view of an experiment or equipment

59. Benefits of XML Standardized access to databases and applications. DB to XML DB Select XML to XML Select Application XML  App XML to DB Config. View Launcher XML  App

60. Benefits of XML Standard routines exist in Perl, C++ and Java for converting between internal and external storage. XML  SIISII  App XML  App SII XML  SIISII  App XML  App SII

61. Hall D Computing Requirements