Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Clusters, Computational Grids and Beyond CCCGSC 2006.

Similar presentations

Presentation on theme: "1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Clusters, Computational Grids and Beyond CCCGSC 2006."— Presentation transcript:

1 1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Clusters, Computational Grids and Beyond CCCGSC 2006

2 Science, Engineering, and “Commercial” Applications Environments: how are they shaping in the future What does it entail for: Clusters, Computational Grids, and Beyond Examples of research supported by two programs: Computer Systems Research (CSR) And Dynamic Data Driven Applications Programs

3 3 Applications Directions –Multi-Modular, Multi-Language, Multi-Developers, Multi-Source Data –Compute Intensive, Data Intensive, –Real-Time, Few Minutes/hours –Visualization, Interactive Steering –Dynamic Data Driven Applications Systems Computational Platforms Directions Distributed Platform MPPNOW SAR tac-com data base fire cntl fire cntl alg accelerator data base SP …. Petaflops Platform (Grid-in-a-Box) Heterogeneous architecture, computational,networks, memory hierarchies, latencies, etc…

4 4 The need for a holistic approach Large-Scale Systems does not entail only “flops” (Giga-, Tera-, Peta-, Zetta-,…) Large-scale “parallel” systems are the POWERFUL nodes/platforms - in balance with other resources in the system Analogy: the “galaxy” within the “cosmos” Methods and Tools needed at all levels, and they need to work together synergistically

5 5 Dynamic Data Driven Application Systems (DDDAS) New Direction for applications/simulations and measurement methodology Multi-agency DDDAS program – NSF, NIH, NOAA with cooperation with the EU/IST & e-Sciences Programs ( 248 proposals received (June 13, 2005); total requested amount $175M 32 projects awarded; total amount of funding $16M (US)

6 6 Measurements Experiments Field-Data User Theory (First Principles) Simulations (Math.Modeling Phenomenology) Experiment Measurements Field-Data (on-line/archival) User Theory (First Principles) Simulations (Math.Modeling Phenomenology Observation Modeling Design) OLD (serialized and static ) NEW PARADIGM (Dynamic Data-Driven Simulation Systems) Challenges : Application Simulations Development Algorithms Measurement Instruments Interfaces Computing Systems Support Dynamic Feedback & Control Loop What is DDDAS (Symbiotic Measurement&Simulation Systems)

7 7 The Instrumented Oil Field of the Future Detect and track changes in data during production Invert data for reservoir properties Detect and track reservoir changes Assimilate data & reservoir properties into the evolving reservoir model Use simulation and optimization to guide future production, future data acquisition strategy PI: Mary Wheeler; Collaboration: UT-Austin, Ohio State U, Rutgers U, Oregon State U., U. of Chicago Safer, More Efficient Oil Drilling & Recovery (DDDAS application)

8 8 Optimize Economic revenue Environmental hazard … Based on the present subsurface knowledge and numerical model Improve numerical model Plan optimal data acquisition Acquire remote sensing data Improve knowledge of subsurface to reduce uncertainty Update knowledge of model Management decision START Dynamic Decision System Dynamic Data- Driven Assimilation Data assimilation Subsurface characterization Experimental design Autonomic Grid Middleware Grid Data Management Processing Middleware Dynamic Data Driven Model Optimization

9 DynaCode: A General DDDAS Framework with Coast and Environment Modeling Applications Gabrielle Allen, LSU, Greg Stone (LSU), Johannes Westerink (Notre Dame), Burak Aksoylu (LSU), Ivor van Heerden (LSU), Ed Seidel (LSU), Robert Twilley (LSU) DynaCode: Hurricane ensemble modeling Coupling ocean circulation, storm surge, wave generation Structural impact on levies Infrastructure & algorithms to couple models, to each other and to external inputs from sensors, wind & databases to optimize execution of complex workflows on grids, invoking appropriate models, meshes, and algorithms, depending on current conditions. –Applications: control algorithms and coupling interfaces for coastal/eco-codes –Maths: Model errors for scenarios and implement basic stats toolkit in Cactus to drive ensembles –Systems: Enhance dynamical capabilities of Cactus/Triana, decision making infrastructure, dynamic recomposition. Add legacy ensemble modeling to Cactus. Think about tracking data flow/data sensitivity through Cactus. –Measurements: Integrate data to scenarios from existing sensors, plan out interfaces for sensor control.

10 10 Examples of Applications benefiting from the new paradigm Engineering (Design and Control) –aircraft design, oil exploration, semiconductor mfg, structural eng –computing systems hardware and software design and runtime ( performance engineering and runtime compiler systems ) Crisis Management and Environmental Systems –transportation systems (planning, accident response) –weather, hurricanes/tornadoes, floods, fire propagation Medical –Imaging, customized surgery, radiation treatment, etc –BioMechanics /BioEngineering Manufacturing/Business/Finance –Supply Chain (Production Planning and Control) –Financial Trading (Stock Mkt, Portfolio Analysis) DDDAS has the potential to revolutionize science, engineering, & management systems

11 11 Some Technology Challenges in Enabling DDDAS Application development –interfaces of applications with measurement systems –dynamically select appropriate application components –ability to switch to different algorithms/components depending on streamed data Algorithms –tolerant to perturbations of dynamic input data –handling data uncertainties Measurements –Multiple modalities, space/time distributed, data management Systems supporting such dynamic environments –dynamic execution support on heterogeneous environments –Extended Spectrum of platforms: assemblies of Sensor Networks and Computational Grid platforms –Architect, management of sensor networks –GRID Computing, and Beyond!!!

12 12 Beyond Grid Computing “Extended Grid’: the Application Platform is the computational&measurement system Applications Computational Platforms Instruments Sensors Archival/ Stored Data MeasurementsComputational Grids

13 13 Why Now is the Time for DDDAS Technological progress has prompted advances in some of the challenges –Computing speeds advances (uni- and multi-processor systems), Grid Computing, Sensor Networks –Systems Software –Applications Advances (parallel & grid computing) –Algorithms advances (e.g.: parallel &grid computing, numeric and non-numeric techniques: adaptive, asynchronous algorithms, dynamic meshing, data assimilation, chaotic Monte-Carlo) DDDAS provides a fertile ground for multidisciplinary research and advances in Applications, algorithms, instrumentation methods, systems software, data management, … new infrastructures

14 14 Components in Computer Systems Research Program (CSR): Advanced Execution Systems (AES) System Modeling and Analysis (SMA) AES: Seeks to create systems software to facilitate the development and runtime support of complex applications executing on large, heterogeneous high-end computing and grid platforms –Programming models and tools –Runtime compiling system (RCS) technology –Application composition system (ACS) technology SMA: Develop methods and tools for modeling, measuring, analyzing, evaluating, and predicting the performance and dependability of complex computing and communications systems taking a “system level view” –Multi-modal hardware and software modeling –Multi-level modeling and measurement approaches –Performance Frameworks

15 15 Dynamically Link & Execute Systems software technology for integrated feedback & control Runtime Compiling System (RCS) and Dynamic Application Composition Application Model Application Program Application Intermediate Representation Compiler Front-End Compiler Back-End Performance Measuremetns & Models Distributed Programming Model Application Components & Frameworks Dynamic Analysis Situation Launch Application (s) Distributed Platform Adaptable computing Systems Infrastructure Distributed Computing Resources MPPNOW SAR tac-com data base fire cntl fire cntl alg accelerator data base SP ….

16 16 Performance Engineering Dynamic Compilers & Application Composition Dynamic Data-Driven Application Systems -- Symbiotic Measurement&Simulation Systems Large-Scale Systems (e.g. Enabling DDDAS) Systems Software (NGS: 1998-2004) (CSR/AES&SMA: 2004-todate) Multidisciplinary Research Applications Modeling & Measurements CS Research

17 17 The need for a holistic approach Large-Scale Systems does not entail only “flops” (Giga-, Tera-, Peta-, Zetta-,…) Large-scale “parallel” systems are the POWERFUL nodes/platforms - in balance with other resources in the system Analogy: the “galaxy” within the “cosmos” Methods andTools needed at all levels, and they need to work together synergistically

18 18 Experimental Dynamic Observations Users ADaM ADAS Tools NWS National Static Observations & Grids Mesoscale Weather Local Observations Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services LEAD: Users INTERACTING with Weather Interaction Level II: Tools and People Driving Observing Systems – Dynamic Adaptation

19 19 Why A Service-Oriented Architecture? Flexible and malleable Platform independence (emphasis on protocols, not platforms) Loose integration via modularity Evolvable and re-usable (e.g. Java) Interoperable by use of standards  robustness LEAD Service-Oriented Architecture Distributed Resources Computation Specialized Applications Steerable Instruments Storage Data Bases Resource Access Services GRAM Grid FTP SSH Scheduler LDM OPenDAP Generic Ingest Service User Interface Desktop Applications IDV WRF Configuration GUI LEAD Portal Portlets Visualization Workflow Education Monitor Control Ontology Query Browse Control Crosscutting Services Authorization Authentication Monitoring Notification Configuration and Execution Services Workflow Monitor MyLEAD Workflow Engine/Factories VO Catalog THREDDS Application Resource Broker (Scheduler) Host Environment GPIR Application Host Execution Description WRF, ADaM, IDV, ADAS Application Description Application & Configuration Services Client Interface Observations Streams Static Archived Data Services Workflow Services Catalog Services RLS OGSA- DAI Geo-Reference GUI Control Service Query Service Stream Service Ontology Service Decoder/ Resolver Service Transcoder Service/ ESML

20 20 Dynamic Workflows Automatically, non-deterministically, and getting the resources needed

21 21 NSF Engineering Research Center for Collaborative Adaptive Sensing of the Atmosphere (CASA) Current (NEXRAD) Doppler weather radars are high-power and long range – Earth’s curvature prevents them from sensing a key region of the atmosphere: ground to 3 km CASA Concept: Inexpensive, dual-polarization phased array Doppler radars on cellular towers and buildings –Easily view the lowest 3 km (most poorly observed region) of the atmosphere –Radars collaborate with their neighbors and dynamically adapt the the changing weather, sensing multiple phenomena to simultaneously and optimally meet multiple end user needs –End users (emergency managers, Weather Service, scientists) drive the system via policy mechanisms built into the optimal control functionality NEXRAD CASA

22 MIPS: A Real-Time Measurement- Inversion-Prediction-Steering Framework for Hazardous Events V. Akcelik (Stanford SLAC) G. Biros (University of Pennsylvania) A. Borzi (Graz) A. Draganescu (Sandia, NM) J. Hill (Sandia, NM) O. Ghattas (UT Austin) B. Van Bloemen Waanders (Sandia, NM) K. Willcox (MIT)

23 23 Sensitivity to sensor array density

24 Measured Response A Homeland Security Simulation Synthetic Environments for Analysis and Simulation (SEAS) Alok Chaturvedi, Director Shailendra Mehta, co-Director Purdue Homeland Security Institute

25 25 Reproduction Model Susceptible Mortality Exposed Infected w/o Symptoms Infected w/ Symptoms Immune recovered Succumb to the disease mortality not due to infection entering incubation period end of incubation period Interventions: Screen, Isolate (camp or shelter), Treat, Vaccinate Get in contact with infected Uninfected Immunized

26 26 Mobility Models Regular Movement Event Traffic Morning/Evening Rush Evacuation Panic Fleeing

27 27 Shared Reality Engine Shared Reality Model

28 28 Virtual Ground Zero Integrate independent cross platform simulation models Provide data exchange capabilities across models Enable mutual dependencies between models Integrated visualization view of multiple simulations

29 29 Interaction between Fire and Structure Models

30 30

31 31 Citizen Agents Retain history of environment/events/ behaviors that affect current behaviors Perceive environment and current individual state from agent attributes Learn from outcomes of behavior Income Class Fundamentalism Arousal Level Valence Level Well-Being DNA Nationalism Ethnicity Religion Organization Membership Blue Government Ethnic/Religious Media Affiliation Terrorist Progression Model Well-Being Model Organization Member Model Media Perception Model Memory Model Perception Model Learning Model Social Contagion Model Insurgency Model Social Group Formation Uprising during time of social unrest Formation of interest groups

32 32 SEAS SoS Concept Geography Population Demographics Well-being & Emotion Behavior Models Media Effect Models Religion Models Continuous Experimentation Design for Robustness Early Warning Response and Recovery Countermeasures Happy Sad Content aroused Susceptible Mortality Exposed Infected w/o Symptoms Infected w/ Symptoms Immunize Internet 2 i-light Gigabit Network Internet 2 i-light Gigabit Network Distributed Tera-Scale Computing Capable Epidemic Endemic Pandemic Traits & Behaviors Emotions Contagion Web-based Collaborative Modeling Tools Gaming, Simulation, & Experimentation Framework Agents’ Traits, Emotions, & Contagion Models Population & Infrastructure Models Computational Models -- Palm top to Teraflop Integrated Infrastructure Models

33 33 Beyond, just High-Performance Computing Examples of Computational Req’s of DDDAS applications) Water Pollution/Contaminant Transport/Detection: Today’s problem: 500nodes- 4.4Pflops;1.2GBmem;.02GB/s -> Large/Projected problem: 10,000nodes-212Pflops; 10.2GBmem;.9GB/sec) Chemical Pollution/Contaminant Transport/Detection Today’s problem: 2000nodes(Lemieux); 4TBmem ; 5hrs -> Large/Projected problem: 10Knodes; 20TBmem; 1hr Need in Real-Time (or near RT): 50-100Knodes Protein Folding Today’s problem: 1024nodes(IBM-BlueGeneL); 6/7days for 1 protein (w 150aminoacids) ElectricPowerGrid Today’s problem: 100Gflops; 50MBmem Aircraft modeling Today’s problem: Full FEM&CFD: 384,000cpu-hrs; 320GBmem ROM: 72secs; 78KB Fire Propagation: Today’s problem: FireModel: 100procs(BG, Teragrid clusters); 30GBmem; 1hr-5hrs Coupled Weather/Fire: 100-1000nodes; 200-400GBmem

34 34 Examples of Areas of DDDAS Impact DDDAS Funded Projects span many areas, including: Physical, Chemical, Biological –Chemical pollution transport, molecular bionetworks, protein folding.. Medical and Health Systems –MRI imaging, cancer treatment, brain seizure control Environmental – Hazard prevention, mitigation, and response –hurricanes, tornados, wildfires, floods, landslides, terrorist attacks Manufacturing, Transportation, Critical Infrastructure systems –Electric power systems, water supply systems, transportation networks and vehicles (air, ground, underwater, space) – condition monitoring, mitigation, … List of Projects in List of Papers/presentations in {Jan 2006 Workshop Report}

35 35 Impact to CyberInfrastructure The CyberInfrastructure that will result when one thinks of the present paradigm of (disjoint) simulations and measurements will be different than the CyberInfrastructure needed to support DDDAS For example, bandwidth requirements, resource allocation and other middleware and systems software policies, prioritization, security, fault tolerance, recovery, QoS, etc…, will be different when one needs to guarantee data streaming to an executing simulation or control of measurement process DDDAS will impact the kind of underlying infrastructure that needs to be provided

36 36 What about Industry Interest Industry has history of –forging new research and technology directions and –adapting and productizing technology which has demonstrated promise Need to strengthen the joint academe/industry research collaborations; joint projects / early stages Technology transfer –establish path for tech transfer from academic research to industry –joint projects, students, sabbaticals (academe industry) Initiatives from the Federal Agencies Cross-agency co-ordination Success stories: VLSI, Networking, and Parallel and Scalable Computing, etc For example: industry is interested in DDDAS

37 37 Summary Thoughts Large scale systems entail: enhanced computation, communication and data management capabilities, in the presence of resource heterogeneity, dynamicity, adaptivity Large Scale Parallel Systems cannot exist in isolation – they will be “nodes” in an (InformationPower)Grid (e.g. TeraGrid) Complexity of applications and platforms presents a significant opportunity for innovative research and technology in systems software (methods & tools) Need to advance the technologies that will automate the mapping of such complex and dynamic applications on complex platforms with multiple and heterogeneous levels of processors, memory, and networks New directions, like DDDAS, create a richer set of challenges and opportunities for novel research and novel capabilities An important item: do we nurture a critical mass of people that will work on these challenges?

Download ppt "1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Clusters, Computational Grids and Beyond CCCGSC 2006."

Similar presentations

Ads by Google