Building PetaScale Applications and Tools on the TeraGrid Workshop December 11-12, 2007 Scott Lathrop and Sergiu Sanielevici.

Slides:

Advertisements

Similar presentations

Audrey Desjarlais, Signetwork Coordinator Survey Findings SPDG Initiative Goals SPDG Initiative Outcomes.

Advertisements

Program Management Office (PMO) Design

Extreme Scalability RAT Report title=RATs#Extreme_Scalability Sergiu Sanielevici,

User Support Coordination Objectives for Plan Years 4 and 5 (8/1/2008 to 3/1/2010) Sergiu Sanielevici, GIG Area Director for User Support Coordination.

Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models.

High Performance Computing Course Notes Grid Computing.

4.1.5 System Management Background What is in System Management Resource control and scheduling Booting, reconfiguration, defining limits for resource.

CoP : Getting Started 4. Engaging the “Right” Stakeholders: Building the “community” 2.

Understanding Operating Systems 1 Overview Introduction Operating System Components Machine Hardware Types of Operating Systems Brief History of Operating.

Business Intelligence Dr. Mahdi Esmaeili 1. Technical Infrastructure Evaluation Hardware Network Middleware Database Management Systems Tools and Standards.

Chapter 4: Beginning the Analysis: Investigating System Requirements

Action Research In Organizational Development. Action Research Coined by Kurt Lewin (MIT) in 1944 Reflective process of progressive problem solving Also.

Chapter 4: Beginning the Analysis: Investigating System Requirements

US NITRD LSN-MAGIC Coordinating Team – Organization and Goals Richard Carlson NGNS Program Manager, Research Division, Office of Advanced Scientific Computing.

Module 3: Business Information Systems Chapter 11: Knowledge Management.

STRATEGIES INVOLVED IN REMOTE COMPUTATION

Organization Mission Organizations That Use Evaluative Thinking Will Develop mission statements specific enough to provide a basis for goals and.

Brad Armosky, TACC Shawn T. Brown, PSC Scott Lathrop, UC/ANL Laura McGinnis, PSC TeraGrid’08 Conference, Las Vegas, Nevada June 11, 2008.

© Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March

Effective User Services for High Performance Computing A White Paper by the TeraGrid Science Advisory Board May 2009.

DIVISION OF TECHNOLOGY, INDUSTRY AND ECONOMICS / UNEP/ SETAC Life Cycle Initiative: Focus of the Life Cycle Inventory program by Guido.

Scaling to New Heights Retrospective IEEE/ACM SC2002 Conference Baltimore, MD.

HPC University Training Meeting Welcome!! March 26-27, plementation.

Demystifying the Business Analysis Body of Knowledge Central Iowa IIBA Chapter December 7, 2005.

What is a life cycle model? Framework under which a software product is going to be developed. – Defines the phases that the product under development.

Environment for Information Security n Distributed computing n Decentralization of IS function n Outsourcing.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.

2Object-Oriented Analysis and Design with the Unified Process The Requirements Discipline in More Detail  Focus shifts from defining to realizing objectives.

Programming Models & Runtime Systems Breakout Report MICS PI Meeting, June 27, 2002.

SAN DIEGO SUPERCOMPUTER CENTER Impact Requirements Analysis Team Co-Chairs: Mark Sheddon (SDSC) Ann Zimmerman (University of Michigan) Members: John Cobb.

Chapter 4 Realtime Widely Distributed Instrumention System.

Coordinating the TeraGrid’s User Interface Areas Dave Hart, Amit Majumdar, Tony Rimovsky, Sergiu Sanielevici.

Incident Management PCCYFS June 27, 2007 Incident Management for the Office of Children, Youth and Families.

AREVA T&D Security Focus Group - 09/14/091 Security Focus Group A Vendor & Customer Collaboration EMS Users Conference September 14, 2009 Rich White AREVA.

1 Metrics for the Office of Science HPC Centers Jonathan Carter User Services Group Lead NERSC User Group Meeting June 12, 2006.

1 PY4 Project Report Summary of incomplete PY4 IPP items.

Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”

Headline in Arial Bold 30pt HPC User Forum, April 2008 John Hesterberg HPC OS Directions and Requirements.

Data Demo: Atmospheric Visualization Collection Christopher Klaus 05/18/05.

Core Banking Transformation: A Roadmap to a Successful Core Banking Product Implementation - PMI Virtual Library | | © 2008 Kannan S. Ramakrishnan.

 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.

Federal Aviation Administration By: Giles Strickler, UCS Program Manager Procurement Policy (AJA-A11) Date:September 22, 2010 Unified Contracting System.

ST-09-01: Catalyzing Research and Development (R&D) Funding for GEOSS Florence Béroud, EC Jérome Bequignon, ESA Kathy Fontaine, US ST Kick-off Meeting.

08/05/06 Slide # -1 CCI Workshop Snowmass, CO CCI Roadmap Discussion Jim Bottum and Patrick Dreher Building the Campus Cyberinfrastructure Roadmap Campus.

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Cyberinfrastructure Requirements and Best Practices Lessons from a study of TeraGrid Ann Zimmerman.

1 IMPLEMENTATION STRATEGY for the 2008 SNA OECD National Accounts Working Party Paris, France 4 to 6 November 2009 Herman Smith UNSD.

Nature Reviews/2012. Next-Generation Sequencing (NGS): Data Generation NGS will generate more broadly applicable data for various novel functional assays.

Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.

SOFTWARE DESIGN AND ARCHITECTURE LECTURE 13. Review Shared Data Software Architectures – Black board Style architecture.

Microsoft Management Seminar Series SMS 2003 Change Management.

Joint Meeting of the AUS, US, XS Working Groups TG10 Tuesday August 3, hrs Elwood II.

1 COMPUTER SCIENCE DEPARTMENT COLORADO STATE UNIVERSITY 1/9/2008 SAXS Software.

Computational Science & Engineering meeting national needs Steven F. Ashby SIAG-CSE Chair March 24, 2003.

Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.

Firmware - 1 CMS Upgrade Workshop October SLHC CMS Firmware SLHC CMS Firmware Organization, Validation, and Commissioning M. Schulte, University.

Education, Outreach and Training (EOT) and External Relations (ER) Scott Lathrop Area Director for EOT Extension Year Plans.

Tool Integration with Data and Computation Grid “Grid Wizard 2”

David Foster LCG Project 12-March-02 Fabric Automation The Challenge of LHC Scale Fabrics LHC Computing Grid Workshop David Foster 12 th March 2002.

Continual Service Improvement Methods & Techniques.

TeraGrid’s Common User Environment: Status, Challenges, Future Annual Project Review April, 2008.

1 Industry Advisory Council’s Enterprise Architecture Shared Interest Group (IAC EA SIG) Collaborative Approach to Addressing Common Government- Industry.

Advanced User Support in the Swedish National HPC Infrastructure May 13, 2013NeIC Workshop: Center Operations best practices.

PEER 2003 Meeting 03/08/031 Interdisciplinary Framework Major focus areas Structural Representation Fault Systems Earthquake Source Physics Ground Motions.

1 Open Science Grid: Project Statement & Vision Transform compute and data intensive science through a cross- domain self-managed national distributed.

November | 1 CONTINUING CARE COUNCIL Report to Forum Year

Education, Outreach and Training (EOT) Scott Lathrop Area Director for EOT January 2010.

Queen’s University Belfast – Institute of Lifelong Learning 4th ALPINE Project Meeting: University of Tartu, Estonia Adults Learning and Participation.

TeraGrid’s Process for Meeting User Needs. Jay Boisseau, Texas Advanced Computing Center Dennis Gannon, Indiana University Ralph Roskies, University of.

Work Plan Work Plan Management (Document 21)

Overview of Workflows: Why Use Them?

Presentation transcript:

Building PetaScale Applications and Tools on the TeraGrid Workshop December 11-12, 2007 Scott Lathrop and Sergiu Sanielevici

Workshop Activities Attendees: ~ 80 participants; 19 peta-apps and SDCI awards; TeraGrid staff; NSF Pre-workshop survey to identify key topics Discussions of goals, needs, and challenges Recommendations for action Post-workshop survey You have draft of workshop report

Algorithms and Techniques Visualization Parallel compilers/languages Parallel solvers and AMR Multi-core programming Workflow Performance tools Communications overhead Parallel I/O Discussion Topics

Recommendations 1) Support a Forum for exchanging experiences among petascale efforts - TG task –Support wiki for information exchange - TG task –Utilize TeraGrid ’08 to bring projects together to share findings - TG task –Regularly solicit needs and findings from peta-apps and SDCI projects - TG task –Invite all PetaApps and SDCI projects to participate in a joint scaling paper - Community Task - proposed –Facilitate scientific mentoring to support petascale development and dissemination among the community - TG and Community Task - near term

Recommendations 2) Provide a repository of petascale info - TG task –Track 2 and 1 detailed tech. specifications - TG task –Mechanisms for making community contributed readers, clients, etc. widely available - TG task –PetaApps algorithmic needs - Community Task - started –Collect performance data, anecdotes, and community wisdom - TG and Community Task - will take time –Libraries, tools, workflow tools - Characterize advantages/strengths to help users select tools - Community Task - will take time –Exemplars, case studies, best practices, etc. - TG and Community Task - will take time –Create “workflow ready” indicator list (e.g. 100K files, 100K jobs, data transfer, visualization); notify projects when workflow tech may be appropriate - TG and Community Task - will take time

Recommendations 3) Enact policies that support advancements in petascale computing - TG Tasks –Improve allocations process - TG Task - evolving –Develop special allocation and job scheduling policy for debugging massively parallel applications - TG task »Sufficient access to test and verify scaling to progressively larger Track 2 and 1 machines –Provide traces (e.g. disk failures to understand failing patterns and to develop proactive schemes) - TG task –Single interface - unified login, password, no certificates - TG task - will take time –Transparent single file system - TG task - long-term –Address scheduling capabilities - TG task - long-term »Dynamic processor allocation for variable-load runs »Dynamic and predictive load balancing and repartitioning

Recommendations 4) Provide petascale support services to advance petascale applications - TG tasks –Provide consulting and mentoring - TG task »Review ASTA support requests and recommend projects that would benefit from the emerging technology »Scientific expertise/support/mentoring to suggest algorithm alternatives and/or adjustments; tools; workflows –Expand training efforts - TG task »HPC University training to get new people involved »Provide expert guidance on effective use of petascale architectures, tools, workflow –Provide petascale related documentation and guides - TG task »How to compile, link, submit, and monitor jobs »Example application readings and problem solutions to show advantages of various petascale tools, workflows »Provide non-technical information to potential petascale users as to advantages of petascale technology

Recommendations 5) Identify and support key software developments –Development of scalable components: subject to Usability*Reusability=constant. - Community Task - will take time –Identify kinds of tools needed to solve current problems - Community Task - will take time »download data »push data back and forth »calculate and move images –Conduct verification and validation of Petascale applications - Community Task - will take time –Support for asynchronous as well as synchronous visualization - TeraGrid Task –Easily customizable clients for visualization applications, for specific domains - TeraGrid Task

Planning Additional Community Events Convene regular events for sharing experiences- TG and Community Tasks – TeraGrid ‘08 - June 9-13, 2008 – SC’08 - November, 2008 Planning for Petascale Workshop Series – Target Community Peta-apps, SDCI, and other interested parties Applications scientists, computer scientists, vendors – Update of applications challenges – Updates of new and emerging tools and capabilities – Identify challenges, barriers, and gaps – Recommendations for further action

Q&A Session Which are roles for TeraGrid? for the community? for NSF? Which recommendations are most critical? A number of actions depend on active community engagement. How can we best facilitate community engagement and sharing?