Ruth Pordes Executive Director University of Washingon Seattle OSG Consortium Meeting 21st August University of Washingon Seattle

OSG Consortium, 2/21/06 2 Thank you to our hosts.

OSG Consortium, 2/21/06 3 The Context OSG has few face to face meetings. The main ones are the semi- annual Consortium meetings. So we use this meeting to a) Review what has happened over the past six months. b) Hear from our contributors and partners on some aspects of mutual interest. c) Share and communicate our goals and plans. d) Move the technical program of the OSG forward. The goal is that we all listen, discuss, complain and construct. Goal is to make the OSG the “A” Grid.

OSG Consortium, 2/21/06 4 This week we will.. Hear from our Contributors about the Science from current use of the OSG. We will Engage with new communities and partners. We will hear from our partners about Campus and peer Grid Infrastructures. Get updates of the status of OSG, the Facility and Education programs; We will cover much about Security --- Risk, Responsibility, Tracking for OSG itself, Virtual Organizations, Sites and interfaces to peer grids ---. We will have sessions where people share their experiences and learn from each other --- Site, Facility and VO Administrators, Users etc. We will have a few Demonstrations tomorrow. Discuss and decide on aspects of the short term Technical Program: Make decisions for the OSG software release and associated VDT releases. Plan the next Steps in: Information Services, Information Management, Data Management, Workload Management, Education, Inclusion, Communication.

OSG Consortium, 2/21/06 5 First the recap..

OSG Consortium, 2/21/06 6 What we have not gotten done since the last Consortium meeting  Site Space management so that VOs and Sites have robust, managed shared storage and space services.  Simple metrics of use and accounting not deployed uniformly across sites.  Fixed severe lack of robustness of Authz components (especially GUMS) and simplicity of use.  VOs knowing and negotiating with Sites to really support their applications.  Changed the perception that OSG is “mainly physics”. Why is this? GADU only has had one major run; suite of math jobs from Football pool problem; Nanohub jobs undergoing troubleshooting.  …

OSG Consortium, 2/21/06 7 What we have Accomplished  Sustained and operated the OSG to the benefit of >15 user organizations. (during ramp down of funded Grid projects and uncertainty in future).  Increased the robustness and scalability of the Compute Element head node by implementing a managed-fork queue and “nfs-less” shared disk areas.  Made effective contributions to WLCG (grid and application) Service Challenges as the US common infrastructure for ATLAS and CMS  Released 3 versions of the Virtual Data Toolkit and released OSG  Increased the average number of jobs on OSG by 1/3 (2000-> 3000) and the number of Sites - Compute and Storage - by a few %. Increased LIGO and STAR jobs a few.  Written a comprehensive Security Risk Assesment of the OSG and an initial security plan.  Achieved Federation of local-area Campus Grids bridging to wide-area cyber-infrastrastructures.  Run the 4th successful Summer Grid Workshop with >45 students who actually adapted their own applications to run across multiple remote sites.  Released OSG RA.

OSG Consortium, 2/21/ Virtual Organizations - participating Groups 3 with >1000 jobs max. (all particle physics) 3 with max. (all outside physics) 5 with max (particle, nuclear, and astro physics)

OSG Consortium, 2/21/06 9 GridCat continues to list the Resources

OSG Consortium, 2/21/06 10 More Documentation

OSG Consortium, 2/21/06 11 And..oh yes.. The OSG proposal was finished and submitted to DOE and NSF in March & February. In June funding the agencies asked for revised budget with more than a 25% reduction to $6M/year: ~33 FTE/year of effort. We are going ahead with our planning on the assumption that funding is coming in the next few months. The project will be accountable in its promises and deliverables to the funding sources; we expect to be reviewed on our project plan and milestones ~3 months after funding. We will have Expectations of the project, which will be documented, tracked and planned across the funded effort. For the next 5 years we must be serious about delivering the fundamental Production Quality Facility for our Users.

OSG Consortium, 2/21/06 12 the OSG Project and its part in the OSG Consortium

OSG Consortium, 2/21/06 13 The OSG Project will deliver Maintainenance of an expanding Production Distributed Facility which provides core capabilities in Operations, Security, Software Releases, the Virtual Data Toolkit, Troublshooting and supports the Engagement of new communities to use the OSG. Education, Training and Outreach with a core focus on expanding the number, location and scope of the Grid Schools and web based training. User Support, Service and System Extensions for increased scale, performance and capability of the common services for the stakeholder VOs. OSG Staff: executive director, project consultant, administrative help and Technical documentation for training system administrators, users and VO administrators.

OSG Consortium, 2/21/06 14 OSG Project Execution Plan (PEP) - FTEs 33Total FTEs 3.0Staff 9.0Extensions in capability and scale. 1.0Facility management 2.0Education, outreach & training 2.0Engagement 6.5Software release and support 4.5Security and troubleshooting 5.0Facility operations FTEs

OSG Consortium, 2/21/06 15 OSG PEP - Organization

OSG Consortium, 2/21/06 16 Part of the OSG Consortium Contributors Project

OSG Consortium, 2/21/06 17 OSG PEP - High Level Milestones 2006Q3Release OSG software stack version Q3Project baseline review 2006Q4Sign off on OSG Security Plan. 2006Q4Meet operational metrics for Q1Accounting reports available for users and resource owners. 2007Q2Production use of OSG by one additional science community. 2007Q2OSG-TeraGrid: software releases based on same NMI software base. 2007Q2Release OSG software version 0.8.0: Complete extensions for LHC data taking. 2007Q2Support for ATLAS and CMS data taking. 2007Q31 year Project Review. 2007Q4Meet 2007 deliverables as defined by science stakeholders. 2007Q4Meet operational metrics for Q4Release OSG software version Q2Production use of OSG by 2 additional science communities. 2008Q3OSG-TeraGrid: production service interoperation. 2008Q32nd year Project Review. 2008Q4Meet 2008 deliverables as defined by science stakeholders. 2008Q4Meet operational metrics for Q2Support for all STAR analysis (10,000 jobs/day). 2010Q1Support for data taking with order of magnitude increase in LIGO sensitivity.

OSG Consortium, 2/21/06 18 OSG PEP - Security, Safety, Risk Management The OSG Facility assesses, monitors and responds to security issues. The Security Officer coordinates these. Each site, user and administrator has responsibility for local security and reporting incidents that may occur. The OSG will have a comprehensive security plan modeled on the NIST process. While Environment, Safety and Health (ES&H) remains the responsibility of the owners of the resources made accessible to the Open Science Grid, the project organization will pay attention to ESH issues during travel, meetings and OSG activities

OSG Consortium, 2/21/06 19 OSG Project Effort Distribution Year 1 Each Institution will have a signed Statement of Work (MOU). Each individual will submit open monthly written reports. Finance Board will review the accounts and deliverables. Executive Board will review the plans and achievements. Activities will be covered by the Project Plan and WBS. Effort distribution will be reviewed and potentially modified each year.

OSG Consortium, 2/21/06 20 Must Support LHC and LIGO Scaling circa Data distribution must routinely exceed 1 GB/Sec at ~10-20 sites Workflows must support >10,000 batch jobs per client Jobs/Day must exceed 20,000 per VO with >99% success rate. Accessible Storage >~10PB. Facility Availability/Uptime must be >99.x% with no single points of failure.

OSG Consortium, 2/21/06 21 Year 1 OSG WBS and the Plans Bakul Banerjee - Project Consultant, working with all coordinators to complete plans and schedules. Plan for initial review at Thursdays Council meeting. Ready for review by Science Advisory Council and/or external reviewers in ~3 months.

OSG Consortium, 2/21/06 22 Operations, Security, Troubleshooting, Software Expect to “Re-Plan”. All plans allow for unanticipated opportunities. Make Plans Help not hinder.

OSG Consortium, 2/21/06 23 Continued focus on OSG Core Competencies Integration: Software, Systems, Virtual Organizations. Operations: Common Support & Grid Services. Inter-Operation: Bridging Administrative & Technical Boundaries. with Integrated Security Operations and Management. with Validation, Verification and Diagnosis at each step.

OSG Consortium, 2/21/06 24 Reminder of the S/W Stack and Deployment Life-Cycle VDT increases the effectiveness of Condor and Globus by integrating them with the other services needed for fully functional Cyber-environments. OSG project funding for VDT will enable more storage and data management services to be included in the future.

OSG Consortium, 2/21/06 25 OSG Facility Operation: Operations, Maintenance Engagement Support OSG Facility Operation: Operations, Maintenance Engagement Support OSG Extensions: Requirements development, & testing on parochial grids Resources Providers (Sites) Applications OSG Facility Provisioning: VDT, Integration, Validation, System Integration Testbed OSG Facility Provisioning: VDT, Integration, Validation, System Integration Testbed Release External Projects: Development & Research Ready OSG Project Does Not Do Software Development Condor, Globus, EGEE-JRA1, dCache, SRM, US LHC S&C, LIGO PIF, Security for Open Science Center for Distributed Science etc.

OSG Consortium, 2/21/ Join OSG 1. VO Registers with with Operations Center.User registers with VO. 2. Sites Register with the Operations Center. 3. VOs and Sites provide Support Center Contact and join Operations groups. The OSG VO 1. A VO for individual researchers small groups. 2. Managed by the OSG itself. 3. Where one can learn how to use the Grid! Core Operations and Common Support

OSG Consortium, 2/21/06 27 Grid of Grids - from local to global CS/IT Campus Grids Science Community Grid e.g LIGO, STAR, D0 … National & International Infrastructures for Science e.g. Teragrid, EGEE, NAREGI… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG…

OSG Consortium, 2/21/06 28 Grid of Grids - OSG is one grid among many CS/IT Campus Grids Science Community Grid e.g LIGO, STAR, D0 … National & International Infrastructures for Science e.g. Teragrid, EGEE, NAREGI… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG… National & International Infrastructures for Science e.g. Teragrid, EGEE, NAREGI… National & International Infrastructures for Science e.g. Teragrid, EGEE, NAREGI… National & International Infrastructures for Science e.g. OSG, Teragrid, EGEE, NAREGI… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG… Campus & Regional Infrastructures e.g. CrimsonGrid, GLOW, NWICG… Users must be able to operate transparently across Grid boundaries. OSG program of work focuses on Interoperability and Bridging of data and jobs across these boundaries.

OSG Consortium, 2/21/06 29 Continuing with this meeting

OSG Consortium, 2/21/06 30 Welcome to those from near and far Most far -- Bob Jones - Director of Enabling Grids for EsciencE- II, CERN. Simon Lin - Director Computing Center Academia Sinica, Taiwan. Kazushige Saga - NAREGI, Tokyo Institute of Technology. Dave Kelsey - Chair of Joint Security Working Group and Rutherford Appleton Laboratory, England. Sergio Andreozzi - gLITE-JRA1 and INFN Most near -- Univeristy of Washington, Seattle and nearby - David Baker, Richard Coffey, James DeRoest, Tony Hey, Margaret Romine, Oren Sreenby, Gordon Watts

OSG Consortium, 2/21/06 31 I look forward to a thoughtful and productive meeting and discussions with you all. OSG is For the Community, By the Community, Throughout the Community.