March 2014 Open Science Grid Operations A Decade of HTC Infrastructure Support Kyle Gross Operations Support Lead Indiana University / Research Technologies.

Slides:

Advertisements

Similar presentations

Background Chronopolis Goals Data Grid supporting a Long-term Preservation Service Data Migration Data Migration to next generation technologies Trust.

Advertisements

Campus High Throughput Computing (HTC) Infrastructures (aka Campus Grids) Dan Fraser OSG Production Coordinator Campus Grids Lead.

Open Science Grid June 28, 2006 Bill Kramer Chair of the Open Science Grid Council NERSC Center General Manager, LBNL.

Open Science Grid Use of PKI: Wishing it was easy A brief and incomplete introduction. Doug Olson, LBNL PKI Workshop, NIST 5 April 2006.

Joining the Grid Andrew McNab. 28 March 2006Andrew McNab – Joining the Grid Outline ● LCG – the grid you're joining ● Related projects ● Getting a certificate.

April 2009 OSG Grid School - RDU 1 Open Science Grid John McGee – Renaissance Computing Institute University of North Carolina, Chapel.

Welcome to HTCondor Week #14 (year #29 for our project)

M.Goldberg/NSFOSGSep 18, 2012 The Open Science Grid 1.

Open Science Ruth Pordes Fermilab, July 17th 2006 What is OSG Where Networking fits Middleware Security Networking & OSG Outline.

US NITRD LSN-MAGIC Coordinating Team – Organization and Goals Richard Carlson NGNS Program Manager, Research Division, Office of Advanced Scientific Computing.

OSG Area Coordinators Meeting Operations Rob Quick 2/22/2012.

MyOSG: A user-centric information resource for OSG infrastructure data sources Arvind Gopu, Soichi Hayashi, Rob Quick Open Science Grid Operations Center.

OSG Grid Workshop in KNUST, Kumasi, Ghana August 6-8, 2012 following the AFRICAN SCHOOL OF FUNDAMENTAL PHYSICS AND ITS APPLICATIONS July 15-Aug 04, 2012.

Ways to Connect to OSG Tuesday afternoon, 3:00 pm Lauren Michael Research Computing Facilitator University of Wisconsin-Madison.

Key Project Drivers - FY11 Ruth Pordes, June 15th 2010.

Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.

Experiment Requirements for Global Infostructure Irwin Gaines FNAL/DOE.

OSG Area Coordinators Campus Infrastructures Update Dan Fraser Miha Ahronovitz, Jaime Frey, Rob Gardner, Brooklin Gore, Marco Mambelli, Todd Tannenbaum,

OSG Area Coordinators Meeting Operations Rob Quick 2/22/2012.

OSG Operations Rob Quick July 10th, 2012 OSG Staff Retreat.

OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.

Open Science Grid For CI-Days Elizabeth City State University Jan-2008 John McGee – OSG Engagement Manager Manager, Cyberinfrastructure.

SG - OSG Improving Campus Research CI Through Leveraging and Integration: Developing a SURAgrid-OSG Collaboration John McGee, RENCI/OSG Engagement Coordinator.

A short introduction to the Worldwide LHC Computing Grid Maarten Litmaath (CERN)

OSG – the next few years Vicky White, Fermilab OSG Council Meeting March 15, 2013.

CSIU Submission of BLAST jobs via the Galaxy Interface Rob Quick Open Science Grid – Operations Area Coordinator Indiana University.

Use of Condor on the Open Science Grid Chris Green, OSG User Group / FNAL Condor Week, April

DOSAR Workshop, Sao Paulo, Brazil, September 16-17, 2005 LCG Tier 2 and DOSAR Pat Skubic OU.

Miron Livny Center for High Throughput Computing Computer Sciences Department University of Wisconsin-Madison Open Science Grid (OSG)

Open Science Grid  Consortium of many organizations (multiple disciplines)  Production grid cyberinfrastructure  80+ sites, 25,000+ CPU.

Discussion Topics DOE Program Managers and OSG Executive Team 2 nd June 2011 Associate Executive Director Currently planning for FY12 XD XSEDE Starting.

Evolution of the Open Science Grid Authentication Model Kevin Hill Fermilab OSG Security Team.

10/24/2015OSG at CANS1 Open Science Grid Ruth Pordes Fermilab

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE – paving the way for a sustainable infrastructure.

Interoperability Grids, Clouds and Collaboratories Ruth Pordes Executive Director Open Science Grid, Fermilab.

Open Science Grid Monitoring and Information Services Interoperability Breakout Session Shaowen Wang August 29, 2005 OSG Blueprint Meeting.

Data Intensive Science Network (DISUN). DISUN Started in May sites: Caltech University of California at San Diego University of Florida University.

The Swiss Grid Initiative Context and Initiation Work by CSCS Peter Kunszt, CSCS.

Grid Operations Lessons Learned Rob Quick Open Science Grid Operations Center - Indiana University.

OSG Consortium Meeting (January 23, 2006)Paul Avery1 University of Florida Open Science Grid Progress Linking Universities and Laboratories.

Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.

The OSG and Grid Operations Center Rob Quick Open Science Grid Operations Center - Indiana University ATLAS Tier 2-Tier 3 Meeting Bloomington, Indiana.

OSG Area Coordinators Meeting Operations Rob Quick 1/11/2012.

Operations Activity Doug Olson, LBNL Co-chair OSG Operations OSG Council Meeting 3 May 2005, Madison, WI.

ATP Future Directions Availability of historical information for grid resources: It is necessary to store the history of grid resources as these resources.

Federations: The New Infrastructure Speaker Name Here Date Here Speaker Name Here Date Here.

Open Science Grid in the U.S. Vicky White, Fermilab U.S. GDB Representative.

Accelerating Campus Research with Connective Services for Cyberinfrastructure Rob Gardner Steve Tuecke.

Area Coordinator Report for Operations Rob Quick 4/10/2008.

1 An update on the Open Science Grid for IHEPCCC Ruth Pordes, Fermilab.

OSG Area Report Production – Operations – Campus Grids June 19, 2012 Dan Fraser Rob Quick.

Operations Area Coordinator Report. 31 Jan Overview Operations Current Initiatives  RSV Version 2  New Probes, Easier Configuration, Improved.

User Support of WLCG Storage Issues Rob Quick OSG Operations Coordinator WLCG Collaboration Meeting Imperial College, London July 7,

Opensciencegrid.org Operations Interfaces and Interactions Rob Quick, Indiana University July 21, 2005.

1 Open Science Grid: Project Statement & Vision Transform compute and data intensive science through a cross- domain self-managed national distributed.

Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.

OSG Facility Miron Livny OSG Facility Coordinator and PI University of Wisconsin-Madison Open Science Grid Scientific Advisory Group Meeting June 12th.

Certificate Security For Users Obtaining and Using Your Personal Certificate using the OSG PKI Kyle Gross – OSG Operations Support Lead Elizabeth Prout.

OSG Production Foundations for 2M+ Hours/Day April 9, 2014 Rob Quick With Help from Shawn McKee and Chander Seghal.

Galaxy based BLAST submission to distributed high throughput computing resources Rob Quick and Soichi Hayashi Open Science Grid Operations Indiana University.

Grid Colombia Workshop with OSG Week 2 Startup Rob Gardner University of Chicago October 26, 2009.

OSG Operations – Lessons Learned CHEP 2010, 18 October 15:10 (Asia/Taipei) – Room 2, BHSS OSG Operations – Lessons Learned CHEP 2010, 18 October 15:10.

Ways to Connect to OSG Tuesday, Wrap-Up Lauren Michael, CHTC.

] Open Science Grid Ben Clifford University of Chicago

Some Odds and Ends About Computational Infrastructure Kyle Gross – Operations Support Manager - Open Science Grid Research Technologies.

Regional Operations Centres Core infrastructure Centres

Operations Interfaces and Interactions

Scaling Science Communities Lessons learned by and future plans of the Open Science Grid Frank Würthwein OSG Executive Director Professor of Physics UCSD/SDSC.

Open Science Grid Progress and Status

Leigh Grundhoefer Indiana University

Presentation transcript:

March 2014 Open Science Grid Operations A Decade of HTC Infrastructure Support Kyle Gross Operations Support Lead Indiana University / Research Technologies Presented by Soichi Hayashi

March 2014 Topics What is OSG? History of OSG OSG Components Services hosted in/out of OSG Operations Center Future of Operations Center Healty Community is the Key

March 2014 Open Science Grid: Mission Statement “The Open Science Grid (OSG) advances science through open distributed computing. The OSG is a multi-disciplinary partnership to federate local, regional, [scientific] community and national Cyberinfrastructures To meet the needs of research and academic communities at all scales.”

What is OSG (and what is not?) and scale of OSG (amount of cpu hours provided, number of sites, etc..) Open Science Grid

March 2014 What is Open Science Grid? Provides service & support to resource providers & scientific institutions 122 member sites OSG does not *own* resources Distributes common software to resource providers and users Forming a larger “distributed” computing grid infrastructure. Designed for High-throughput computing jobs 14 million cpu hours / week

March 2014 iVDGL, GridPP, GriPhyN, Grid2003, Grid3, Trillium Designed for ATLAS experiments They needed to join clusters together to process BigData 7 Original Partners: - FNAL - BNL - LBNL - University of Wisconsin - Indiana University - University of Chicago - RENCI In the beginning…

March 2014 OSG Timeline 2004 Consortium Formed Ribbon Cutting LHC Becomes Operational Operations Services SLA Adoption 20000th Ticket Opened July 4 Announcement of Higgs Discovery OSG Starts PKI Service OSG 1ExtensionOSG Prime 10000th Ticket Opened 100th VO “Darkside” Joins OSG

March 2014 OSG Components Infrastructure Monitoring Accounting Support Interoperability Resource Provisioning

March 2014 GOC Services Services hosted at Indiana University Communications Hub Trouble Ticket (GOC Ticket) Ticket Exchange 24x7 Phone and Availability OSG Public Key Infrastructure OSG BDII OIM – OSG Topography Database

March 2014 GOC Services Services hosted at Indiana University (cont.) MyOSG – OSG Information Portal OASIS (CVMFS) OSG Home Page OSG-XSEDE JIRA VOMS (MIS/CSIU) Others..

March 2014 Distributed Operations Fermilab −Gratia Accounting −OSG VOMS University of California San Diego −Glidein Factory Service/Support University of Chicago −OSG Connect & Campus Infrastructure Communities University of Nebraska - Lincoln −Provides various UI for Gratia accounting information. Others..

March 2014 The Future of OSG Operations Match users with appropriate resources they need Reach out to other scientific & academic communities Host new services used by multiple campus communities Evolve multilevel monitoring to help debugging and problem resolution Continue to provide excellent OSG Infrastructure Services and Support

March 2014 Community Building Continue to grow a stronger sense of community for users, resource providers, and OSG staff Built in continued quasi-daily one-on-one interactions Done in dialogues not monologues Caring is necessary for community building “What should young people do with their lives today? Many things, but the most daring thing is to create stable communities... ” - Kurt Vonnegut

March 2014 Open Science Grid Operations OSG Operations Team