1. CD FY09 Tactical Plan Review FY09 Tactical Plans for Database Services J.Trumbo Sept. 24, 2008

2. CD FY09 Tactical Plan Review 2 FY09 Tactical Plan for Database Services CD DocDB Document 2889-v1 2009 Tactical Plan Database Services Tactical Plan Leader: J.Trumbo Organizational Unit home: LSCS/DBI/DBA

3. CD FY09 Tactical Plan Review 3 FY09 Tactical Plan: Strategies and Goals Tactical Plan Strategies –Develop stronger expertise and support for open source databases, reducing the dependency on Oracle, with a goal of reducing development and support costs. Tactical Plan Goals –Continue to provide a stable, professional operation of laboratory scientific and information systems databases, providing maximum uptime and technical support.

4. CD FY09 Tactical Plan Review 4 FY09 Tactical Plan: Objectives Tactical Plan Objectives 1. Database & Systems Operations Support running experiments at appropriate levels. 2. Database Support. Promote Postgres and Oracle as a 1st tier database and MS-SQL and MySQL as second-tier databases. Support production, integration and development on all database platforms. Configure, tune, test, and maintain databases. 3. Agile response to new and rapid shifts of responsibilities and demands accompanying the end of Run II data taking and reduction of effort in the Run II experiments.

5. CD FY09 Tactical Plan Review 5 FY09 Tactical Plan: Objectives 4. Kerberize Oracle using Advanced Security Option. 5. Maintain expertise for new features. Work with databases on vm environments, oracle rac testing, etc. 6. Implement Oracle Internet Directory (OID). 7. Complete upgrade to Oracle 10 for the information systems databases. 8. Upgrade information systems hardware for databases, including major apps, computer security, 24.7 applications and nimi development. 9. Continued Deployment of the 3par San technology. 10. Ramp SDSS support down to maintenance levels.

6. CD FY09 Tactical Plan Review 6 FY09 Tactical Plan: Past Action Items Resolution of Past Action Items: SDSS Planning hardware/software layouts for the next DR7 and final DR8 releases. Obtaining new machines and licenses promptly. Reconfigured number of existing machines to meet our hardware requirements for the future maintenance of the SDSS machines to keep less machines in the rack. Implemented 2 mirror sites for DR5 Data Release in the Libraries of JHU and University of Chicago. SDSS team working close with team of the librarians to setup SDSS help desk support using new web application Question Point. SDSS DBA team successfully completed transition CasJobs web service from SQL2K to SQL2K5. Completed incremental loading from DR6 to DR7.1 (BestDR7.1, TargDR7.1, SegueDR7.1 - total more than 6TB data) and released it to the SDSS collaboration on February 14 - two weeks early. Working close with team of developers from JHU and Princeton to implement and test new code related to schema modifications for DR7.2 with major changes in data model and merging SegueDR7.1 to BestDR7.2. Completed loading 3TB data into new database Stripe82, this is a new addition to the SDSS databases in respond to the SDSS scientists to merge stripe 82 runs into one database. Successfully upgraded 7TB RunsDB database from SQLServer2000 to SqlServer 2005 environment. Working constantly on SDSS documentation and improving monitoring tools to maintain stable production environment.

7. CD FY09 Tactical Plan Review 7 FY09 Tactical Plan: Past Action Items Resolution of Past Action Items Open Source: To be clear, points below include effort from sysadmins as well as dbas! Established best practices, standards, how tos for open source implementations. Cross training as much as possible. Established postgres general purpose boxes for dev/int/prd. Installed several users including content_mgmt, ilc ground motion and edu office lasso project dbs. Took mgmt of mysql general purpose boxes. they need work to upgrade and bring to stds, work is pending. Migrating support of gratia db to dba. Benchmarked MyISAM vs InnoDB engines for Gratia. Evaluated several backup products for mysql operating under InnoDB engine. Established nova dev/int/prd postgres boxes. Took over support of NIMI. Modifying to standardized installs, upgrade os and db. Continuing cross training of group. Attempted to set up an onsite postgres class for developers and dbas, but furloughs disrupted attendance planning. Assisted minerva with mysql and postgres installs on general purpose boxes. Established the Content Mgmt postgres database dev/int/prd. Still not quite transitioned minos production mysql db to dba.

8. CD FY09 Tactical Plan Review 8 Service Activity: Database Support, including Oracle, SqlServer & Open Source Support running experiments at appropriate levels, insure a stable, performant, and smooth running environment. Goals Related to this Activity –Install, configure, and test new databases, perform version upgrades of existing databases. –Administer and operate production and development database information systems. –Participate in a 24x7 on-call rotation schedule for production issues. –Perform database backups and data restores. –Disaster recovery for databases. –Replication –Diagnose problems, and monitor database performance. –Perform tuning of SQL queries. –Install and configure Oracle Internet Directory (OID) for Oracle v10 dbs.

9. CD FY09 Tactical Plan Review 9 Service Activity: Database Support Key Metrics 1.Uptime Statistics 2.http://www-css.fnal.gov/dsg/external/database_uptime/ 3.http://www-css.fnal.gov/dsg/internal/SDSS/metrics/metrics.htm Issues and Risks (specific to this activity, includes allocation impact) 1.Hardware failure 2.Aging Hardware

10. CD FY09 Tactical Plan Review 10 Project Activity: Kerberosize Oracle with ASO Continue Kerberos for Oracle Implementation Goals Related to this Activity continue oracle aso kerberos project implement Kerberos authentication for DBA tasks install aso for users as needed define an action plan to support aso Key Milestones 1.Implementation into production databases. 2.Metrics, % of kerberized dba tasks, % of kerberized user tasks Issues and Risks (specific to this activity, includes allocation impact) 1.Manpower

11. CD FY09 Tactical Plan Review 11 Project Activity: Complete Upgrade to Oracle 10 Upgrade Infrastructure Databases (miscomp) to Oracle 10 Goals Related to this Activity os upgrade Information systems databases upgrade 10gR2 Key Milestones 1.Os upgrade, databases upgraded. 2.Metrics, % of instances upgraded Issues and Risks (specific to this activity, includes allocation impact) 1.Application readiness

12. CD FY09 Tactical Plan Review 12 Project Activity: Upgrade Infrastructure Hardware Upgrade infrastructure with new hardware Goals Related to this Activity plan separation of information systems apps, including 24.7 instances and misprod1. plan for data movement to new databases Implement new databases, including oid repository and new remedy. Key Milestones 1.New hardware implementation successes 2.Metrics, % of upgraded Issues and Risks (specific to this activity, includes allocation impact) 1.manpower

13. CD FY09 Tactical Plan Review 13 Project Activity: Migrate Databases to 3Par Move databases to 3Par array Goals Related to this Activity –Move dev/int cdf offline databases to 3par to replace aging and unsupported disk array. –Move dev/int d0 luminosity databases/snapshots to 3par to allow for a full and complete testing environment. (complete) –Move fcdfora6 replica to 3par –Assess and move if possible the cdf online dev/int environments of aging array. Key Milestones 1.Number of successful moves successes 2.Metrics, uptime, $ saving on time and materials repairs to unsupported hardware.Issues and Risks (specific to this activity, includes allocation impact) Issues and Risks (specific to this activity, includes allocation impact) 3par readiness

14. CD FY09 Tactical Plan Review 14 Project Activity: SDSS to Maintenance Mode Ramp down SDSS to maintenance levels. Goals Related to this Activity –Load and Release DR7 –Load and Release DR8 –Loading Runs Database –Standardizing the Idrea Monitoring. –Support the transition from SDSS operations to long-term maintenance. –Consultancy JHU and UC for hosting DR5 –UC DR7 implementation Key Milestones 1.Meet release dates 2.Metrics, less time spent on SDSS, uptime stats Issues and Risks (specific to this activity, includes allocation impact) Manpower, $

15. CD FY09 Tactical Plan Review 15 FY09 Resource Request, Preliminary Allocation Level 0 Activity: Application Support FY09 Tactical PlanActivity Level 1 FTE Request At Activity Level 1 M&S Request At Activity Level 1 FTE Allocation At Activity Level 1 M&S Allocation At Activity Level 1 Database Services D0 (on&offline) CDF (on&offline) Minos DES Standby database Nova Gratia Infrastructure CMS Gen.Support, Open Src 1.25.25.1 1.0.05.500 Oracle License Travel/Edu/Conf New Hdw/Licenses 000000 370,000 79,250 93,000 SDSS complete dr7 SDSS maintenance.25.50000 Total5.75$542.2500$0

16. CD FY09 Tactical Plan Review 16 Impact of Preliminary Allocation SDSS will be annually funded $60k for continuing support. In addition 3 fte months are allotted for Oct/Nov, Oct (2 ftes) completing public release, Nov. (1fte) tying up loose ends. Additional funding needs to be procured if DR7 at U of C is to be completed, needing hardware and manpower.

17. CD FY09 Tactical Plan Review 17 Template Guidance: (not part of the presentation) Activity = Activity Level 1 or lower explicitly listed in Tactical Plan Activities have been classified in Tactical Plans as either being more Project-like or Service-like Projects: finite defined lifetime, defined deliverable. Includes R&D work. A.k.a Upgrades, … Services: same operation or task repeatedly. A.k.a. Operations, Maintenance, Support, … Whether an activity is a project or service determines which format slide to use later. Terminology Obligation Budget: Allocated budget. This may differ from the budget request. YTD Obligations: Fully incurred expenses. Does not include RIPs, Req.s in CD, etc. Current FY08 Forecast: Estimate of all obligations to be incurred by the end of FY08. M&S: Equipment + Operating. Operating includes Travel, Material Purchases, Professional Services (contractors), Maintenance (licenses), and Other. RIP: Requisitions in Progress