We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published bySkylar Fredricks
Modified over 2 years ago
© 2012 IBM Corporation z/OS New Year's resolutions for Saving CPU Cycles & Improving I/O Utilization February 12, 2013
© 2012 IBM Corporation 2 Session Agenda DB2 and IMS Database and Storage Integration Overview DB2 and IMS System Level Backup Methodologies and Storage System Integration DB2 and IMS Back Ups Using Storage-Based Fast-replication Exposing DFSMShsm Resource Utilization Optimizing Your Batch Window
© 2012 IBM Corporation 3 Database and Storage Administration Trends and Directions Large IMS and DB2 systems require high availability –Fast and non-intrusive backup and cloning facilities are required –Fast recovery capabilities minimize downtime and promote high availability –Most backup, recovery and cloning solutions do not leverage storage- based fast-replication facilities Storage-based fast-replication facilities are under-utilized –Tend to be used by storage organizations –Tend not to be used by database administrators (DBAs) Storage-aware database products allow DBAs to use fast- replication in a safe and transparent manner –Provides fast and non-intrusive backup and cloning operations –Simplifies recovery operations and reduces recovery time –Simplifies disaster recovery procedures
© 2012 IBM Corporation 4 Database and Storage Integration Mainframe Database Systems Storage-Aware Database Tools Application and Database Management Domain Storage Administration and Business Continuity Domain Organizational Integration New Backup Methods New Recovery Strategies Business Recovery Monitoring Cloning Automation Disaster Restart Solutions Source Database Backup, Clone, DR
© 2012 IBM Corporation 5 Host Based Data Copy Options Volume copy options –DFSMSdss (IBM) –FDR (Innovation Data Processing) –TDMF (IBM) –FDRPAS (Innovation Data Processing) Data set copy options –DFSMSdss (IBM) –FDR (Innovation Data Processing) Host-based Copy Process Data copy processes use host based CPU and I/O facilities More costly and slower than storage-based fast replication
© 2012 IBM Corporation 6 What is Storage-based Fast Replication? An instant copy of a volume/data set at a specific point in time –Builds a bitmap to describe the source volume –After the bitmap has been created, the source and target volume data can be used immediately Data movement (CPU and I/O) offloaded to storage processor –Frees up resources on host processor –No host CPU or I/O costs For volume replication a relationship is established between a source and a target –Geometrically similar devices Consistency Groups –Group of volumes copied at exactly the same point in time while maintaining the order of dependent writes Storage Processor-based Copy Process
© 2012 IBM Corporation 7 Advantages Using Storage-Based Fast Replication Fast Copies data instantaneously Provides high availability Provides a consistent copy of production without sacrificing availability Allows clones or recoveries to be available quicker Provides huge cost savings Doesnt use host CPU or I/O resources –Copy process is done in the storage processor >Save CPU and I/O costs Save personnel time 7
© 2012 IBM Corporation 8 Database System Level Backup Overview A Backup or Clone of the entire DB2 or IMS environment at a point in time –Recorded in metadata repositories Leverages storage-based fast replication to drive the volume backup –Backup instantly - performed in seconds –Offloading data copy process to the storage processor saves CPU and I/O resources –Faster than data set copies Backup DB2 and IMS without affecting applications –Backup windows reduced by replacing image copies –Extends processing windows Data consistency ensures data is dependent-write consistent –DB2 suspend, IMS suspend –Storage-based consistency functions –Equivalent to a power failure 8 Source DB2 or IMS Volumes Storage Processor APIs Target Volumes DB2 or IMS System Backup Storage aware DB2 and IMS backup DB2 or IMS
© 2012 IBM Corporation 9 Database System Level Backup Overview Backup validation each time ensures successful recoveries –Insurance that a backup is available Automated backup offload (archive/recall) –Copies system backup from fast replication disk to tape for use at either local or disaster site (or both) Can be used in combination with image copies Tape Processing Storage Processor APIs Storage-Aware Backup and Recovery Offload SLB System Backup Source Database Volumes DB2 or IMS
© 2012 IBM Corporation 10 Benefits of SLB over Image Copies and Change Accums Creating SLB with Fast Replication is equivalent to : –Creating all Image Copies with < 1 second of IMS or DB2 unavailable time –SLB created using storage processor CPU (not Host CPU) –Significant CPU cost savings Guaranteed Recoverability –Validation of IMS and DB2 configuration each time SLB is created Fast restore with Parallel Log Apply –Reduces recovery time and complexity –Executes the restore in parallel with the log apply Change Accumulations may not be needed –System Level Backups can be created frequently –Save host CPU and I/O Significantly reduce costs by using less CPU and I/O resources –Reduce costs to create backups –Save cost by reducing number of image copies needed
© 2012 IBM Corporation 11 SLB Disaster Recovery Benefits Simplifies disaster recovery operations –System level backup for restart –System level backup and roll forward Taking full volume dumps for disaster recovery? System level backups add automation and a meta-data repository Can now use the backup for multiple purposes Basis for tape-based DB2 and IMS coordinated recovery - Restore IMS and DB2 systems back to a transactionally consistent point which is the backup time or end of the last common log Simplify Automate Coordinate
© 2012 IBM Corporation 12 Integrating SLBs into Recovery Using an Intelligent Recovery Manager Recovers application, individual database, or indexes –Using Current, Timestamp, or PITR Application profile is created in advance –Single database or group of databases –Logically related databases and indexes can be included automatically Determines best recovery method –Restores from either IC or SLB –Indexes that can not be restored are rebuilt –Recovery using log apply needs one pass of the logs –Access to DBs is automatically stopped and restarted at end of recovery Storage-based fast-replication performs restore –Performs an instantaneous data set restore process
© 2012 IBM Corporation Customer Experience Using SLB Resource Assessment Tool
© 2012 IBM Corporation 14 Customer Experience EXCP Consumption for Image Copies over 28 day period –Top 5 systems IMS SystemEXCPs 1.IMS1 573,323,342 2.IMS2 549,197,344 3.IMS3 547,836,773 4.IMS4 446,749,090 5.IMS5 263,317,210 DB2 SystemEXCPs 1. DB2188,390,971 2.DB2285,007,495 3.DB2378,792,982 4.DB2453,788,217 5.DB25 34,337,687
© 2012 IBM Corporation 15 Customer Experience Minimizing EXCP Consumption –Product using Fast Replication Technologies –Offloads the backup processing From the CPU to the Storage Processor –Reducing number of EXCPs results in: CPU reduction Elapsed time to execute Frees up resources for other business processing EXCPs consumed today vs. Estimated EXCPs using SLBs IMS DB2
© 2012 IBM Corporation 16 Customer Experience Backup Processing –9 IMS systems More than 60 hours of elapsed time running Image Copy backups –15 DB2 systems More than 57 hours of elapsed time running Image Copy backups
© 2012 IBM Corporation 17 Financials Projected Image Copy vs. SLB Cost Savings for IMS Note: Costs of CPU and EXCPs are agreed upon by Rocket Software and customer. Defaults values were used for the purpose of this assessment. CPU cost per second used is $0.118 and cost per 1000s EXCPs used is $0.04.
© 2012 IBM Corporation 18 Financials Projected Image Copy vs. SLB Cost Savings for DB2 Note: Costs of CPU and EXCPs are agreed upon by Rocket Software and customer. Defaults values were used for the purpose of this assessment. CPU cost per second used is $0.118 and cost per 1000s EXCPs used is $0.04.
© 2012 IBM Corporation 19 Financial Summary Projected Image Copy vs. SLB Cost Savings Summary –IMS –DB2
© 2012 IBM Corporation 20
© 2012 IBM Corporation Exposing DFSMShsm Resource Utilization and Associated Costs
© 2012 IBM Corporation 22 A Look Inside Your DFSMShsm Costs –What resources are used by DFSMShsm to perform scheduled and requested work? –Migration / Recall / Backup / Recycle –Successful vs. Unsuccessful (failures) –Data duplication Efficiency –Where can performance and configuration tuning help? –Reduce failed migrations, improve backup failures Savings –Can the reclaimed resource savings save CPU? –Lost tapes, questionable old DFSMShsm data, failed cycles, thrashing, etc. 22
© 2012 IBM Corporation 23 DFSMShsm Migration Failures Data that wont migrate –HSM attempts to migrate the data sets every day, using both CPU and I/O until the processes fails –This can go on every day for months, even years because the administrator is not aware that its failing –These data sets remain on disk, occupying space that should have been released for new allocations Why wont the data migrate? –Structural errors –Not enough space on ML1 –Unknown DSORG or otherwise not manageable by HSM
© 2012 IBM Corporation 24 Migration Failure Error Summary Example Rc Count Message NO MIGRATION VOLUME AVAILABLE DUPLICATE DSN IN MCDS 16 8 PRIMARY COPY READ ERROR DATA SET IN USE 24 4 DATA SET NOT AVAILABLE FOR MIGRATION NO SPACE ON MIGRATION VOLUME 39 1 RACF PROCESSING ERROR MIGRATION OR DBA DBU FAILED TAPE MIGRATION UNSUPPORTED UNSUPPORTED DS 24
© 2012 IBM Corporation 25 DFSMShsm Backup Failures Data that Fails Backup –HSM attempts to backup the data every day –Sometimes this goes on every day for months, even years because the administrator is not aware that its failing –These data sets may rely on HSM for backup Why data cant be backed up? –Data sets are in use during backup –Unknown DSORG or otherwise not manageable by HSM –Errors in the data set Does this data really need to be backed up by HSM –Are multiple back ups of the data occurring? –Which back up is the right back up?
© 2012 IBM Corporation 26 DFSMShsm Recall Failures Data that Fails Recall –HSM must move the data from ML1 or ML2 storage back to primary DASD –When a recall request fails, the requesting application may fail as well; causing an outage –If the data set cannot be recalled and no backup copy exist; application disruption may occur until the situation is resolved Why data recalls fail? –Data sets are not migrated, not managed by HSM –Users issue multiple recalls for the same data set –Tape volume not available
© 2012 IBM Corporation 27 DFSMShsm Data Thrashing Analysis Data Sets that are Thrashing –Thrashing is data that is migrated and recalled, migrated and recalled, migrated and recalled in a short period of time These data are typically production GDGs that are created earlier in the month and then used again in weekly or monthly processing Thrashing Costs in Terms of CPU –HSM uses CPU and I/O to migrate and recall data; compressing and decompressing data from ML1 The compression/decompression is all CPU Data migrated to ML2 uses both CPU and I/O – the data may be compressed by the hardware but not by DFSMShsm If data on ML2 is being recalled from physical tape, this typically takes longer (wall- clock time) than ML1 A high number of recalls can place a burden on a virtual tape subsystem since the data has been de-staged to physical tape and must be re-staged into the cache –Executing jobs (or TSO sessions) wait for recalls
© 2012 IBM Corporation Managing Aged (Unreferenced Data) in DFSMShsm 28
© 2012 IBM Corporation 29 Retaining Data in DFSMShsm DFSMShsm is a Life Cycle Management System –It makes perfect sense to retain data in DFSMShsm until it expires; that is why we have DFSMShsm! –However, there is a substantial cost associated with it! Where are the costs? –In daily RECYCLE –In the daily backup of the DFSMShsm control data sets –In duplicating ML2 tapes and or a mirrored virtual tape subsystem –In moving the data every 3 years or so to refresh storage media
© 2012 IBM Corporation 30 Cost of Managing Aged Data in DFSMShsm Managing inactive data in DFSMShsm for long periods of time has a cost in terms of daily CPU, I/O and Storage Resources Inactive data is data that is 2 years old or older and has not been recalled (used) in 1 year or more –CPU and I/O to RECYCLE the tapes (recycle typically runs daily) Recycle is the act of deleting expired data and moving non-expired data to another tape –Data Storage Costs DFSMShsm data is typically stored on DASD, physical TAPE or virtual tape –DFSMShsm Backup and Reorganization Costs Every migrated data set has at least 2 CDS records; 3 if VSAM Every data set that is backed up has at least 2 CDS records plus 1 MCC record for each backup copy DFSMShsm Control Data Sets are backed up daily; catalogs are backed up multiple times per day –Duplication of ML2 and Backup Data The cost of storing inactive data in DFSMShsm is further exasperated by the duplication of this data for DR purposes
© 2012 IBM Corporation 31 Cost of Managing Aged Data in DFSMShsm Bottom Line…the costs associated are: –Daily recycle –Daily backup of the control data sets –Daily backup of catalogs –Data duplication (remote mirroring for Business Continuity) Recycle Backups Duplication
© 2012 IBM Corporation 32 In Addition… The majority of customers polled are now using a virtual tape subsystem for DFSMShsm ML2 data –More tape drives available –Faster recalls (typically) –Capability to mirror remotely A virtual tape subsystem has an average life span of just 3 years –Data must be copied from one virtual tape subsystem to another every 3 years Data must be migrated to newer technology when disk is replaced –This adds to the cost of data storage for long term retention Data that needs to be retained longer than three years will outlive the virtual tape its stored on –Data with long term storage requirements must be housed on media that can support the requirement Generally, all tape media used in zOS environments meet these requirements Current tape media has a year life span
© 2012 IBM Corporation 33 If You MUST Keep This Data… Data migrated more than 2 years and has not been recalled are candidates for archival –Possible solution is to use an Archive Manager –Deletes entries from the MCDS, BCDS and Catalog –Improves performance and saves CPU resources Benefits of archiving aged data –HSM MCDS and BCDS record count reduction –DASD space requirements reduction for CDSs and CDS backup copies –Saved CPU and I/O from moving aged from tape to tape during recycle –Tape recycle activity and CPU time reduction –Related data archived together; expires together –Possible catalog record count reduction, catalog backup and CPU time reduction
© 2012 IBM Corporation 34
© 2012 IBM Corporation Optimizing Your Batch Window
© 2012 IBM Corporation 36 Performance Challenges are Increasing Online availability requirement are increasing –High demand for information access and up to the minute data Service Level Agreements are more stringent Determining system-wide impact of application tuning activities is difficult Batch windows are getting smaller –Batch jobs get bottlenecked by extensive I/Os, blocking their ability to run at peak speed –Little time to optimize batch jobs 36
© 2012 IBM Corporation 37 Why is Optimization of I/O Important? Growth and batch processing window constraints Business needs out of alignment with application design Legacy application integration with e-business Data center consolidations Extending business without the need to upgrade systems Time = $$$ Optimization Saves Time 37
© 2012 IBM Corporation 38 Batch Window Constraints Batch processing is composed of: –CPU Cycles –Memory –I/O How can I/O constraints be reduced to improve batch elapsed time? 38
© 2012 IBM Corporation 39 Reducing Batch I/O Constraints Look for an intelligent, intuitive and integrated optimization tool that: –Significantly reduces elapsed times of batch processing –Reduces batch processing requirements –Is storage platform independent –Automatically enhances buffering to improve batch cycles 39
© 2012 IBM Corporation 40 I/O Without Using an Optimization Product Inefficient I/O operations Relying on system defaults Improper tuning Lack of flexibility when change is required from sequential to random access (or vice versa) Low performance System is not utilized to its maximum capacity Buffer Get / Put Program EXCPI/O Small buffer sizeMany I/O operations HOSTDASD 40
© 2012 IBM Corporation 41 I/O Using an Optimization Product Reduces number of I/Os dramatically Automatically adjusts the buffers Increases performance Frees system resources Buffer Program EXCPI/O Large buffer size Fewer I/O operations HOSTDASD No need for application modification Get / Put 41
© 2012 IBM Corporation 42 Optimization Product Functions Automated Batch I/O Tuning Solution –Significantly improve system-wide performance for VSAM and non- VSAM batch processing –Reductions of batch elapsed time in the range of 25-75% –Benefits VSAM, non-VSAM (QSAM, BSAM) and VSAM loads Accomplishes this by: –Reducing CPU overhead associated with I/O (EXCPs) –Exploiting locality of reference principle in real storage Refers to reuse of specific data, and/or resources, within a relatively small time duration –Adapting NSR/LSR Buffering to changes in file processing –Enabling VSAM LSR and Hiperspace for high level code 42
© 2012 IBM Corporation 43 Customer Experience Results of I/O Optimization – Wall Clock Savings Without I/O Optimization With I/O Optimization Percent Improvement VSAM Job100:00: :00: VSAM Job 200:01: :00: VSAM Job 300:01: :00: Non-VSAM Job100:00: :00: Non-VSAM Job200:00: :00: Load Job100:01: :00: Load Job200:00: :00: Load Job300:03: :00:
© 2012 IBM Corporation 44 Customer Experience Results of I/O Optimization – EXCP Savings EXCPs Without I/O Optimization EXCPs With I/O Optimization Percent Improvement VSAM Job 41,457,551110,46192 VSAM Job 5847,28789,24789 VSAM Job 62,589,771334,05887 Non-VSAM Job34,839,7081,995,82558 Non-VSAM Job43,800,7291,454,56061 Load Job 49,498,212227,17797 Load Job 58,665,813205,98197 Load Job 68,694,282184,25797
© 2012 IBM Corporation 45
Break Time Remaining 10:00. Break Time Remaining 9:59.
13:00 Clock will move after 1 minute PPT – VCIC Timer 15.ppt.
DLMSO Classroom Timer Select a time to count down from the clock above 60 min 45 min 30 min 20 min 15 min 10 min 5 min or less.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
Symantec 2010 Windows 7 Migration EMEA Results. Methodology Applied Research performed survey 1,360 enterprises worldwide SMBs and enterprises Cross-industry.
Symantec 2010 Windows 7 Migration Global Results.
I n t e g r i t y - S e r v i c e - E x c e l l e n c e Headquarters U.S.A.F. 1 Commodity Councils 101 NAME (S) SAF/AQCDATE.
Time for a BREAK! You have 45 Minutes. Time Left 44.
© SafeNet Confidential and Proprietary Administering SafeNet StorageSecure Smart Card Module 3: Lesson 5 SafeNet StorageSecure Storage Security Course.
×1= 9 4 1×1= 1 5 8×1= 8 6 7×1= 7 7 8×3= 24.
Mark A. Magumba Storage Management. What is storage An electronic place where computer may store data and instructions for retrieval The objective of.
C Copyright © 2005, Oracle. All rights reserved. Practice Solutions.
© 2006 DataCore Software Corp DataCore Traveller Travel in Time : Do More with Time The Continuous Protection and Recovery (CPR) Solution Time Optimized.
Threads, SMP, and Microkernels Chapter 4 1. Process Resource ownership - process includes a virtual address space to hold the process image Scheduling/execution-
Troubleshooting Startup Problems Lesson 6. Objectives 2.
Simulations The basics for simulations. Simulation is a way to model random events, such that simulated outcomes closely match real-world outcomes. By.
Murach’s OS/390 and z/OS JCLChapter 16, Slide 1 © 2002, Mike Murach & Associates, Inc.
1 Dr. Scott Schaefer Least Squares Curves, Rational Representations, Splines and Continuity.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
Silberschatz, Galvin and Gagne Operating System Concepts Chapter 10: Virtual Memory Background Demand Paging Process Creation Page Replacement.
Oracle Advanced Compression – Reduce Storage, Reduce Costs, Increase Performance Session: S Gregg Christman -- Senior Product Manager Vineet Marwah.
4.1 © 2004 Pearson Education, Inc. Exam Managing and Maintaining a Microsoft® Windows® Server 2003 Environment Lesson 4: Organizing a Disk for Data.
1 Disk Based Disaster Recovery & Data Replication Solutions Gavin Cole Storage Consultant SEE.
1 INSTALLING "WINDOWS 8.." SO THAT IT CREATES A LOCAL USER ACCOUNT.
Jane Reid, BSc/IT DB, QMUL, 11/3/02 1 Lecture plan Transaction processing Concurrency control Recovery techniques.
1 Oracle Database 11g Advanced Compression Option.
CALENDAR NEW CALENDAR
Information Means The World.. Enhanced Data Recovery Agenda EDR defined Backup to Disk (DDT) Tape Emulation (Tape Virtualization) Point-in-time Copy Replication.
Petersons Practice AP Exam Cengage Learning Infotrack Database.
5 minutes 4 minutes 3 minutes 2 : 00 1 : 59 1 : 58.
1 BACKGROUND ALARM SYSTEM OF ZXJ10 Training Center Zhongxing Telecom Pakistan (Pvt.) Ltd.
CONTROL VISION Set-up. Step 1 Step 2 Step 3 Step 5 Step 4.
Copyright © Action Works 2008 All Rights Reserved - Photos by David D. Kempster 1.
User Friendly Price Book Maintenance A Family of Enhancements For iSeries 400 DMAS from Copyright I/O International, 2006, 2007, 2008, 2010 Skip Intro.
11 1 Chapter 11 Database Performance Tuning and Query Optimization Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
IBM Software Group ® Backup & Recovery. IBM Software Group | DB2 Information Management Software Agenda Backup Types Backup Process Model Backup.
3 : 00 2 : 59 2 : 58 2 : 57 2 : 56 2 : 55 2 : 54.
The 5S numbers game. 1. This sheet represents our current work place. Our job during a 20 second shift, is to strike out the numbers 1 to 49 in correct.
13 Copyright © 2005, Oracle. All rights reserved. Monitoring and Improving Performance.
1 Budapest University of Technology and Economics, BME, 1872 Budapest University of Technology and Economics, BME, 1872 Happy New Year 2012.
Solving a Sudoku Puzzle By Marlene Walther ATeacherFirst.com.
Mr. Deven Patel, AITS, Rajkot. 1 Process Description and Control Chapter 3.
BMU - E I 1 Development of renewable energy sources in Germany in
1 Copyright © 2013 Elsevier Inc. All rights reserved. Chapter 6 Processes and Operating Systems.
Processes Management. 3.2 Silberschatz, Galvin and Gagne ©2011 Operating System Concepts Essentials – 8 th Edition Process Management A process is a program.
1 Input/Output Chapter Principles of I/O hardware 5.2 Principles of I/O software 5.3 I/O software layers 5.4 Disks 5.5 Clocks Terminals.
1 tRelational/DPS Overview. 2 ADABAS Data Transfer: business needs and issues tRelational & DPS Overview Summary Questions? Demo Agenda.
1 Click here to End Presentation Software: Installation and Updates Internet Download CD release NACIS Updates.
© 2016 SlidePlayer.com Inc. All rights reserved.