Presentation on theme: "1 Deployment of Exalogic And Exadata Increases Business Efficiency A success story at GIA."— Presentation transcript:
1 Deployment of Exalogic And Exadata Increases Business Efficiency A success story at GIA
2 Krish Lakshmin Director, Global Databases and Application, GIA Saroj Dash Architect, Engineered Systems TCS Soumendra Dash Delivery Lead, Engineered Systems, TCS With you Today
3 Agenda 1 1 About GIA & TCS 2 2 Business Benefits 3 3 Challenges & Best Practices 4 4 Q & A Features of Engineered Systems used 7 7 Technology Landscape Business & Technology Goals
4 About GIA and TCS Established in 1931, GIA is the world’s foremost authority on diamonds, colored stones, and pearls. A public benefit, nonprofit institute, GIA is the leading source of knowledge, standards, and education in gems and jewelry Industry. TATA Consultancy Services Limited (TCSL) is a multinational information technology (IT) service, consulting and business solutions company headquartered in India. It is a subsidiary of TATA group. TCS is a leader in the global marketplace and among the top 10 IT services provider in the world.
5 Understanding Global GIA’s IT needs GIA is a global Institution having nine Grading Laboratories around the globe. Its major computing needs are supported by two major production data centers and two DR data centers. GIA’s Grading Data is very global as it is created, distributed and shared among all its labs to support various grading services for the global customer base. A combination of high OLTP plus resource heavy back end processing supported by a detailed workflow engine. GIA’s global customers demand fast stone turnaround time as trade operates with very low profit margin among its supply chain. High turnaround time is understandably unacceptable. One of GIA’s missions is to empower trade and general public alike with as much grading data as possible in a timely fashion for making informed decisions. Finally, Diamonds are forever and hence data archival is out of question. We need to keep all our data from day one.
6 NGGS Initiative Goals for NGGS Scalable and efficient Next Generation Grading System (NGGS) to help GIA service its growing customer base with quicker turnaround time and to address ongoing application performance issues. Highly Scalable platform with less or no interruption of service to support rapidly growing global expansion. Performance. Performance for high throughput for both internal and external users. Platform Consolidation and Standardization. Address Itanium end of life support issue from Oracle. Highly available architecture like ExaData to address Business Continuity / Downtime / Stability issues.
7 Selection of ExaData and ExaLogic and Timeline We compared and deliberated on different hardware platforms like IBM, HP and Oracle. No comparable hardware were available to match the configurations and features of ExaData and ExaLogic. We ran and tested our database copy in Oracle’s Engineered Systems lab in Santa Clara for performance of our most used queries and processes. Very impressive performance numbers just out of the box with no tuning of any sort. Implementation Timeline. Engineered System was Implemented in Carlsbad during July, 2013 timefreame Engineered System was implemented in Mumbai during Nov, 2013 timeframe. DR Databases in NY and Hong Kong was implemented using ODA in early Upgraded to Quarter Rack from 1/8 ExaData in July, Upgraded to Quarter Rack from 1/8 ExaLogic in August 2014 Both the upgrades (scalability feature) went without any hiccup.
8 Mumbai East Data Center Mumbai East Data Center Hong Kong East Disaster Recovery Hong Kong East Disaster Recovery New York, West Disaster Recovery New York, West Disaster Recovery Carlsbad West Data center Carlsbad West Data center GIA Data Centers.
9 Engineered System Implementation
11 A Paradigm shift in Data Integration and availability today at GIA ExaData helped us enable and integrate our live grading service data with many customers using web services APIs 24x7. Large customers today run their customer portals accessing data from our database live. We integrate 80-90% of our data with all our customers as part of our mission to help trade and public with all the data that they need to be well informed. Due to our old hardware constraints we were quite restricted on how much of API service data we can enable in fear of production performance 24x7. From 2-4 customers in the past, today we serve close to 300 customers using web services without impacting production performance. Business expanded their operation hours from one shift to three shifts which is in all practical purposes a 24x7 operation around the globe. Struggled with balancing performance and data availability during backup and other database maintenance window. In Exadata, now business runs 24x7 with no impact on its performance while we carry out all database maintenance activities like backups, stat collection and merge activities behind the scenes. Additional reporting and compliance databases for business operations.
12 Top Five -1- Business Benefits Performance improved 30x without any change to Query or Process. We can run records query against any size stone without restriction. Performance in Records Query. An integrated and critical process step in the grading workflow. Each stone data is compared against archived data (~ 160 Million) for stone identification and fraud detection at multiple steps. Poor Query/database performance resulting in long grading delays.
13 Top Five -2- Business Benefits Service Interruptions. Very Frequent Application Java run-time Error and Restart of Application JVM and Service Interruptions. Upgrade to ExaLogic from x86 based Platform. Not a single instance of java run-time Application error or JVM restart ever since we upgraded to ExaLogic. No more unable allocate memory error.
14 Top Five-3- Business Benefits Data Consolidation between databases. A SQL Merge Process of about 200G every four hours. Process time reduced to 30 minutes from 8 hours between databases. This helped us service our global customers with improved current stone status, order additional services, reorder services and Workflow status etc. Limits on how many stones we can transfer and merge between databases were removed.
15 Top Five -4- Business Benefits Server Consolidations. ExaData for Database Servers Consolidation and ExaLogic for Application Servers Consolidation. Consolidated three database servers into one ExaData System (1/8 of a Rack). Consolidated Nine Application Servers into one ExaLogic System (1/8 of a Rack) By End of 2014 we would have consolidated all our ERP Databases into this single box. A single box will cater to all of GIA’s production database needs.
16 Top Five -5- Business Benefits RMAN Backup time reduced from 5 hours to 30 minutes. Export Backup time reduced from 6 hours to less than one hour. Statistics collection gets completed in 10 minutes from 6-8 hours Backup and Maintenance, 24x7 Business Operation Longer backup time and maintenance window was impacting 24x7 business operation requirement, application and database performance. RMAN Backup time reduced from 5 hours to 30 minutes. Export Backup time reduced from 6 hours to less than one hour. Statistics collection gets completed in 10 minutes from 6-8 hours Additional reporting and compliance databases for data mining and intelligence. Business operations continues without any impact from these activities.
17 Features of ExaData and ExaLogic used in GIA Grid Link Database Connection Fast connection failover Runtime connection load balancing SCAN Address “WriteBack” Flash Cache Technology In release , FlashCache is “WriteThrough” by default Smart Flash cache provides the enhanced read and writes performance. Exadata storage software version is the minimum version required allowing writes to go into Smart Flash cache. This means database writes first will happen on Flash Cache and database will gives acknowledgement. Storage Cell Software writes them back onto hard disk Use of SDP InfiniBand Listener in ExaData High throughput Performance Use of ZFS Storage over InfiniBand Network Database Backup completes 10X faster
18 Features of ExaData and ExaLogic used in GIA Use of ZFS Storage for Database Backup ZFS Storage Snapshot Database as a Service( DBaaS) initiative in GIA Multi-tenancy Features as part of GIA Roadmap.
19 3X Response time Records Query 30X better Backup time 10X better Batch Transfers 4X better User Experience on ExaData and ExaLogic User Experience 300 min 30 min 3:36:31 sec 0:06:15 sec 2400 Stones Stones
20 Current Architecture for Business Continuity ExaData ExaLogic Archive Logs written to ZFS Storage Cross Cabled ExaData ExaLogic Building C Building A Standby ExaData is used for Business Continuity. Business Continuity and Disaster Recovery are different for GIA. Standby ExaData is used for Business Continuity. Business Continuity and Disaster Recovery are different for GIA. The same ZFS Storage mounted on other ExaData
21 Current Architecture for Disaster Recovery ExaData ExaLogic Cross Cabled Oracle Database Appliance (ODA) Carlsbad New York ODA (Oracle Database Appliance) is used for Disaster Recovery. Business Continuity and Disaster Recovery are different for GIA. ODA (Oracle Database Appliance) is used for Disaster Recovery. Business Continuity and Disaster Recovery are different for GIA. Log Shipping Data Guard Standby Database
22 Key Activities Database Migration (Linux-Itanium) to Oracle Exadata (Linux). Application Migration (Oracle Application Server) to Exalogic. Install and Configure Exalogic server. Database upgrade from 10g ( ) to 11g ( ). Setup Backup and Recovery using ZFS storgae. Setup Oracle DataGuard. Performance Tuning.
24 Capacity Planning Capacity PlanningReport MigrationApproachReport MigrationScript Exadata Migration ToolKit Usage Auto Data Collection –Host Information –Database Parameters Total Core Total RAM Max CPU Memory utilization Disk Type Disk Redundancy Auto Data Collection –Host Information –Database Parameters Total Core Total RAM Max CPU Memory utilization Disk Type Disk Redundancy Input: Capacity Planning Report Choose Database Migration Approach Report Generated Input: Capacity Planning Report Choose Database Migration Approach Report Generated Input: Migration Approach Input Parameters Generate Migration Scripts Input: Migration Approach Input Parameters Generate Migration Scripts Fine Tuning Database Performance Capacity Planning Data Collection Migration Approach Database Migration Post Migration
25 Architecture before Engineered Systems Operating System: Linux Itanium 11iv3 Database Server: Oracle Database 10g version Application Server: Oracle AS 10g
26 Engineered System Implementation
Approach & Methodology for Migration Creating New Tablespace on Oracle Exadata Storage Server Oracle Logic Standby Oracle Streams and Oracle Data Pump Oracle Data Pump ASM Redundancy Physical Standby Database Recovery Manager Transportable Database Transportable Tablespace TCS chooses Logical Approach - Oracle data pump. This strategy has following pros and cons: This is simpler solution because data is not changing during migration but it incurs more downtime. No need to upgrade the source database. Platform migration is supported. Oracle data pump provides the network import feature that eliminates the need to stage outside of the database. When using this feature with large partitioned tables, investigate using the QUERY parameter manually separate the data being imported across several jobs so that all available computing capacity can be leveraged. Logical Approach Physical Approach TCS Recommendation
28 Snap clone – Sun ZFS Storage – GIA Roadmap Activities : Clone the production source (standby database) to a file system a NAS/ZFS Cancel the Managed Recovery Take a snapshot of the mounted database in NAS/ZFS Clone the Database from the Snapshot using NAS clone feature Mount the Cloned database to the target server Create the database using the Clone file system for One Instance Using the same Snapshot Business Benefit : Huge Storage Savings Clone time Savings Activities : Clone the production source (standby database) to a file system a NAS/ZFS Cancel the Managed Recovery Take a snapshot of the mounted database in NAS/ZFS Clone the Database from the Snapshot using NAS clone feature Mount the Cloned database to the target server Create the database using the Clone file system for One Instance Using the same Snapshot Business Benefit : Huge Storage Savings Clone time Savings
29 DBaaS Setup and Enable – GIA Roadmap Register Storage Server Define Roles and Users Install Management Agent on Unmanaged Hosts Configure Privilege Delegation Setting Configure the Software Library Set Up Provisioning Credentials Provision Oracle Database Software Configure the Listener Snap Clone RMAN Backup DBCA Template Schema as a Service PDB as a Service End Create Database Create Container Database Super Administrator Cloud Administrator Super Administrator Cloud Administrator Performed by Self Service Administrator Plug-ins Required to Enable Database as a Service (DBaaS) Defining Roles and Assigning Users EM_CLOUD_ADMINISTRATOR EM_SSA_ADMINISTRATOR EM_SSA_USER Provision Database Software on PaaS Create Container Database (12c) Configure Oracle Listener Register & Manage Storage Servers Setup the Database Cloud Self Service Portal Use DBaaS to Provision Databases: Snap Clone RMAN Backup DBCA Template PDBs Metering and Charge back
30 Challenges & Best Practices JAR Compatibility Issues (ADF, commons etc.) JVM compatibility Issues (jrockit v/s Sun) Heap Parameter tuning (more memory required especially due to 64 bit) http parameter tuning (session time out, client connections) Code specific issues due to changing library JAR (WebLogic parser is strict parser, hence java scripts needs to be revisited, required code change) Application specific parameter (weblogic.xml, weblogi-application.xml for session parameters, file size, filters, mime type etc.) Analyze existing AWR reports to isolate the critical tables and queries import using sorted order based on the AWR analysis Write back flash cache to absorb random reads SDP listener between exadata and exalogic NFS over infiniband for high speed connectivity for Backups as well as imports. RAC gridlink for RAC aware connectivity Weblogic clustering to be explored for even better session availability and resilience. Typical Challenges Best Practices