Teradata Leaders in Enterprise Data Warehousing

Teradata Leaders in Enterprise Data Warehousing
John Tulley Vice President, Teradata Canada Office:

NCR Corporate Overview
2004 Revenue by Business Unit Fortune 500 company Global operations in more than 100 countries & territories 28,500 employees 2004 Revenue $5.984B >51% revenue growth Teradata Data Warehouse Retail Solutions Financial Worldwide Customer Services Systemedia

Top Industry Leaders Rely on Teradata
Teradata Top 10 Leading industries Banking Government Insurance & Healthcare Manufacturing Retail Telecommunications Transportation Logistics Travel World class customer list More than 800 customers Over 1200 installations Global presence Over 100 countries 4,000 world-wide professionals dedicated to data warehousing 80% of Top 10 Global Telco Firms 60% of Top 10 Most Admired Global Companies 60% of Top 10 Global Airlines Represent x of Top 10; 50% of Top 10 Global Retailers 50% of the Top 10 Transportation Logistic Firms FORTUNE Global Rankings, July 2005

The Teradata Difference
What We Do…. Enterprise data warehouse Windows 2003/Unix/Linux scales from Intel laptop to MPP Analytic capabilities transform data into information. Extreme high availability Industry leader in analytical applications Integration with SAP, Siebel, Hyperion Partnerships include Accenture, Bearingpoint, CAPGemini, Deloitte, EDS, Lockheed Martin Strong customer references All we do is Data Warehousing!

Teradata - the recognized leader in data warehousing and high-performance decision analytics. ….Gartner ASEM IBM S/390 OS/390 DB2 EEE Sun Enterprise Solaris Oracle HP HP9000 HP-UX Oracle IBM SP RS/6000 AIX DB2 EEE Compaq Alpha Tru64 Oracle Generic Intel IA-32 Win2000 SQL Server Unisys ES7000 Win2000 SQL Server Teradata Data Mgmt. Data Admin. Scalability and Suitability Concurrent Query Mgmt. DW Track Record Query Perform. Source: Gartner ASEM Ratings 2004 Worst Best

Industry Leadership Recognition
Gartner - “Dominant Lead” – 5th Consecutive Year “DBMS is surely the place where NCR Teradata sets the gold standard. As in previous years, the Teradata score was 98%, leaving little scope (and need) for improvement.” Gartner's [Application Server Evaluation Model] ASEM Data Warehouse Server Update, A. Butler, K. Strange, J. Enck, M. Chuba, November 2004 Teradata[database management system] DBMS capabilities remain unchallenged by its competitors in the market.” Gartner’s Magic Quadrant for Data Warehouse DBMSs, 2004, Kevin H. Strange, June 2004 “Teradata continues to drive a strong vision.” Gartner Research, MarketScope: Customer Relationship Marketing, 1Q04, G. Herschel, J. Radcliffe, Feb 2004 Gartner Dataquest recognized Teradata as the growth leader in the RDBMS market, with above market growth of 17.4%. 2005 Teradata is rated “Positive” in Gartner’s MarketScope for Campaign Management, the highest rating awarded 2005 META Group “Teradata has displayed unmatched (but often copied) strength of vision and focus in the [enterprise data warehouse] EDW market.” METAspectrum Market Summary, Enterprise Data Warehouse METAspectrumSM Evaluation, 2004 If this slide is used, the full analyst reports must be available to the audience.

Industry Awards and Recognition - 2005
BI Excellence Award Sponsor: Gartner Group Continental Airlines - winner Cardinal Health - finalist Technology Leadership Award Sponsor: Frost & Sullivan Teradata selected for Leadership Award – CRM Analytics TDWI Best Practices Award sunrise TDC Switzerland AG – winner - Customer Relationship Management 1to1 Impact Award Sponsor: Peppers & Rogers Continental Airlines recognized as Technology Optimization winner Editors’ Choice Awards Sponsor: Intelligent Enterprise Teradata selected for the “Dozen” Most Influential BI Companies Winner, Customer Analytics category NEXUS Awards Sponsor: New Zealand Direct Marketing Association Bank of New Zealand, silver award - data mining & analytics; bronze award - data management NEXUS Awards Winner of Gartner BI Excellence Award to be announced March 9th at Gartner BI Summit.

Government Agencies with Teradata Presence
US Air Force US Navy US Transportation Command Defense Commissary Agency Army, Air Force Exchange Intelligence Community US Postal Service Italian Post Office Dept. of Justice Dept. of Housing and Urban Development Dept. of Agriculture Arizona, Iowa, Florida, Texas, Illinois, New York, Utah, Michigan RAMQ – Quebec Australian Tax Office South African Tax Office

Teradata Solutions Methodology
Project Management Teradata Solutions Methodology Strategy Research Analyze Design Equip Build Integrate Manage Opportunity Assessment Enterprise Assessment Business Value EDW Roadmap Data Mapping Application Requirement Infrastructure & Education Logical Model Custom Component System Architecture Test Plan Package Adaptation Education Plan Support Management Hardware Platform Operational Mentoring Software Platform Technical Education User Curriculum Information Exploitation Physical Database Operational Applications ECTL Application Backup & Recovery Value Assessment User Training Production Install Components for Testing Initial Data System Test Acceptance Testing Availability SLA HW/SW Upgrade System Performance System DBA Help Desk Business Continuity Capacity Planning Solution Architect Data Migration Information Sourcing Technology Neutral Services The phases are broken down into specific services. With each service is a methodology definition of tasks, work products, job role assignments, and tools. The methodology is highly modular to allow it to be adapted to different data warehouse situations. In some cases, the Client or another vendor may have performed some of the services, so we may not need to execute all services if there is existing work product we can utilize. In other cases, the vendor of a data warehouse based package may perform those services most closely aligned to their package. {click} The methodology doesn’t become Teradata specific until we reach Design: Systems Architecture. The identification of business opportunities, requirements, data models, and analysis of infrastructure are vendor neutral. The methodology is iterative. After completing one data warehouse increment, we can loop back and add additional content to the data warehouse. We have built in formal technical reviews to each service to catch flaws in the analysis, design, and construction before they become major problems. We have evolved the methodology over 16 years, making changes and enhancements to the methodology to avoid problems. With Teradata Solutions Methodology, you get to benefit from other’s experiences: both good and bad. {click} Teradata’s success is the combination of hardware, software and methodology

Data Warehouse Needs Will Evolve
ACTIVATING MAKE it happen! Query complexity grows Workload mixture grows Data volume grows Schema complexity grows Depth of history grows Number of users grows Expectations grow OPERATIONALIZING WHAT IS happening? PREDICTING WHAT WILL happen? Event-Based Triggering Takes Hold ANALYZING WHY did it happen? Workload Complexity REPORTING WHAT happened? Batch Ad Hoc Analytics Analytical Modeling Grows Increase in Ad Hoc Analysis Continuous Update/Short Queries Event-Based Triggering Primarily Batch & Some Ad Hoc Reports Data Sophistication

Enterprise Analytical Topologies
Data Mart Centric Virtual, Distributed, Federated Hub-and- Spoke Data Warehouse Enterprise Data Warehouse Sources Users Marts Sources Users Middleware Sources Users DW Marts Sources Users DW ODS Independent Data Marts Leave Data Where it Lies Dependent Data Marts Centralized Integrated Data With Direct Access Pros Easy to Build Organizationally Easy to Build Technically No need for ETL No need for separate platform Allows easier customization of user interfaces & reports Enterprise view Design consistency & data quality Data reusability Cons Business Enterprise view unavailable Redundant data costs High ETL costs High App costs High DBA and operational costs No ETL Meta data issues Network bandwidth and join complexity issues Only viable for low volume Business Enterprise view challenging Data latency ODS duplication Requires vision Requires Data Owners to willingly participate Virtual DW is getting a lot of hype lately because of IBM’s pitching it. One customer wanted to do reference lookup/distributed query brining looking up HR data to supplement some info to decide if it were releavant to bring data into the warehouse permanently… This is different that distributed DW…. The data isn’t instantiated, nor does it need to be. When you get into calcuations and complex analysis on the data, the sematic would immediately come up making it invalid for anything more.

Typical Data Warehouse Architecture
What’s wrong with this picture? 3. The solution is too complex. Every line on the chart represents an ETL process that requires $$ for Life Cycle Maintenance Transaction Systems 1. There are too many copies of the data. Will they all be the same? Operational Data Stores Central store, Hub, Clearing house 4. The solution is too expensive. There are numerous components that lead to increased costs. Costs often hidden in distributed organization. 2. There is too much latency - too long to get the data to the people who need it. Everyone sees different inconsistent points in time Data Marts

Teradata’s Enterprise Data Warehouse An Integrated, Centralized Data Warehouse Solution
Middleware/Enterprise Message Bus Metadata Logical Data Model Physical Data Base Design Enterprise, System, & Database Management Business & Technology – Consultation Support & Education Services Transactional Users Transactional Data Data Transformation Optional ETL Hub Operational Data Store (ODS) Optional Optional ELT Single version of data “Enterprise” Data Warehouse Data Replication Logical (Views) Application Dimensional Co-Located Dependent DM Data Marts Optional Virtual Views Decision Users Strategic Users Tactical Users Reporting OLAP Users Data Miners Event-driven/ Closed Loop

TERADATA is an Open System
Virtually any application or middleware framework can be integrated with TERADATA !!! Messages WEB JMS JMS JSP IIOP ASP JAVA EJB TAP Appl CORBA .NET JDBC JDBC JDBC ODBC OLE-DB Message Bus As you would expect, any system of this level of maturity must be an open system. TERADATA is committed to supporting data interchange with mainframes. TERADATA is committed to supporting open standards as they emerge. TERADATA TERADATA Utilities Adapter(s) Publish & Subscribe TERADATA Utilities Adapter(s) Queues

Teradata’s Real-Time Enterprise Reference Architecture
Legacy Environment Enterprise Users — (Browsers and/or Portal) Legacy Environment C/S EDI Consumers Suppliers Internal Partners EDI C/S WAN / VAN Internet / Intranet WAN / VAN Transactional Services Analytic & Decision Making Services Enterprise Message Bus NW MSG-MW MSG-MW MSG-MW MSG-MW MSG-MW MSG-MW NW ASP / JSP TX1 APPL TX2 APPL TX3 APPL TX4 APPL Strategic APPL Tactical APPL BI APPL BI APPL Service Brokers DA-MW DA-MW DA-MW DA-MW DA-MW DA-MW DA-MW DA-MW QD QD MSG-MW MSG-MW MSG-MW I’d like to review with you Teradata’s Real-Time Enterprise Reference Architecture but before we start, what is the first thing you notice about this slide besides all the confusing lines and colors? The answer is that it’s one of the first charts from Teradata that doesn’t have a big, fat disk right smack in the middle of the slide representing the center of the data warehouse universe. This doesn’t mean that Teradata isn’t the center of the universe when representing the Enterprise Data Warehouse, as it’s still the best platform for an integrated, centralized Enterprise Data Warehouse. However, as we extend our analytical capabilities to support both strategic and tactical queries to support both strategic and operational decision making, we need to articulate how we integrate into your enterprise architecture. Event Notification Business Rules Event Detection EDW — A EDW — B DA-MW DA-MW DA-MW RS OLTP1 OLTP2 OLTP3 OLTP4 RDBMS Based Event Processing Business Process Automation Streaming Batch Transactional Repositories Data Acquisition & Integration Analytic & Decision Making Repositories

Enterprise Application Integration
Teradata Active Data Warehouse in action Front Line Base Supply Secure Wireless DOD Supplier Warfighter Support Strategic & Tactical Queries 5.Warfighter receives alert via Secure Blackberry, adjusts Battle Plans to align with rush replenishment 1.Continuous Transaction feeds on supplies usage 4. and or DOD Vendor notified and reorders Enterprise Application Integration Web Services Secure DOD Network Web- Sphere Tibco (EAI) .NET Secure DOD Network Business Services OLAP Queries Intel Agents Rules Engine Event Engine 2. Conditioning & Loading of trans data 3.Stored Procedures trigger based event detection sends alert to Warfighter, Warfighter Support, & DOD Supplier via MSTR Narrowcaster Ascential Informatica Information Exchange MQ Adapter T-Pump, MQ Adapter TERADATA Stored Procedures Q Tables UDF, Triggers Fast Export Legacy Systems Direct Data Access Data Acquisition T-Pump, MQ Adapter Fast Load, Multi Load Transactional Environment Decision Making Environment

So what is Teradata ? Obviously the world has changed much in the past 25 years. In business, however, profitable growth has remained constant.

What is Teradata? RDBMS designed to run the world’s largest databases
Latest Intel technology nodes UNIX-MP-RAS, Windows 2003 Linux in Fall 2005 Scales linearly from Laptop to MPP Has a parallel aware optimizer that allows multiple complex queries to run concurrently Standard access language (SQL) Uses a “Shared-Nothing” architecture Unlimited, unconditional parallelism Linear Scalability allows for increased workload without decreased throughput.

Teradata Hardware Architecture
PE SMP Node1 AMP SMP Node2 SMP Node3 SMP Node4 BYNET Interconnect SMP Nodes Latest Intel SMP CPUs Configured in 2 to 8 node cliques Windows, Unix or Linux BYNET Interconnect Fully scalable bandwidth 1 to 1024 nodes Connectivity Fully scalable Channel - ESCON LAN, WAN Storage Independent I/O Scales per node Server Management One console to view the entire system Server Management

Teradata Shared Nothing Architecture
P P P P FSB FSB Memory I/O I/O Memory P P P P FSB FSB Memory I/O I/O Memory Similar to Large SMP, except Interconnect runs at I/O Rates and not Memory Rates Longer Lifetime: I/O Interfaces have a 3-5 Year Lifetime Scaling Is By Increasing Link Data Rates and Parallel Links

SMP vs. MPP: The Teradata Advantage
2-Way SMP 1.8 Relative CPU’s 4 GB Memory 3.2 GB/Sec BUS 3.2 GB/Sec Memory 1.5 GB/Sec I/O 4-Way SMP 3.1 Relative CPU’s 4 GB Memory 3.2 GB/SEC BUS 3.2 GB/Sec Memory 2 2-Way Teradata Nodes 3.6 Relative CPU’s 8 GB Memory 6.4 GB/Sec BUS 6.4 GB/Sec Memory 3 GB/Sec I/O 32 2-Way Teradata Nodes 57.6 Relative CPU’s 128 GB Memory 102.0 GB/Sec BUS 102.0 GB/Sec Memory 48 GB/Sec I/O

Teradata Data Distribution Dividing the Work
Rows are distributed evenly by hash partitioning Done in real-time as data are loaded, appended, or changed. No reorgs, repartitioning, space management Shared nothing software: Each VAMP owns an equal slice of the data. Each VAMP works exclusively & independently on its rows Nothing centralized: No single point of control for any operation (I/O, Buffers, Locking, Logging, Dictionary) Table A Table B Table C Prime Index Teradata Parallel Hash Function RowHash (Hash Bucket) Data Fields VAMP1 VAMP2 VAMP3 VAMP4 ………………………………………………………VAMPn

File System File system architecture is fundamentally different
Broke all the rules No Pages, BufferPools, TableSpaces, Extents,... Data location and management are entirely automatic Space allocation is entirely dynamic Absolutely minimal labor required No reorgs Don’t even have a reorg utility No index rebuilds No re-partitioning No detailed space management Easy database and table definition Minimum ongoing maintenance All performed automatically

Self Managing Architecture
Teradata’s self-managing philosophy provides the lowest total cost of ownership of any RDBMS Automatic, random and even data distribution Parallel-aware optimizer eliminates query tuning Parallel utilities with low setup and checkpoint restart Single operational view of entire MPP complex (AWS) Single point of control for the DBA (Teradata Manager) SQL-ready database management information (log files) Teradata provides the lowest cost of ownership of any database. This points back to the differentiators we discussed earlier, where Teradata does a lot of the work for you or its architecture eliminates much of it altogether. [Enter any extra notes here; leave the item ID line at the bottom] Avitage! Item ID: {{1AFCF36E A-8DB19DA3802D}}

Teradata DBAs Don’t Worry About!
Install the Database Understand, monitor and tune extensive operating system parameters Understand, monitor and tune extensive database parameters Determine the size and physical location and/or space allocations of tables and index partitions Perform periodic table and index re-orgs Manually restart multi-step load process when failure occurs Ability to run queries and data maintenance 24x7 Sort data before loading Calculate and configure fail-over plans in a clustered multiprocessing environment Spend a lot of time planning and expanding the system Query tuning for decision support

Teradata High Availability
Teradata software provides high availability beyond other databases Compensates for hardware failures: Automatic failover for dynamic workload rebalancing (migrating VPROCS) Online, continuous backup (Fallback) Recycles before the operating system completes its reboot (multi-node system) BYNET Interconnect SMP Node1 SMP Node2 SMP Node3 SMP Node4 PE PE AMP PE PE AMP PE PE AMP PE PE AMP AMP AMP AMP AMP AMP AMP AMP AMP AMP AMP AMP AMP

Teradata Dual Active Systems
Benefits Dual System Solution Recovery in Hours or Less Increase Processing Power Return on Investment for all systems Dual Load of Source Same Performance as single loading Resource Efficient Flexible Scalable Query Director Workload Balancing to maximize Business Value Query Director Teradata System A Teradata System B Source and ETL Updates

Teradata’s Multidimensional Scalability (It’s more than just big data)
Concurrent Users Amount of Detailed Data Multiple Subject Areas Simple Direct at the start Moderate Multi-table Join Regression analysis Query tool support Sophisticated Queries

EDW Requires Multi-dimensional Scalability
Data Volume (Raw, User Data) Mixed Workload Query Concurrency Data Freshness Query Complexity Query Freedom Schema Sophistication Query Data Volume

The Teradata Difference “Multi-dimensional Scalability”
Data Volume (Raw, User Data) Mixed Workload Query Concurrency Teradata can Scale Simultaneously Across Multiple Dimensions Driven by Business! Competition Scales One Dimension at the Expense of Others Limited by Technology! Data Freshness Query Complexity Query Freedom Schema Sophistication Query Data Volume

Data Volume (Raw, User Data) Mixed Workload Query Concurrency Teradata can Scale Simultaneously Across Multiple Dimensions Driven by Business! Competition Scales One Dimension at the Expense of Others Limited by Technology! The Teradata Difference! Data Freshness Query Complexity Query Freedom Schema Sophistication Query Data Volume

Data Storage (raw, user data) Teradata Others 20 TB 100’s TBs + Multiple, Integrated Stars and Normalized 15 TB 1,000’s # of Concurrent Queries Schema Sophistication Normalized 10 TB Multiple, Integrated Stars 5 TB Simple Star Batch Reporting, Repetitive Queries 3-5 Way Joins “Iterative”, Ad Hoc Queries Data Analysis/Mining 5-10 Way Joins MBs 15+ way Joins + OLAP operations + Aggregation + Complex “Where” constraints + Views Parallelism Near Real Time Data Feeds Active Data Warehousing GBs Workload Mix Query Data Volumes Query Complexity TBs

State of Michigan, Department of Community Health (DCH)
Teradata Customer Since 1991 Customer Profile As the largest department in the State of Michigan, DCH is responsible for managing delivery of health care services to more than 1.2 million clients and overseeing an annual budget of $9.5 billion. DCH administers many of the state’s most critical programs, including Medicaid, WIC, and child immunizations. Business Solutions Data warehouse integrates claims/encounters; beneficiary eligibility data; provider data; birth records; death records; long-term care assessments; WIC data; immunizations; lead screening; newborn screening; & notifiable diseases. Fraud & abuse Contract management with health plans Healthcare cost & quality assessment Overpayment & COB analysis Program effectiveness Predict State’s healthcare needs Prioritize health initiatives for future Implementation Summary Integrated data from nine separate health-related agencies Managed and used by agency subject matter/programmatic experts, not by the IT department Over 200 users in Medicaid and 8,000 state-wide Realizations and ROI Estimated annual savings of $75 million–$100 million due to advanced health care analysis Medicaid administrative costs have been reduced by 25 percent Recoveries for Medicaid Fraud has doubled Maximized Medicaid program savings while sustaining quality care Warehouse helped Michigan go from “last to first” in child immunization rates Track and substantiate savings in Medicaid pharmacy costs 2004 TDWI Best Practice Award Winner – Government and Non-Profit Category 1) 8 node 5250 for DR and Test 2) 8 Node 5380 for production. 14TB of spinning disk; 7TB of user space

Teradata Customer Since 1999
The New York State Department of Health (DoH) Teradata Customer Since 1999 Customer Profile New York’s Medicaid program provides critical health care services to more than 3.7 million participants – 2.4 million in New York City alone. To serve this constituency, the state processes and analyzes more than 300 million claims totaling more than $38 billion annually. It is the largest Medicaid program in the US. Business Solutions New York is making more rapid, informed decisions about programs, policies, and people across its vast Medicaid system. Fraud & abuse Tracking bio-terrorism indicators daily by pharmaceutical purchases with acute illness data from hospital emergency rooms Determining disease patterns and trends and the best possible treatment Tracking drug pattern usage to prevent abuse Program effectiveness Service delivery effectiveness Enhanced audit control Forecasting the cost and utilization of expensive prescription drugs Identification of overpayments Responding quickly to legislative inquiries Implementation Summary More than five years of History 1.3 Billion Claims 650 users from 17 counties that is expected to grow to thousands Realizations and ROI First year in operation paid for entire implementation of the DW! Better analysis of integrated data resulted in recoveries in the millions! $16m - Coordination of Benefits, $5m - duplicate payments, $1 million - overpayments $187 million saved due to better policy decisions based on medical and pharmaceutical analysis Millions saved due to efficiency of analysis such as Audit process reduced to 2 hours from 8 weeks 2004 NASCIO Award – Best Information Architecture Category 1) 14 Node 5250 for Production. 9 TB of user space. 2) 4 Node 5255 for Ingenix 3) 2 Node 5200 for Test & Developement

Iowa Department of Revenue
Tax Compliance Have more accurate leads because of better information Experienced substantial savings; staff can -- Analyze greater volumes of data Manage a greater number of cases Exercise a higher level of control over taxpaying behavior Before the EDW, this additional work would have caused for a 20-25% increase of the audit staff Generated $69.7M in incremental collections and refund reductions in 2003 $30.6M through office examinations $17.4M in refund reductions $ 9.1M from tax gap revenues $ 7.5M in out-of-state audits of multi-state businesses $ 5.1M from in-state field audits Business Benefits

The Teradata Mission Teradata Active Data Warehousing Sources Users
strategic tactical event-driven decision making in a single centralized mission-critical up-to-date version of the enterprise data Sources Users Active Data Warehouse tactical strategic Give brief high level overview of where we are going just to set the stage. “Any Question, By Any User, At Any Time” All Decision Making…from One Copy of the Data.

The Industry Leader in Data Warehousing

Teradata Leaders in Enterprise Data Warehousing

Similar presentations

Presentation on theme: "Teradata Leaders in Enterprise Data Warehousing"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Teradata Leaders in Enterprise Data Warehousing

Similar presentations

Presentation on theme: "Teradata Leaders in Enterprise Data Warehousing"— Presentation transcript:

Similar presentations

About project

Feedback