2“A Relational Model for Large Shared Databanks” The Beginning...Data Model with StructureData Independent of CodeSet-oriented1977 the work begins“A Relational Model for Large Shared Databanks”It started with one profound ideaIn 1970 E.F. Codd wrote the book….the mathematical model that would define the future of RDMBS.He later hooked up with C.J. Date of IBM and Codd and Date would become known as father and grandfather of relational theory.Codd and Date became mythical figures dispensing truths about the purity of various implementations of RDBMs. Had 12 rules of relational databases.Data Model with Structure, Integrity Rules, OperationsData Defined Independently of ProgramsSet-oriented, Declarative LanguageE.F. Codd
3GPS 1978 No one thought it could be done It wouldn’t be fast enough, no one would buy itThe early adopters came to us.First test outperformed Cullinet’s DBMS – immediately realized this was a break-through, and the company would be worth 100’s of millionsThe FIRST Commercial SQL RDBMSAn Impressive First SQLNon-procedural power (joins, subqueries) for ad hoc accessA single language for data definition, control, accessUnique Oracle extensions: outer join, CONNECT BYA Simple ServerNo transactions – like MySQLLimited reliabilityPortability from the Start – driven by customer requirements from the beginning.Digital PDP-11 operating systems: RSX, RSTS, IAS, UnixDEC VAX (compatibility mode)
4First RDBMS: Version 2 June 1979 FIRST Commercial SQL RDBMSImpressive First SQLJoins, SubqueriesOuter Joins, Connect ByA Simple ServerNo transactions, ‘Limited’ ReliabilityPortability from the StartWritten in FortranBut multi-platform – PDP11, Dec VAXNo one thought it could be doneIt wouldn’t be fast enough, no one would buy itThe early adopters came to us.First test outperformed Cullinet’s DBMS – immediately realized this was a break-through, and the company would be worth 100’s of millionsThe FIRST Commercial SQL RDBMSAn Impressive First SQLNon-procedural power (joins, subqueries) for ad hoc accessA single language for data definition, control, accessUnique Oracle extensions: outer join, CONNECT BYA Simple ServerNo transactions – like MySQLLimited reliabilityPortability from the Start – driven by customer requirements from the beginning.Digital PDP-11 operating systems: RSX, RSTS, IAS, UnixDEC VAX (compatibility mode)
5IBM model number 5150, introduced on August 12, 1981. IBM PC – 1981IBM model number 5150, introduced on August 12, 1981.
6Internet (as we know it) – 1983 The first TCP/IP-based wide-area network was operational by January 1, 1983 when all hosts on the ARPANET were switched over from the older NCP protocols.
7Portability: Version 3 March 1983 New Implementation Designed for PortabilityWritten in ‘C’Single SourceArchitectural ChangesTransactions, multi-versioning, no read consistencyAI/BI filesOracle Corporation – name establishedInnovation – “C” code was portable, leading edge and the future of the day – Ken Jacobs – why did we decide on C?And it was as practical as it was “visionary” b/c we had 2 customers on 3 different os – efficiency of developmentOracle Corp name established to align product and companyDefinition: the source of wisdom, a person who sees the future – prophetic for the visionary approachNew Implementation Designed for PortabilityAll code written in CArchitectural ChangesTransactions, but no read consistencyOracle Corporation – name establishedSelling the idea of Relational DatabaseMeanwhile, what was going on at IBM?Didn’t want to undercut IMS – IMS loyalists fought to keep relational from being released – until SQL/DS on(second biggest mistake in the history of Software)Ask, what was the biggest mistake?Biggest mistake in software history was giving the PC Operating System to MicrosoftSpent the majority of our time selling the idea that RDBMS could be commercially viableThe biggest mistake IBM would make in the history of Software was outsourcing the development of MS DOS to MSFT2nd biggest mistake CLINGING to the present, status quo, SQL/DS – not embracing the RDBMS early onLack of vision, conservative business values, not risk takers“IBM could have had the all of the relational database Market if they’d been more aggressive”“I don’t know of any place or any time where there aren’t great possibilities”At about the same time Xerox Corporation was doing something similar for a local entrepreneur(Steve Jobs) who visited them in Palo Alto Research CenterSteve Jobs, Apple – the UILarry Ellison, Oracle – the RDBMSBill Gates, MSFT – the MS DOS o/sIBM close to collapse
10Cooperative Server: Version 5 April 1985 My First Oracle Experience1st Client/ServerCooperative ServerDistributed ProcessingParallel ServerPortabilityV5 was first to go beyond 640K memory on PCsSingle-user for Macintosh o/sSQL_TRACEselect trace('sql',1),1 from dual;Client Server is bornLove/hate with Bill Gates, and Microsoft – “Microsoft does not suffer from the not-invented-here syndrome”Oracle releases Early parallel server, precursor to today’s RACMS-DOS, GUI was MAC, Browser was Netscape,…Innovative choicesIBM releases DB2 for MVS – 6 years late on their own research paperMartin Pickard, recent hire form MIT, wrote SQL*PME, Protected Mode Executive, to extend beyond the 640K memory boundary, b/c no one told him he couldn’tInnovation!Selling points – NOW 3 Cs and a PCompatibility with Industry StandardsConnectibility to heterogeneous DBs(V5)Capability robust, fast, fully functional DB(aka Reliability)AndPortability1st Client/ServerSQL*NetCooperative ServerDistributed Process, SQL*ConnectParallel ServerV5 was first to go beyond 640K memory on PCsSingle-user Oracle released for Macintosh o/sPortability , Compatibility, Capability, ConnectabilityIBM – DB2 is released for MVS
11Transaction Processing: Version 6 July 1988 New ArchitecturePerformance (first SMP)AvailabilityTPOPL/SQLV6 Lays Architectural Groundwork for the FutureThis was a rewrite of the entire database fundamentallyLarry took a HUGE RISK and decided to do a complete rewrite and redesign of the DB form the bottom up** Had he not taken that RISK and done that INNOVATION, I doubt we'd have won the rdbms game. 3 years of development investment- similar to Project Fusion.He hired many MIT and other grads from top schools in what would become known as the class of 1986.It was this redesign that allowed us to blow away Sybase and be the Scalable db for the next 20 years.First Commercial level Parallel Server (precursor to RAC) that allowed us to win the TPC BM # going forwardBench Marking was the name of the gameShut off the Oxygen Campaigns against Sybase and Ingres and Kill Informix Campaigns followed and we defeated all of our RDBMS competitorsWith Speed, scalability and robustness.
12World Wide Web – 1990’ishThe World Wide Web was created in 1989 by British scientist Tim Berners-Lee, working at the European Organization for Nuclear Research (CERN) in Geneva, Switzerland, and released in 1992.
13Oracle7.3 February 1996 Partitioned Views Bitmapped Indexes Asynchronous read ahead for table scansStandby DatabaseDeferred transaction recovery on instance startupUpdatable Join ViewSQLDBA no longer shipped.Index rebuildsDBV introducedContext OptionPL/SQL - UTL_FILESpatial Data OptionTablespaces changes - Coalesce, Temporary Permanent,Trigger compilation, debugUnlimited extents on STORAGE clause.Some init.ora parameters modifiable - TIMED_STATISTICSHASH Joins, AntijoinsHistogramsOracle TraceAdvanced Replication Object Groups
16Data Warehouses Growing Rapidly Tripling In Size Every Two Years Size of the Largest Data Warehouses200400600800100019982000200220042006200820102012Terabytes of DataSource: Winter TopTen Survey, Winter Corporation, Waltham MA, 2008.
18Three Major Database Focus Areas Engineered for Innovation SOCIALBLOGSMART METERCLOUD COMPUTING PRIVATE AND PUBLICENGINEERED SYSTEMSBIG DATA
19Building Managed Server and Storage Pools Real Application Clusters Automatic Storage ManagementEnterprise ManagerReal Application ClustersmtIn-Memory Database Cache19
20Service Level Management Resource Manager and Instance Caging Resource Manager allocates CPU and MemoryInstance caging allocates cores per instanceCapacity-on-demand for elastic cloud computing
21Network SQL Monitoring and Blocking Complete Data Security Firewall, Encryption, Separation of Duty and MonitoringEncryptedDatabaseDataMaskingActivity AuditNetwork SQL Monitoring and BlockingData DiscoveryApplicationsCompliance ScanMost companies have implemented the 3 As at some level: Authentication, Authorization, auditing. Some still use generic shared accounts, which while convenient cannot be used to track the actual user who made the change. Some still grant overly excessive privileges instead of going through the exercise of least privileges. This can be very dangerous as any SQL injection attack or stealing of pwd can be disastrous. In the end you want to audit your privilege users, but unless you monitor the audit logs, auditing may not be much of a deterrent.A small percentage of the companies do patching.This means that most others are very vulnerable to attacks discussed in previous slides. We recommend that in addition to 3 As and patching, you seriously consider the following:A database firewall to reduce the threats from the web users. The firewall can understand the SQL coming through the wire, analyzing it, and then determining whether it should be allowed to go the database. The firewall should also be able to both monitor and block unauthorized traffic including SQL injection attacks.Encrypt the database data to reduce the threats from the OS administrator or any other software running on the database machine. As the data gets backed up, you may want to encrypt the backup to reduce threats from inadvertent losses or theft of the tapes. You also want to encrypt the traffic to reduce the risk of someone sniffing the wire. Finally, to reduce the threats from your test or dev teams, you want to mask the data before you give it to them.The other threat that you want to handle is the threat from the DBA or hackers with DBA password credentials. You want the DBAs to do the job of tuning and managing the system, but you do not want them to view or modify your sensitive data. You also want to ensure that even if someone has the DBA password, they meet other factors before they are given access to the database.Now at the end of the day, you have to allow your privileged users to do their jobs. And of course you trust them, but by auditing their activities throughout the enterprise you can implement “trust but verify” principle. You may want to collect audit data from throughout your enterprise, analyze the data, raise alerts if needed, and then create reports for your auditors.Vulnerability ScanPatch AutomationUnauthorized DBA ActivityMulti-factor authorization
22Create Reference Configuration Provision Database on Cloud Provisioning Software to the Cloud Lower complexity via Reference ConfigurationsGold image reference configurationsStandardized deployments via profilesRapidly provision databases to the CloudMonitor change centrally to ensure complianceCreate Reference ConfigurationProvision Database on CloudStage as Gold ImageManage Centrally
23Metering and Chargeback Enterprise Manager 12c Resource usage meteringHistorical usage trendsCost allocation and charge plan evaluationReporting for cloud self- service application2550100Number of serversNumber of CPUsMemoryLocal Storage (GB)$100KDiscover & PlanTrack UsageCharge User
24Optimized, Pre-Integrated Cloud Platform Oracle Exadata Database Machine Database Server PoolOracle Database 11g Release 2Oracle Real Application ClustersAutomatic Storage ManagementStorage Server PoolUp to 336 TB disk5 TB flash storageOracle Exadata Storage SoftwareInfiniBand Network40 Gb/sec redundant switches
25Improve Data Warehouse performance 10x Exadata Smart Scans Select sum(sales) where salesdate= ‘21-Sep-2010’…SumFilter Sales for SepWhat Were Yesterday’s Sales?Data intensive processing runs in Exadata Storage ServersRows and columns filtered as data streams from disks
26Storage Optimization Next Generation Compression Technology Up toOLTPCompressionIncrease compression as data agesImprove query performance for table scansImprove cache density for OLTP performanceNo changes to existing applicationsCascade storage savings throughout data centerUp to50XHybrid ColumnarCompression (Exadata, Pillar Axion, Sun ZFS Storage Appliance)
27Extreme Performance for OLTP Exadata Smart Flash Cache Full rack has 5 TB of Smart Flash CacheCan process over 1 million IOs per second
28Public Cloud Oracle Public Cloud (cloud.oracle.com) Self ServiceMonthly SubscriptionSimple PricingServicesApplicationPlatformPlatform ServicesJava Cloud ServiceDatabase Cloud Service
29Database Applications in the Public Cloud Database Cloud Access and Applications Oracle Application ExpressApplicationsRESTful Web ServicesSQL DeveloperOracle Java Cloud
30Big Data Buzz“Ten reasons why Big Data will change the travel industry”“The challenge– and opportunity– of big data”“Why big data is a big deal”InfoWorld – 9/1/11McKinsey Quarterly—5/11Tnooz -8/15/11“Keeping Afloat in a Sea of 'Big Data”“Getting a Handle on Big Data with Hadoop”“The promise of Big Data”ITBusinessEdge – 9/6/11Businessweek-9/7/11Intelligent Utility-8/28/11
31Big Data Use Cases Today’s Challenge New Data What’s Possible Healthcare Expensive office visitsRemote patient monitoringPreventive care, reduced hospitalizationManufacturing In-person supportProduct sensorsAutomated diagnosis, supportLocation-Based Services Based on home zip codeReal time location dataGeo-advertising, traffic, local searchPublic Sector Standardized servicesCitizen surveysTailored services, cost reductionsRetail One size fits all marketingSocial mediaSentiment analysis segmentation
32What Makes it Big Data? VOLUME VELOCITY VARIETY VALUE SOCIALBLOGSMART METERVOLUMEVELOCITYVARIETYVALUE
35Make Better Decisions Using Big Data Big Data in Action DECIDE ACQUIRE ANALYZEORGANIZE
36Acquire all available data Big Data in ActionDECIDEACQUIREAcquire all available dataANALYZEORGANIZE
37Oracle NoSQL Database … Key value pair database Dynamic data model NodesEastWestCentralNoSQL DriverApplication…ReadDeleteUpdateKey value pair databaseDynamic data modelHighly scalable, availableTransparent load balancingBuilt using BerkeleyDB
38Organize and distill big data using massive parallelism Big Data in ActionDECIDEACQUIREOrganize and distill big data using massive parallelismANALYZEORGANIZE
39Analysis Sandbox Provides analysis workspace Controlled access to resources and dataDoesn’t impact production system
41Analyze all your data, at once Big Data in ActionANALYZEDECIDEACQUIREAnalyze all your data, at onceANALYZEORGANIZE
42R Statistical Programming Language Open source language and environmentUsed for statistical computing and graphicsStrength in easily producing publication-quality graphsHighly extensible
43Decide based on real-time big data Big Data in ActionDECIDEACQUIREDecide based on real-time big dataANALYZEORGANIZE
44Dashboard Analytics Oracle Business Intelligence Enterprise Edition Advanced dashboard visualizationRuns BI and EPM applicationsIntegrating R AnalyticsEmbed R script’s web interface in BI dashboardGraphics will stream to BI dashboard
45Oracle Integrated Solution Stack for Big Data ACQUIREOracle NoSQLDatabaseHDFSEnterpriseApplicationsORGANIZEHadoop (MapReduce)Oracle Loader for HadoopOracle Data IntegratorANALYZEIn-Database AnalyticsDataWarehouseDECIDEAnalytic Applications
46Oracle Exalytics Hardware Engineered for extreme analytics40 Intel processor cores1 Terabyte main memory40 Gb InfiniBand connection to Oracle Exadata
47Oracle Exalytics Software Oracle TimesTen In-Memory DatabaseAdaptive in-memory caching of analyticsIn-memory columnar compressionTightly integrated with Oracle ExadataEnables speed-of-thought visualizationOracle Business Intelligence Foundation Suite
48Oracle Big Data Appliance Hardware 18 Sun X4270 M2 Servers48 GB memory per node = 864 GB memory12 Intel cores per node = 216 cores24 TB storage per node = 432 TB storage40 Gb p/sec InfiniBand10 Gb p/sec Ethernet
49Oracle Big Data Appliance Software Oracle LinuxJava Hotspot VMApache Hadoop DistributionR DistributionOracle NoSQL DatabaseOracle Data Integrator for HadoopOracle Loader for Hadoop