Scaleable Computing Jim Gray Microsoft Corporation ™

Thesis: Scaleable Servers  Scaleable Servers  Commodity hardware allows new applications  New applications need huge servers  Clients and servers are built of the same “stuff”  Commodity software and  Commodity hardware  Servers should be able to  Scale up (grow node by adding CPUs, disks, networks)  Scale out (grow by adding nodes)  Scale down (can start small)  Key software technologies  Objects, Transactions, Clusters, Parallelism

1987: 256 tps Benchmark  14 M$ computer (Tandem)  A dozen people  False floor, 2 rooms of machines Simulate 25,600 clients A 32 node processor array A 40 GB disk array (80 drives) OS expert Network expert DB expert Performance expert Hardware experts Admin expert Auditor Manager

1988: DB2 + CICS Mainframe 65 tps  IBM 4391  Simulated network of 800 clients  2m$ computer  Staff of 6 to do benchmark 2 x 3725 network controllers 16 GB disk farm 4 x 8 x.5GB Refrigerator-sized CPU

1997: 10 years later 1 Person and 1 box = 1250 tps  1 Breadbox ~ 5x 1987 machine room  23 GB is hand-held  One person does all the work  Cost/tps is 1,000x less 25 micro dollars per transaction 4x200 Mhz cpu 1/2 GB DRAM 12 x 4GB disk Hardware expert OS expert Net expert DB expert App expert 3 x7 x 4GB disk arrays

What Happened?  Moore’s law: Things get 4x better every 3 years (applies to computers, storage, and networks)  New Economics: Commodity classprice/mips software $/mips k$/year mainframe 10, minicomputer microcomputer 10 1  GUI: Human - computer tradeoff optimize for people, not computers mainframe mini micro time price

What Happens Next  Last 10 years: 1000x improvement  Next 10 years: ????  Today: text and image servers are free 25  $/hit => advertising pays for them  Future: video, audio, … servers are free “You ain’t seen nothing yet!” performance

Kinds Of Information Processing It’s ALL going electronic Immediate is being stored for analysis (so ALL database) Analysis and automatic processing are being added Point-to-pointBroadcast Immediate Time- shifted ConversationMoney LectureConcert Mail BookNewspaper Network Database

Why Put Everything In Cyberspace? Low rent - min $/byte Shrinks time - now or later Shrinks space - here or there Automate processing - knowbots Point-to-pointORbroadcast Immediate OR time-delayed Network Database LocateProcessAnalyzeSummarize

Magnetic Storage Cheaper Than Paper  File cabinet :cabinet (four drawer)250$ paper (24,000 sheets)250$ space 10$/ft 2 )180$ total700$ 3¢/sheet  Disk :disk (4 GB =)800$ ASCII: 2 mil pages 0. 04¢/sheet (80x cheaper)  Image : 200,000 pages 0.4¢/sheet (8x cheaper)  Store everything on disk

Databases Information at Your Fingertips ™ Information Network ™ Knowledge Navigator ™  All information will be in an online database (somewhere)  You might record everything you  Read: 10MB/day, 400 GB/lifetime (eight tapes today)  Hear: 400MB/day, 16 TB/lifetime (three tapes/year today)  See: 1MB/s, 40GB/day, 1.6 PB/lifetime (maybe someday)

Database Store ALL Data Types  The new world:  Billions of objects  Big objects (1 MB)  Objects have behavior (methods)  The old world:  Millions of objects  100-byte objects People NameAddress Mike Won David NY Berk Austin People Name AddressPapersPicture Voice Mike Won David NY Berk Austin  Paperless office  Library of Congress online  All information online  Entertainment  Publishing  Business  WWW and Internet

Billions Of Clients  Every device will be “intelligent”  Doors, rooms, cars…  Computing will be ubiquitous

Billions Of Clients Need Millions Of Servers Mobile clients Fixed clients Server Superserver Clients Servers  All clients networked to servers  May be nomadic or on-demand  Fast clients want faster servers  Servers provide  Shared Data  Control  Coordination  Communication

Thesis Many little beat few big  Smoking, hairy golf ball  How to connect the many little parts?  How to program the many little parts?  Fault tolerance? $1 million $100 K $10 K Mainframe Mini Micro Nano 14" 9" 5.25" 3.5" 2.5" 1.8" 1 M SPECmarks, 1TFLOP 10 6 clocks to bulk ram Event-horizon on chip VM reincarnated Multiprogram cache, On-Chip SMP 10 microsecond ram 10 millisecond disc 10 second tape archive 10 nano-second ram Pico Processor 10 pico-second ram 1 MM TB 1 TB 10 GB 1 MB 100 MB

Future Super Server: 4T Machine  Array of 1,000 4B machines  1 bps processors  1 BB DRAM  10 BB disks  1 Bbps comm lines  1 TB tape robot  A few megabucks  Challenge:  Manageability  Programmability  Security  Availability  Scaleability  Affordability  As easy as a single system Future servers are CLUSTERS of processors, discs Distributed database techniques make clusters work CPU 50 GB Disc 5 GB RAM Cyber Brick a 4B machine

The Hardware Is In Place… And then a miracle occurs ?  SNAP: scaleable network and platforms  Commodity-distributed OS built on:  Commodity platforms  Commodity network interconnect  Enables parallel applications

Thesis: Scaleable Servers  Scaleable Servers  Commodity hardware allows new applications  New applications need huge servers  Clients and servers are built of the same “stuff”  Commodity software and  Commodity hardware  Servers should be able to  Scale up (grow node by adding CPUs, disks, networks)  Scale out (grow by adding nodes)  Scale down (can start small)  Key software technologies  Objects, Transactions, Clusters, Parallelism

Scaleable Servers BOTH SMP And Cluster Grow up with SMP; 4xP6 is now standard Grow out with cluster Cluster has inexpensive parts Cluster of PCs SMP super server Departmental server Personal system

SMPs Have Advantages  Single system image easier to manage, easier to program threads in shared memory, disk, Net  4x SMP is commodity  Software capable of 16x  Problems:  >4 not commodity  Scale-down problem (starter systems expensive)  There is a BIGGEST one SMP super server Departmental server Personal system

Building the Largest Node  There is a biggest node (size grows over time)  Today, with NT, it is probably 1TB  We are building it (with help from DEC and SPIN2)  1 TB GeoSpatial SQL Server database  (1.4 TB of disks = 320 drives).  30K BTU, 8 KVA, 1.5 metric tons.  Will put it on the Web as a demo app.  10 meter image of the ENTIRE PLANET.  2 meter image of interesting parts (2% of land) One pixel per meter = 500 TB uncompressed.  Better resolution in US (courtesy of USGS). Support files 1-TB SQL Server DB Satellite and aerial photos Todo loo da loo-rah, ta da ta-la la la 1-TB home page TM

What’s TeraByte?  1 Terabyte: 1,000,000,000 business letters 150 miles of book shelf 1,000,000,000 business letters 150 miles of book shelf 100,000,000 book pages 15 miles of book shelf 100,000,000 book pages 15 miles of book shelf 50,000,000 FAX images 7 miles of book shelf 50,000,000 FAX images 7 miles of book shelf 10,000,000 TV pictures (mpeg) 10 days of video 4,000 LandSat images 16 earth images (100m) 10,000,000 TV pictures (mpeg) 10 days of video 4,000 LandSat images 16 earth images (100m) 100,000,000 web page 10 copies of the web HTML 100,000,000 web page 10 copies of the web HTML  Library of Congress (in ASCII) is 25 TB 1980: $200 million of disc 10,000 discs 1980: $200 million of disc 10,000 discs $5 million of tape silo 10,000 tapes 1997: $200 k$ of magnetic disc 48 discs 1997: $200 k$ of magnetic disc 48 discs $30 k$ nearline tape 20 tapes $30 k$ nearline tape 20 tapes Terror Byte !

TB DB User Interface TB DB User Interface Next

Tpc-C Web-Based Benchmarks  Client is a Web browser (7,500 of them!)  Submits  Order  Invoice  Query to server via Web page interface  Web server translates to DB  SQL does DB work  Net:  easy to implement  performance is GREAT! HTTP ODBC SQL SQL IIS = Web

Grow UP and OUT 1 billion transactions per day SMP super server Departmental server Personal system 1 Terabyte DB Cluster: a collection of nodes as easy to program and manage as a single node

Clusters Have Advantages  Clients and servers made from the same stuff  Inexpensive:  Built with commodity components  Fault tolerance:  Spare modules mask failures  Modular growth  Grow by adding small modules  Unlimited growth: no biggest one

Windows NT Clusters  Microsoft & 60 vendors defining NT clusters  Almost all big hardware and software vendors involved  No special hardware needed - but it may help  Fault-tolerant first, scaleable second  Microsoft, Oracle, SAP giving demos today  Enables  Commodity fault-tolerance  Commodity parallelism (data mining, virtual reality…)  Also great for workgroups!

Billion Transactions per Day Project  Building a 20-node Windows NT Cluster (with help from Intel) > 800 disks  All commodity parts  Using SQL Server & DTC distributed transactions  Each node has 1/20 th of the DB  Each node does 1/20 th of the work  15% of the transactions are “distributed”

How Much Is 1 Billion Transactions Per Day? Millions of transactions per day , Btpd Visa AT&T BofA NYSE Mtpd  1 Btpd = 11,574 tps (transactions per second) ~ 700,000 tpm (transactions/minute)  AT&T  185 million calls (peak day worldwide)  Visa ~20 M tpd  400 M customers  250,000 ATMs worldwide  7 billion transactions / year (card+cheque) in 1994

Parallelism The OTHER aspect of clusters  Clusters of machines allow two kinds of parallelism  Many little jobs: online transaction processing  TPC-A, B, C…  A few big jobs: data search and analysis  TPC-D, DSS, OLAP  Both give automatic parallelism

Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey Kinds of Parallel Execution Pipeline Partition outputs split N ways inputs merge M ways Any Sequential Program Any Sequential Program Any Sequential Any Sequential Program

Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey Data Rivers Split + Merge Streams River M Consumers N producers Producers add records to the river, Consumers consume records from the river Purely sequential programming. River does flow control and buffering does partition and merge of data records River = Split/Merge in Gamma = Exchange operator in Volcano. N X M Data Streams

Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey Partitioned Execution Spreads computation and IO among processors Partitioned data gives NATURAL parallelism

Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey N x M way Parallelism N inputs, M outputs, no bottlenecks. Partitioned Data Partitioned and Pipelined Data Flows

The Parallel Law Of Computing Grosch's Law: Parallel Law: Parallel Law:Needs: Linear speedup and linear scale-up Not always possible 1 MIPS 1 $ 1,000 MIPS 1,000 $ 2x $ is 2x performance 1 MIPS 1 $ 1,000 MIPS 32 $ 32 $.03$/MIPS 2x $ is 4x performance

Thesis: Scaleable Servers  Scaleable Servers  Commodity hardware allows new applications  New applications need huge servers  Clients and servers are built of the same “stuff”  Commodity software and  Commodity hardware  Servers should be able to  Scale up (grow node by adding CPUs, disks, networks)  Scale out (grow by adding nodes)  Scale down (can start small)  Key software technologies  Objects, Transactions, Clusters, Parallelism

The BIG Picture Components and transactions  Software modules are objects  Object Request Broker (a.k.a., Transaction Processing Monitor) connects objects (clients to servers)  Standard interfaces allow software plug-ins  Transaction ties execution of a “job” into an atomic unit: all-or-nothing, durable, isolated Object Request Broker

ActiveX and COM  COM is Microsoft model, engine inside OLE ALL Microsoft software is based on COM (ActiveX)  CORBA + OpenDoc is equivalent  Heated debate over which is best  Both share same key goals:  Encapsulation: hide implementation  Polymorphism: generic operations key to GUI and reuse  Versioning: allow upgrades  Transparency: local/remote  Security: invocation can be remote  Shrink-wrap: minimal inheritance  Automation: easy  COM now managed by the Open Group

Linking And Embedding Objects are data modules; transactions are execution modules  Link: pointer to object somewhere else  Think URL in Internet  Embed: bytes are here  Objects may be active ; can callback to subscribers

Commodity Software Components Inexpensive OS, DBMS…and plug-ins  Recent TPC-C prices  Oracle on DEC UNIX: 30.4 k 305$/tpmC  Informix on DEC UNIX: 13.6 k 277$/tpmC  DB2 on Solaris: $/tpmC  SQL Server on Compaq, Windows NT: $/tpmC (using Web, no TP monitor!)  Oracle on Windows NT: $/tpmC  Net: “Open” solutions can do even biggest jobs; thousands of online users per “node” of cluster  ActiveX, VBX, and Java plug-ins  Spreadsheets, GeoQuery, FAX, voice, image libraries, commodity component market

Objects Meet Databases The basis for universal data servers, access, & integration DBMSengine  object-oriented (COM oriented) programming interface to data  Breaks DBMS into components  Anything can be a data source  Optimization/navigation “on top of” other data sources  A way to componentized a DBMS  Makes an RDBMS and O-R DBMS (assumes optimizer understands objects) Database Spreadsheet Photos Mail Map Document

42 The Pattern: Three Tier Computing  Clients do presentation, gather input  Clients do some workflow (Xscript)  Clients send high-level requests to ORB (Object Request Broker)  ORB dispatches workflows and business objects -- proxies for client, orchestrate flows & queues  Server-side workflow scripts call on distributed business objects to execute task Database Business Objects workflow Presentation

43 The Three Tiers Web Client HTML VB or Java Script Engine VB or Java Virt Machine VBscritpt JavaScrpt VB Java plug-ins Internet ORB HTTP+ DCOM Object server Pool Middleware ORB TP Monitor Web Server... DCOM (oleDB, ODBC,...) Object & Data server. LU6.2 IBM Legacy Gateways

44 Why Did Everyone Go To Three-Tier?  Manageability  Business rules must be with data  Middleware operations tools  Performance (scaleability)  Server resources are precious  ORB dispatches requests to server pools  Technology & Physics  Put UI processing near user  Put shared data processing near shared data Database Business Objects workflow Presentation

45 DAD’sRaw Data Customer comes to store Takes what he wants Fills out invoice Leaves money for goods Easy to build No clerks Why Put Business Objects at Server? Customer comes to store with list Gives list to clerk Clerk gets goods, makes invoice Customer pays clerk, gets goods Easy to manage Clerks controls access Encapsulation MOM’s Business Objects

46 What Middleware Does ORB, TP Monitor, Workflow Mgr, Web Server  Registers transaction programs workflow and business objects (DLLs)  Pre-allocates server pools  Provides server execution environment  Dynamically checks authority (request-level security)  Does parameter binding  Dispatches requests to servers  parameter binding  load balancing  Provides Queues  Operator interface

47 Server Side Objects Easy Server-Side Execution  Give simple execution environment  Object gets  start  invoke  shutdown  Everything else is automatic  Drag & Drop Business Objects Network Thread Pool Queue Connections Context Security Shared Data Receiver Synchronization Service logic Configuration Management A Server

A new programming paradigm  Develop object on the desktop  Better yet: download them from the Net  Script work flows as method invocations  All on desktop  Then, move work flows and objects to server(s)  Gives  desktop development  three-tier deployment  Software Cyberbricks

Transactions Coordinate Components (ACID)  Transaction properties  Atomic: all or nothing  Consistent: old and new values  Isolated: automatic locking or versioning  Durable: once committed, effects survive  Transactions are built into modern OSs  MVS/TM Tandem TMF, VMS DEC-DTM, NT-DTC

Transactions & Objects  Application requests transaction identifier (XID)  XID flows with method invocations  Object Managers join (enlist) in transaction  Distributed Transaction Manager coordinates commit/abort

Transactions Coordinate Components (ACID)  Programmer’s view: bracket a collection of actions  A simple failure model  Only two outcomes: Begin() action action Commit() Success! Begin()actionactionactionRollback()Begin()actionactionactionRollback() Failure! Fail !

Distributed Transactions Enable Huge Throughput  Each node capable of 7 KtmpC (7,000 active users!)  Can add nodes to cluster (to support 100,000 users)  Transactions coordinate nodes  ORB / TP monitor spreads work among nodes

Distributed Transactions Enable Huge DBs  Distributed database technology spreads data among nodes  Transaction processing technology manages nodes

Thesis: Scaleable Servers  Scaleable Servers Built from Cyberbricks  Allow new applications  Servers should be able to  Scale up, out, down  Key software technologies  Clusters (ties the hardware together)  Parallelism: ( uses the independent cpus, stores, wires  Objects (software CyberBricks)  Transactions: masks errors.

Computer Industry Laws (Rules of thumb)  Metcalf’s law  Moore’s first law  Bell’s computer classes (7 price tiers)  Bell’s platform evolution  Bell’s platform economics  Bill’s law  Software economics  Grove’s law  Moore’s second law  Is info-demand infinite?  The death of Grosch’s law

Metcalf’s Law Network Utility = Users 2  How many connections can it make?  1 user: no utility  100,000 users: a few contacts  1 million users: many on Net  1 billion users: everyone on Net  That is why the Internet is so “hot”  Exponential benefit

 XXX doubles every 18 months 60% increase per year  Micro processor speeds  Chip density  Magnetic disk density  Communications bandwidth WAN bandwidth approaching LANs  Exponential growth:  The past does not matter  10x here, 10x there, soon you’re talking REAL change  PC costs decline faster than any other platform  Volume and learning curves  PCs will be the building bricks of all future systems Moore’s First Law

Bumps In The Moore’s Law Road  DRAM:  1988: United States anti-dumping rules  : ?price flat $/MB of DRAM , $/MB of DISK  Magnetic disk:  : 10x/decade  : 4x/3year! 100X/decade

Gordon Bell’s 1975 VAX Planning Model... He Didn’t Believe It!  5x: Memory is 20% of cost 3x: DEC markup.04x: $ per byte  He didn’t believe: the projection $500 machine  He couldn’t comprehend the implications System Price = 5 x 3 x.04 x memory size/ 1.26 (t-1972) K$

Gordon Bell’s Processing Memories, And Comm 100 Years Processing Pri. Mem Sec. Mem. POTS(bps) Backbone

Gordon Bell’s Seven Price Tiers 10$: wrist watch computers 10$: wrist watch computers 100$:pocket/ palm computers 100$:pocket/ palm computers 1,000$:portable computers 1,000$:portable computers 10,000$: personal computers (desktop) 10,000$: personal computers (desktop) 100,000$: departmental computers (closet) 100,000$: departmental computers (closet) 1,000,000$:site computers (glass house) 1,000,000$:site computers (glass house) 10,000,000$:regional computers (glass castle) 10,000,000$:regional computers (glass castle) Super server: costs more than $100,000 “Mainframe”: costs more than $1 million Must be an array of processors, disks, tapes, comm ports

Bell’s Evolution Of Computer Classes Technology enables two evolutionary paths: 1. constant performance, decreasing cost 2. constant price, increasing performance ?? Time Mainframes (central) Minis (dep’t.) PCs (personals) Log price WSs 1.26 = 2x/3 yrs -- 10x/decade; 1/1.26 = = 4x/3 yrs --100x/decade; 1/1.6 =.62

Gordon Bell’s Platform Economics Computer type Mainframe WSBrowser Price (K$) Volume (K) Application price  Traditional computers: custom or semi-custom, high-tech and high-touch  New computers: high-tech and no-touch

Software Economics  An engineer costs about $150,000/year  R&D gets [5%…15%] of budget  Need [$3 million… $1 million] revenue per engineer Microsoft: $9 billion R&D16% SG&A34% Product and Service 13% Tax13% Profit24% Intel: $16 billion R&D8% SG&A 11% P&S47% Tax 12% Profit 22%R&D8% SG&A22% P&S59%Tax5% Profit6% IBM: $72 billion R&D9% SG&A 43% Tax7% Profit15% P&S26% Oracle: $3 billion

Software Economics: Bill’s Law  Bill Joy’s law (Sun): don’t write software for less than 100,000 million engineering expense, $1,000 price  Bill Gate’s law: don’t write software for less than 1,000,000 engineering expense, $100 price  Examples:  UNIX versus Windows NT: $3,500 versus $500  Oracle versus SQL-Server: $100,000 versus $6,000  No spreadsheet or presentation pack on UNIX/VMS/...  Commoditization of base software and hardware Price Fixed_Cost Marginal _Cost = Units +

Grove’s Law The New Computer Industry  Horizontal integration is new structure  Each layer picks best from lower layer  Desktop (C/S) market  1991: 50%  1995: 75% Intel & Seagate Silicon & Oxide Systems Baseware Middleware Applications SAP Oracle Microsoft Compaq Integration EDS Operation AT&T Function Example

Moore’s Second Law  The cost of fab lines doubles every generation (three years)  Money limit hard to imagine:  $10-billion line  $20-billion line  $40-billion line  Physical limit  Quantum effects at 0.25 micron now 0.05 micron seems hard 12 years, three generations  Lithograph: need Xray below 0.13 micron $1 $10 $100 $1,000 $10, Year $million/ Fab Line

 Constant work :  One SuperServer can do all the world’s computations  Constant dollars:  The world spends 10% on information processing  Computers are moving from 5% penetration to 50%  $300 billion to $3 trillion  We have the patent on the byte and algorithm Constant Dollars Versus Constant Work

Crossing The Chasm Oldmarket OldtechnologyNewtechnology Very hard Hard Boringcompetitive slow growth No product no customers Product finds customers customers Customers find product Hard Newmarket