Fundamentals of Distributed Systems .

Slides:



Advertisements
Similar presentations
Números.
Advertisements

Trend for Precision Soil Testing % Zone or Grid Samples Tested compared to Total Samples.
AGVISE Laboratories %Zone or Grid Samples – Northwood laboratory
Trend for Precision Soil Testing % Zone or Grid Samples Tested compared to Total Samples.
PDAs Accept Context-Free Languages
ALAK ROY. Assistant Professor Dept. of CSE NIT Agartala
EuroCondens SGB E.
Worksheets.
Distributed Systems Architectures
Clustering Technology For Scaleability Jim Gray Microsoft Research
Copyright © 2013 Elsevier Inc. All rights reserved.
Addition and Subtraction Equations
18 Copyright © 2005, Oracle. All rights reserved. Distributing Modular Applications: Introduction to Web Services.
Add Governors Discretionary (1G) Grants Chapter 6.
CALENDAR.
Data recovery 1. 2 Recovery - introduction recovery restoring a system, after an error or failure, to a state that was previously known as correct have.
Chapter 6 File Systems 6.1 Files 6.2 Directories
1 Chapter 12 File Management Patricia Roy Manatee Community College, Venice, FL ©2008, Prentice Hall Operating Systems: Internals and Design Principles,
1 Advanced Tools for Account Searches and Portfolios Dawn Gamache Cindy Bylander.
The 5S numbers game..
突破信息检索壁垒 -SciFinder Scholar 介绍
Database Systems: Design, Implementation, and Management
© Tally Solutions Pvt. Ltd. All Rights Reserved Shoper 9 License Management December 09.
A Fractional Order (Proportional and Derivative) Motion Controller Design for A Class of Second-order Systems Center for Self-Organizing Intelligent.
Welcome. © 2008 ADP, Inc. 2 Overview A Look at the Web Site Question and Answer Session Agenda.
The basics for simulations
A sample problem. The cash in bank account for J. B. Lindsay Co. at May 31 of the current year indicated a balance of $14, after both the cash receipts.
Database Performance Tuning and Query Optimization
Employee & Manager Self Service Overview
MM4A6c: Apply the law of sines and the law of cosines.
Figure 3–1 Standard logic symbols for the inverter (ANSI/IEEE Std
TCCI Barometer March “Establishing a reliable tool for monitoring the financial, business and social activity in the Prefecture of Thessaloniki”
Advanced data management Jiaheng Lu Department of Computer Science Renmin University of China
Dynamic Access Control the file server, reimagined Presented by Mark on twitter 1 contents copyright 2013 Mark Minasi.
TCCI Barometer March “Establishing a reliable tool for monitoring the financial, business and social activity in the Prefecture of Thessaloniki”
Progressive Aerobic Cardiovascular Endurance Run
Lecture plan Transaction processing Concurrency control
Chapter 9: The Client/Server Database Environment
CSE 6007 Mobile Ad Hoc Wireless Networks
MaK_Full ahead loaded 1 Alarm Page Directory (F11)
1 Processes and Threads Chapter Processes 2.2 Threads 2.3 Interprocess communication 2.4 Classical IPC problems 2.5 Scheduling.
TCCI Barometer September “Establishing a reliable tool for monitoring the financial, business and social activity in the Prefecture of Thessaloniki”
When you see… Find the zeros You think….
1 How Do I Order From.decimal? Rev 05/04/09 This instructional training document may be updated at anytime. Please visit and check the.
2011 WINNISQUAM COMMUNITY SURVEY YOUTH RISK BEHAVIOR GRADES 9-12 STUDENTS=1021.
Before Between After.
2011 FRANKLIN COMMUNITY SURVEY YOUTH RISK BEHAVIOR GRADES 9-12 STUDENTS=332.
ST/PRM3-EU | | © Robert Bosch GmbH reserves all rights even in the event of industrial property rights. We reserve all rights of disposal such as copying.
Subtraction: Adding UP
1 Non Deterministic Automata. 2 Alphabet = Nondeterministic Finite Accepter (NFA)
Distributed Database Management Systems
Static Equilibrium; Elasticity and Fracture
Resistência dos Materiais, 5ª ed.
Lial/Hungerford/Holcomb/Mullins: Mathematics with Applications 11e Finite Mathematics with Applications 11e Copyright ©2015 Pearson Education, Inc. All.
WARNING This CD is protected by Copyright Laws. FOR HOME USE ONLY. Unauthorised copying, adaptation, rental, lending, distribution, extraction, charging.
A Data Warehouse Mining Tool Stephen Turner Chris Frala
Outlook 2013 Web App (OWA) User Guide Durham Technical Community College.
1 Non Deterministic Automata. 2 Alphabet = Nondeterministic Finite Accepter (NFA)
Introduction Embedded Universal Tools and Online Features 2.
Schutzvermerk nach DIN 34 beachten 05/04/15 Seite 1 Training EPAM and CANopen Basic Solution: Password * * Level 1 Level 2 * Level 3 Password2 IP-Adr.
Introduction to cloud computing Jiaheng Lu Department of Computer Science Renmin University of China
4/25/ Application Server Issues for the Project CSEP 545 Transaction Processing for E-Commerce Philip A. Bernstein Copyright ©2003 Philip A. Bernstein.
1 Module 6 Log Manager COP Log Manager Log knows everything. It is the temporal database –The online durable data are just their current versions.
Chapter 15 Recovery. Topics in this Chapter Transactions Transaction Recovery System Recovery Media Recovery Two-Phase Commit SQL Facilities.
Distributed Transaction & Long-running transactions Rossen Zhivkov Freelance SharePoint Consultant January 19 th, 2008 Sofia, Bulgaria Krasimir Parushev.
Transactions: Structure for Distributed Applications Jim Gray Microsoft Format Revised and Some Typing Errors corrected
6. Application Server Issues for the Project
Distributed system (Lecture 02)
The Client/Server Database Environment
Outline Introduction Background Distributed DBMS Architecture
Presentation transcript:

Fundamentals of Distributed Systems . Jim Gray Researcher Microsoft Corp. Gray@Microsoft.com Prof. Andreas Reuter Professor U. Stuttgart Reuter@Informatik.uni-stuttgart.de ™

Outline Concepts and Terminology Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts Goal: What you need to know to understand Microsoft Transaction Server (or CORBA or …)

What’s a Distributed System? Centralized: everything in one place stand-alone PC or Mainframe Distributed: some parts remote distributed users distributed execution distributed data

Why Distribute? No best organization Companies constantly swing between Centralized: focus, control, economy Decentralized: adaptive, responsive, competitive Why distribute? reflect organization or application structure empower users / producers improve service (response / availability) distributed load use PC technology (economics)

What Should Be Distributed? Users and User Interface Thin client Processing Trim client Data Fat client Will discuss tradeoffs later Presentation workflow Business Objects Database

Transparency in Distributed Systems Make distributed system as easy to use and manage as a centralized system Give a Single-System Image Location transparency: hide fact that object is remote hide fact that object has moved hide fact that object is partitioned or replicated Name doesn’t change if object is replicated, partitioned or moved.

Naming- The basics Names are context dependent: Many naming systems Objects have Globally Unique Identifier (GUIDs) location(s) = address(es) name(s) addresses can change objects can have many names Names are context dependent: (Jim @ KGB not the same as Jim @ CIA) Many naming systems UNC: \\node\device\dir\dir\dir\object Internet: http://node.domain.root/dir/dir/dir/object LDAP: ldap://ldap.domain.root/o=org,c=US,cn=dir Address guid Jim James

Name Servers in Distributed Systems North Name servers translate names + context to address (+ GUID) Name servers are partitioned (subtrees of name space) Name servers replicate root of name tree Name servers form a hierarchy Distributed data from hell: high read traffic high reliability & availability autonomy root Northern names South root Southern names

Autonomy in Distributed Systems Owner of site (or node, or application, or database) Wants to control it If my part is working , must be able to access & manage it (reorganize, upgrade, add user,…) Autonomy is Essential Difficult to implement. Conflicts with global consistency examples: naming, authentication, admin…

Security The Basics Authentication server subject + Authenticator => (Yes + token) | No Security matrix: who can do what to whom Access control list is column of matrix “who” is authenticated ID In a distributed system, “who” and “what” and “whom” are distributed objects subject Object Permissions

Security in Distributed Systems Security domain: nodes with a shared security server. Security domains can have trust relationships: A trusts B: A “believes” B when it says this is Jim@B Security domains form a hierarchy. Delegation: passing authority to a server when A asks B to do something (e.g. print a file, read a database) B may need A’s authority Autonomy requires: each node is an authenticator each node does own security checks Internet Today: no trust among domains (fire walls, many passwords) trust based on digital signatures

Clusters The Ideal Distributed System. Cluster is distributed system BUT single location manager security policy relatively homogeneous communications is high bandwidth low latency low error rate Clusters use distributed system techniques for load distribution storage execution growth fault tolerance

Cluster: Shared What? Shared Disk Cluster Shared Nothing Cluster Shared Memory Multiprocessor Multiple processors, one memory all devices are local DEC or SGI or Sequent 16x nodes Shared Disk Cluster an array of nodes all shared common disks VAXcluster + Oracle Shared Nothing Cluster each device local to a node ownership may change Tandem, SP2, Wolfpack

Outline Concepts and Terminology Why Distribute Distributed data & objects Partitioned Replicated Distributed execution Three tier architectures Transaction concepts

Partitioned Data Break file into disjoint groups Orders Exploit data access locality Put data near consumer Less network traffic Better response time Better availability Owner controls data autonomy Spread Load data or traffic may exceed single store N.A. S.A. Europe Asia

How to Partition Data? How to Partition Problem: to find it must have by attribute or random or by source or by use Problem: to find it must have Directory (replicated) or Algorithm Encourages attribute-based partitioning N.A. S.A. Europe Asia

Replicated Data Place fragment at many sites Pros: Improves availability Disconnected (mobile) operation Distributes load Reads are cheaper Cons: N times more updates N times more storage Placement strategies: Dynamic: cache on demand Static: place specific Catalog

Updating Replicated Data When a replica is updated, how do changes propagate? Master copy, many slave copies (SQL Server) always know the correct value (master) change propagation can be transactional as soon as possible periodic on demand Symmetric, and anytime (Access) allows mobile (disconnected) updates updates propagated ASAP, periodic, on demand non-serializable colliding updates must be reconciled. hard to know “real” value

Replication and Partitioning Compared 1 TPS server 100 Users Base case a 1 TPS system 2 TPS server 200 Users Scaleup to a 2 TPS centralized system Central Scaleup 2x more work Partition Scaleup 2x more work Replication Scaleup 4x more work Partitioning Two 1 TPS systems 1 TPS server 100 Users O tps Two 2 TPS systems 2 TPS server 100 Users 1 tps Replication

Outline Concepts and Terminology Why Distribute Distributed data & objects Partitioned Replicated Distributed execution remote procedure call queues Three tier architectures Transaction concepts

Distributed Execution Threads and Messages Thread is Execution unit (software analog of cpu+memory) Threads execute at a node Threads communicate via Shared memory (local) Messages (local and remote) shared memory messages

Peer-to-Peer or Client-Server Peer-to-Peer is symmetric: Either side can send Client-server client sends requests server sends responses simple subset of peer-to-peer request response

Connection-less or Connected request contains client id client context work request client authenticated on each message only a single response message e.g. HTTP, NFS v1 Connected (sessions) open - request/reply - close client authenticated once Messages arrive in order Can send many replies (e.g. FTP) Server has client context (context sensitive) e.g. Winsock and ODBC HTTP adding connections

Remote Procedure Call: The key to transparency Object may be local or remote Methods on object work wherever it is. Local invocation y = pObj->f(x); f() x val y = val; return val;

Remote Procedure Call: The key to transparency Remote invocation marshal un x proxy y = pObj->f(x); Obj Local? x un marshal pObj->f(x) x stub val y = val; f() return val; Obj Local? x val f() return val; val

Object Request Broker (ORB) Orchestrates RPC Registers Servers Manages pools of servers Connects clients to servers Does Naming, request-level authorization, Provides transaction coordination (new feature) Old names: Transaction Processing Monitor, Web server, NetWare Transaction Object-Request Broker

History and Alphabet Soup Microsoft DCOM based on OSF-DCE Technology DCOM and ActiveX extend it 1985 Solaris International UNIX OSF DCE Foundation (OSF) Open software NT X/Open 1990 Management Group (OMG) Object CORBA ODBC XA / TX DCE RPC GUIDs IDL Kerberos DNS COM 1995 Open Group

Using RPC for Transparency Partition Transparency Send updates to correct partition send to correct partition x y = pfile->write(x); part Local? x un marshal pObj->write(x) x x val write() return val; val

Using RPC for Transparency Replication Transparency Send updates to EACH node Send to each replica x y = pfile->write(x); x val

Client/Server Interactions All can be done with RPC Request-Response response may be many messages Conversational server keeps client context Dispatcher three-tier: complex operation at server Queued de-couples client from server allows disconnected operation C S S C S S S C S

Queued Request/Response Time-decouples client and server Three Transactions Almost real time, ASAP processing Communicate at each other’s convenience Allows mobile (disconnected) operation Disk queues survive client & server failures Submit Perform Response Client Server

Why Queued Processing? Prioritize requests ambulance dispatcher favors high-priority calls Manage Workflows Deferred processing in mobile apps Interface heterogeneous systems EDI, MOM: Message-Oriented-Middleware DAD: Direct Access to Data Order Build Ship Invoice Pay

Outline Concepts and Terminology Why Distributed Distributed data & objects Distributed execution remote procedure call queues Three tier architectures what why Transaction concepts

Work Distribution Spectrum Fat Thin Presentation and plug-ins Workflow manages session & invokes objects Business objects Database Presentation workflow Business Objects Database

Transaction Processing Evolution to Three Tier Intelligence migrated to clients Mainframe cards Mainframe Batch processing (centralized) Dumb terminals & Remote Job Entry Intelligent terminals database backends Workflow Systems Object Request Brokers Application Generators green screen 3270 Server TP Monitor ORB Active

Web Evolution to Three Tier Intelligence migrated to clients (like TP) Server WAIS Character-mode clients, smart servers GUI Browsers - Web file servers GUI Plugins - Web dispatchers - CGI Smart clients - Web dispatcher (ORB) pools of app servers (ISAPI, Viper) workflow scripts at client & server archie ghopher green screen Mosaic NS & IE Active

PC Evolution to Three Tier Intelligence migrated to server Stand-alone PC (centralized) PC + File & print server message per I/O PC + Database server message per SQL statement PC + App server message per transaction ActiveX Client, ORB ActiveX server, Xscript IO request reply disk I/O SQL Statement Transaction

The Pattern: Three Tier Computing Presentation Clients do presentation, gather input Clients do some workflow (Xscript) Clients send high-level requests to ORB (Object Request Broker) ORB dispatches workflows and business objects -- proxies for client, orchestrate flows & queues Server-side workflow scripts call on distributed business objects to execute task workflow Business Objects Database

The Three Tiers DCOM (oleDB, ODBC,...) Object server Pool HTTP+ DCOM Web Client HTML VB or Java Script Engine Virt Machine VBscritpt JavaScrpt VB Java plug-ins Internet ORB HTTP+ DCOM Object server Pool Middleware TP Monitor Web Server... DCOM (oleDB, ODBC,...) Object & Data server. LU6.2 IBM Legacy Gateways

Why Did Everyone Go To Three-Tier? Manageability Business rules must be with data Middleware operations tools Performance (scaleability) Server resources are precious ORB dispatches requests to server pools Technology & Physics Put UI processing near user Put shared data processing near shared data Presentation workflow Business Objects Database

Why Put Business Objects at Server? Customer comes to store with list Gives list to clerk Clerk gets goods, makes invoice Customer pays clerk, gets goods Easy to manage Clerks controls access Encapsulation MOM’s Business Objects DAD’sRaw Data Customer comes to store Takes what he wants Fills out invoice Leaves money for goods Easy to build No clerks

What Middleware Does ORB, TP Monitor, Workflow Mgr, Web Server Registers transaction programs workflow and business objects (DLLs) Pre-allocates server pools Provides server execution environment Dynamically checks authority (request-level security) Does parameter binding Dispatches requests to servers parameter binding load balancing Provides Queues Operator interface

Server Side Objects Easy Server-Side Execution A Server ORB gives simple execution environment Object gets start invoke shutdown Everything else is automatic Drag & Drop Business Objects Network Receiver Queue Management Connections Context Security Configuration Thread Pool Service logic Synchronization Shared Data

Why Server Pools? Server resources are precious. Clients have 100x more power than server. Pre-allocate everything on server preallocate memory pre-open files pre-allocate threads pre-open and authenticate clients Keep high duty-cycle on objects (re-use them) Pool threads, not one per client Classic example: TPC-C benchmark 2 processes everything pre-allocated N clients x N Servers x F files = N x N x F file opens!!! Pool of DBC links HTTP IE 7,000 clients IIS SQL

Classic Three-Tier Example TPC-C 7,000 Web clients Transaction Processing Performance Council (TPC): standard performance benchmarks 5 transaction types order entry , payment , status (oltp) delivery (mini-batch) restock (mini-DSS) Metrics: Throughput, Price/Performance Shows best practices: everyone three tier 2 processes at server everything pre-allocated HTTP IIS = Web Pool of DBC links ODBC SQL

Classic Mistakes Thread per terminal fix: DB server thread pools fix: server pools Process per request (CGI) fix: ISAPI & NSAPI DLLs fix: connection pools Many messages per operation fix: stored procedures fix: server-side objects File open per request fix: cache hot files

Outline Three tier architectures Transaction concepts Why Distributed Distributed data & objects Distributed execution Three tier architectures why: manageability & performance what: server side workflows & objects Transaction concepts Why transactions? Using transactions Two Phase Commit How transactions?

Thesis Transactions are key to structuring distributed applications ACID properties ease exception handling Atomic: all or nothing Consistent: state transformation Isolated: no concurrency anomalies Durable: committed transaction effects persist 2

What Is A Transaction? Success! Failure! Programmer’s view: Bracket a collection of actions A simple failure model Only two outcomes: Begin() action Commit() Begin() action Rollback() Begin() action Rollback() Fail ! Success! Failure! 3

Why Bother: Atomicity? ? ? ? RPC semantics: At most once: try one time At least once: keep trying ’till acknowledged Exactly once: keep trying ’till acknowledged and server discards duplicate requests ? ? ? 4

Why Bother: Atomicity? Example: insert record in file At most once: time-out means “maybe” At least once: retry may get “duplicate” error or retry may do second insert Exactly once: you do not have to worry What if operation involves Insert several records? Send several messages? Want ALL or NOTHING for group of actions 5

Why Bother: Consistency Begin-Commit brackets a set of operations You can violate consistency inside brackets Debit but not credit (destroys money) Delete old file before create new file in a copy Print document before delete from spool queue Begin and commit are points of consistency State transformations new state under construction Begin Commit 6

Why Bother: Isolation Running programs concurrently on same data can create concurrency anomalies The shared checking account example Programming is hard enough without having to worry about concurrency Begin() read BAL add 10 write BAL Commit() Begin() read BAL Subtract 30 write BAL Commit() Bal = 100 Bal = 100 Bal = 110 Bal = 70 7

Isolation It is as though programs run one at a time No concurrency anomalies System automatically protects applications Locking (DB2, Informix, Microsoft® SQL Server™, Sybase…) Versioned databases (Oracle, Interbase…) Begin() read BAL add 10 write BAL Commit() Bal = 100 Begin() read BAL Subtract 30 write BAL Commit() Bal = 110 Bal = 110 Bal = 80 8

Why Bother: Durability Once a transaction commits, want effects to survive failures Fault tolerance: old master-new master won’t work: Can’t do daily dumps: would lose recent work Want “continuous” dumps Redo “lost” transactions in case of failure Resend unacknowledged messages 9

Why ACID For Client/Server And Distributed ACID is important for centralized systems Failures in centralized systems are simpler In distributed systems: More and more-independent failures ACID is harder to implement That makes it even MORE IMPORTANT Simple failure model Simple repair model 11

ACID Generalizations Taxonomy of actions Unprotected: not undone or redone Temp files Transactional: can be undone before commit Database and message operations Real: cannot be undone Drill a hole in a piece of metal, print a check Nested transactions: subtransactions Work flow: long-lived transactions 10

Outline Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts Why transactions? ACID: atomic, consisistent, isolated, durable Using transactions programming save points nested, chained workflow Two Phase Commit How transactions?

Programming & Transactions The Application View You Start (e.g. in TransactSQL): Begin [Distributed] Transaction <name> Perform actions Optional Save Transaction <name> Commit or Rollback You Inherit a XID Caller passes you a transaction You return or Rollback. You can Begin / Commit sub-trans. You can use save points Begin Begin RollBack Commit XID RollBack Return Return

Transaction Save Points Backtracking within a transaction BEGIN WORK:1 Allows app to cancel parts of a transaction prior to commit This is in most SQL products (save transaction in MS SQL Server) action action SAVE WORK:2 ROLLBACK WORK(2) action SAVE WORK:4 SAVE WORK:3 SAVE WORK:5 action SAVE WORK:6 action SAVE WORK:7 action ROLLBACK WORK(7) action SAVE WORK:8 action COMMIT WORK

Chained Transactions Commit of T1 implicitly begins T2. Carries context forward to next transaction cursors locks other state Transaction #1 Transaction #2 C o m i t B e g n Processing context established used

Nested Transactions Going Beyond Flat Transactions Need transactions within transactions Sub-transactions commit only if root does Only root commit is durable. Subtransactions may rollback if so, all its subtransactions rollback Parallel version of nested transactions T12 T121 T122 T123 T1 T11 T13 T112 T114 T131 T132 T133 T111 T113

Workflow: A Sequence of Transactions Application transactions are multi-step order, build, ship & invoice, reconcile Each step is an ACID unit Workflow is a script describing steps Workflow systems Instantiate the scripts Drive the scripts Allow query against scripts Examples Manufacturing Work In Process (WIP) Queued processing Loan application & approval, Hospital admissions… Presentation workflow Business Objects Database

Workflow Scripts Workflow scripts are programs (could use VBScript or JavaScript) If step fails, compensation action handles error Events, messages, time, other steps cause step. Workflow controller drives flows fork Source join branch case loop Compensation Action Step

Workflow and ACID Workflow is not Atomic or Isolated Results of a step visible to all Workflow is Consistent and Durable Each flow may take hours, weeks, months Workflow controller keeps flows moving maintains context (state) for each flow provides a query and operator interface e.g.: “what is the status of Job # 72149?”

ACID Objects Using ACID DBs The easy way to build transactional objects Application uses transactional objects (objects have ACID properties) If object built on top of ACID objects, then object is ACID. Example: New, EnQueue, DeQueue on top of SQL SQL provides ACID SQL dim c as Customer dim CM as CustomerMgr ... set C = CM.get(CustID) C.credit_limit = 1000 CM.update(C, CustID) .. Business Object: Customer Business Object Mgr: CustomerMgr SQL Persistent Programming languages automate this.

ACID Objects From Bare Metal The Hard Way to Build Transactional Objects Object Class is a Resource Manager (RM) Provides ACID objects from persistent storage Provides Undo (on rollback) Provides Redo (on restart or media failure) Provides Isolation for concurrent ops Microsoft SQL Server, IBM DB2, Oracle,… are Resource managers. Many more coming. RM implementation techniques described later

Outline Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts Why transactions? Using transactions programming save points nested, chained workflow Two Phase Commit Prepare and commit phases Transaction & Resource Managers How transactions?

Transaction Manager Transaction Manager (TM): manages transaction objects. XID factory tracks them coordinates them App gets XID from TM Transactional RPC passes XID on all calls manages XID inheritance TM manages commit & rollback TM begin XID enlist App RM call(..XID)

TM Two-Phase Commit Dealing with multiple RMs If all use one RM, then all or none commit If multiple RMs, then need coordination Standard technique: Marriage: Do you? I do. I pronounce…Kiss Theater: Ready on the set? Ready! Action! Act Sailing: Ready about? Ready! Helm’s a-lee! Tack Contract law: Escrow agent Two-phase commit: 1. Voting phase: can you do it? 2. If all vote yes, then commit phase: do it! 20

Two-Phase Commit In Pictures Transactions managed by TM App gets unique ID (XID) from TM at Begin() XID passed on Transactional RPC RMs Enlist when first do work on XID TM Begin XID Call(..XID..) Enlist App RM1 Call(..XID..) Enlist RM2 21

When App Requests Commit Two Phase Commit in Pictures TM tracks all RMs enlisted on an XID TM calls enlisted RM’s Prepared() callback If all vote yes, TM calls RM’s Commit() If any vote no, TM calls RM’s Rollback() 4. TM decides Yes, broadcasts 4 Commit 1. Application requests Commit Commit 1 TM 5. RMs acknowledge 5 Yes Yes 3 3. RMs all vote Yes Prepare 2 2. TM broadcasts prepared? yes 6. TM says App RM1 RM2 22

X/Open Standardizes Two-Phase Commit Standardized APIs for apps and to RMs Points to OSI/TP for interoperation TX: begin commit rollback Comm mgr TM TM XA+: outgoing incoming XA: enlist, Prepare Commit Client Comm mgr Comm mgr Server SQL or MTS or.. RM RM 23

How Does This Relate To Microsoft? SQL Server is transactional (so is Oracle, DB2, Informix, Sybase) MS Distributed Transaction Coordinator (DTC) packaged with SQL Server, MTS, and other RMs Connects to CICS, Encina, Topend, Tuxedo Any RM (SNA LU6.2, DB2, Oracle, Sybase, Informix, …) can participate in transactions 25

OLE Transactions: the Movie Two styles: (1) Bind an RM connection to the transaction. All work on that connection is now part of that transaction. (2) pass transaction object on every RM call. Not shown: client can get async notification of transaction outcome. DtcGetTransactionManager() TM I TransactionDispenser BeginTransaction Client ITransaction GetTransactionInfo Commit Abort Transaction Comm Mgr Resource Manager aka (sql, viper,…) begin commit rollback Commit / Abort

OLE Transactions RM Enlist DtcGetTransactionManager() TM IResourceManagerFactory Create RM registers with TM RM Enlists in transaction (provides callbacks) ITransactionResourceAsync PrepareRequest CommitRequest AbortRequest TMDown ENLIST ()!!!! IResourceManager Enlist ReEnlist ReEnlistmentComplete RM begin commit rollback Resource Manager aka (sql, viper,…) Transaction ITransactionEnlistmentAsync PrepareReqDone CommitReqDone AbortReqDone Enlistment

OLE Transactions RM Commit TM begin commit rollback Resource Manager aka (sql, viper,…) Two phase commit Enlisted RMs get prepare & commit callbacks Abort callbacks are similar ITransactionResourceAsync PrepareRequest CommitRequest AbortRequest TMDown ITransactionEnlistmentAsync PrepareReqDone CommitReqDone AbortReqDone Enlistment

Outline Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts Why transactions? Using transactions Two Phase Commit Prepare and commit phases Transaction and Resource Managers How transactions? logging locking or versioning

Implementing Transactions Atomicity The DO/UNDO/REDO protocol Idempotence Two-phase commit Durability Durable logs Force at commit Isolation Locking or versioning 12

DO/UNDO/REDO Each action generates a log record Has an UNDO action Has a REDO action New state Old state DO Log New state Old state UNDO Log New state Old state REDO Log 13

What Does A Log Record Look Like? Log record has Header (transaction ID, timestamp… ) Item ID Old value New value For messages: just message text and sequence # For records: old and new value on update Keep records small ? Log ? 14

Transaction Is A Sequence Of Actions Each action changes state Changes database Sends messages Operates a display/printer/drill press Leaves a log trail New state Old state DO Log Log New state Old state DO DO Log New state Old state Old state DO Log New state 15

Transaction UNDO Is Easy Read log backwards UNDO one step at a time Can go half-way back to get nested transactions New state Old state UNDO Log New state Old state UNDO Log New state Old state UNDO Log New state Old state UNDO Log 16

Durability: Protecting The Log When transaction commits Put its log in a durable place (duplexed disk) Need log to redo transaction in case of failure System failure: lost in-memory updates Media failure (lost disk) This makes transaction durable Log is sequential file Converts random IO to single sequential IO See NTFS or newer UNIX file systems Write Log 17

Recovery After ASystem Failure During normal processing, write checkpoints on non-volatile storage When recovering from a system failure… return to the checkpoint state Reapply log of all committed transactions Force-at-commit insures log will survive restart Then UNDO all uncommitted transactions New state Old state REDO Log New state Old state REDO Log REDO New state Old state Log Log Old state REDO New state 18

Idempotence Dealing with failure What if fail during restart? REDO many times What if new state not around at restart? UNDO something not done Old state REDO New state Log New state UNDO Old state Log 19

Idempotence Dealing with failure Solution: make F(F(x))=F(x) (idempotence) Discard duplicates Message sequence numbers to discard duplicates Use sequence numbers on pages to detect state (Or) make operations idempotent Move to position x, write value V to byte B… Old state REDO New state Log New state UNDO Old state Log

Recap ACID makes it easy to program distributed applications DO/UNDO/REDO + log allows atomicity Multiple logs need two-phase commit Persistent log gives durability Recover from system failure Recover from media failure 24

Outline Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts Why transactions? Using transactions Two Phase Commit How transactions? logging locking or versioning

Concurrency Control Locking How to automatically prevent concurrency bugs? Serialization theorem: If you lock all you touch and hold to commit: no bugs If you do not follow these rules, you may see bugs Automatic Locking: Set automatically (well-formed) Released at commit/rollback (two-phase locking) Greater concurrency for locks: Granularity: objects or containers or server Mode: shared or exclusive or… 26

Reduced Isolation Levels It is possible to lock less and risk fuzzy data Example: want statistical summary of DB But do not want to lock whole database Reduced levels: Repeatable Read: may see fuzzy inserts/delete But will serialize all updates Read Committed: see only committed data Read Uncommitted: may see uncommitted updates 27

Multiversion Concurrency Control Run transaction at some timestamp in the past No locking needed, reconstruct “old” state from log Add in your transaction’s updates At commit assure updates do not collide with other committed transactions Almost as good as serializable (only obscure bugs) 28

Summary ACID eases error handling Atomic: all or nothing Consistent: correct transformation Isolated: no concurrency bugs Durable: survives failures Allows you to build robust distributed applications ACID becoming standard part of systems It’s real 29

Outline Why Distributed Distributed data & objects 2-Tier 3-Tier Acid Atomic Autonomy Commit Consistent Delegation Durable Fat Client Idempotent Isolated Lock Log ORB Partitioned Data Queue Queued or Direct Replicated Data Resource Manager Rollback (Abort) RPC Serializable Server Pool Thin Client Transaction Manager Two Phase Commit Undo/Redo Update Anywhere Workflow XID Why Distributed Distributed data & objects Distributed execution Three tier architectures Transaction concepts

References Essential Client/Server Survival Guide 2nd ed. Orfali, Harkey & Edwards, J. Wiley, 1996 Principles of Transaction Processing Bernstein & Newcomer, Morgan Kaufmann, 1997 Transaction Processing Concepts and Techniques Gray & Reuter, Morgan Kaufmann, 1993 30