1. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.1 Outline  Introduction à What is a distributed DBMS à Problems à Current state-of-affairs  Background  Distributed DBMS Architecture  Distributed Database Design  Semantic Data Control  Distributed Query Processing  Distributed Transaction Management o Parallel Database Systems o Distributed Object DBMS o Database Interoperability o Current Issues

2. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.2 File Systems program 1 data description 1 program 2 data description 2 program 3 data description 3 File 1 File 2 File 3

3. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.3 Database Management database DBMS Application program 1 (with data semantics) Application program 2 (with data semantics) Application program 3 (with data semantics) description manipulation control

4. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.4 DB Clients, Servers, and Environments DB-Server, a collection of programs that execute all DBMS function DB-Client, any application program that needs to connect to a DB-Server DB Environment (DBE), one or more DBs along with any software providing at least minimum set of required data operation and management.

5. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.5 DBE Architectural concept Service, logical collections of related functionality. Example: Query Service Sites, represents a logical location in an architectural diagram or deployment diagram Component and Subsystem (COS) à Component, Deployable bundle of software that provide reasonability cohesive set of functionality à Subsystem, collection of one or more components that work together toward a common goal

6. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.6 DBE Architectures Required Services à Data Read Service (Drd-S) à Security Service (Sec-S) à Semantic Integrity Service (Semi-S)

7. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.7 DBE Architectures Basic Services à Data Read Service (Drd-S) à Security Service (Sec-S) à Semantic Integrity Service (Semi-S) à Data Write Service (Dwr-S)

8. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.8 DBE Architectures Expected Service à Data Read Service (Drd-S) à Security Service (Sec-S) à Semantic Integrity Service (Semi-S) à Data Write Service (Dwr-S) à Query Request Service (Qreq-S) à Query Optimization Service à Execution Service à Execution Optimization Service

9. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.9 DBE Architectures Expected Subsystem à Data Read Service (Drd-S) à Security Service (Sec-S) à Semantic Integrity Service (Semi-S) à Data Write Service (Dwr-S) à Query Request Service (Qreq- S) à Query Optimization Service à Execution Service à Execution Optimization Service à User Interface

10. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.10 DBE Architectures Typical DBMS Service à Drd-S, Sec-S, Semi-S, Dwr-S, Qreq-S à Query Optimization Service à Execution Service à Execution Optimization Service à User Interface à Transaction Management (Trans-S) à Locking Service (Lock-S) à Timestamping Service (Times- S) à Deadlock Handling Service à Fallback and Recovery Service

11. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.11 Motivation Database Technology Computer Networks integrationdistribution integration integration ≠ centralization Distributed Database Systems

12. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.12 DBMS Schema Architecture

13. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.13 DDBMS Schema Architecture

14. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.14 Top Down DDBMS Software Architecture

15. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.15 Bottom Up DDBMS Software Architecture

16. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.16 Generic DDBMS architecture

17. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.17 Distributed Computing A concept in search of a definition and a name. A number of autonomous processing elements (not necessarily homogeneous) that are interconnected by a computer network and that cooperate in performing their assigned tasks.

18. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.18 Synonymous terms à distributed function à distributed data processing à multiprocessors/multicomputers à satellite processing à backend processing à dedicated/special purpose computers à timeshared systems à functionally modular systems Distributed Computing

19. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.19 Processing logic Functions Data Control What is distributed …

20. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.20 What is a Distributed Database System? A distributed database (DDB) is a collection of multiple, logically interrelated databases distributed over a computer network. A distributed database management system (D–DBMS) is the software that manages the DDB and provides an access mechanism that makes this distribution transparent to the users. Distributed database system (DDBS) = DDB + D–DBMS

21. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.21 A timesharing computer system A loosely or tightly coupled multiprocessor system A database system which resides at one of the nodes of a network of computers - this is a centralized database on a network node What is not a DDBS?

22. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.22 Centralized DBMS on a Network Site 5 Site 1 Site 2 Site 3 Site 4 Communication Network

23. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.23 Distributed DBMS Environment Site 5 Site 1 Site 2 Site 3 Site 4 Communication Network

24. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.24 Implicit Assumptions Data stored at a number of sites  each site logically consists of a single processor. Processors at different sites are interconnected by a computer network  no multiprocessors à parallel database systems Distributed database is a database, not a collection of files  data logically related as exhibited in the users’ access patterns à relational data model D-DBMS is a full-fledged DBMS à not remote file system, not a TP system

25. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.25 Shared-Memory Architecture Examples :symmetric multiprocessors (Sequent, Encore) and some mainframes (IBM3090, Bull's DPS8) P1P1 PnPn M D

26. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.26 Shared-Disk Architecture Examples :DEC's VAXcluster, IBM's IMS/VS Data Sharing D P1P1 M1M1 PnPn MnMn

27. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.27 Shared-Nothing Architecture Examples :Teradata's DBC, Tandem, Intel's Paragon, NCR's 3600 and 3700 P1P1 M1M1 D1D1 PnPn MnMn DnDn

28. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.28 Manufacturing - especially multi-plant manufacturing Military command and control EFT Corporate MIS Airlines Hotel chains Any organization which has a decentralized organization structure Applications

29. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.29 Distributed DBMS Promises  Transparent management of distributed, fragmented, and replicated data  Improved reliability/availability through distributed transactions  Improved performance  Easier and more economical system expansion

30. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.30 Transparency Transparency is the separation of the higher level semantics of a system from the lower level implementation issues. Fundamental issue is to provide data independence in the distributed environment à Network (distribution) transparency à Replication transparency à Fragmentation transparency  horizontal fragmentation: selection  vertical fragmentation: projection  hybrid

31. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.31 Example TITLESAL PAY Elect. Eng. 40000 Syst. Anal.34000 Mech. Eng. 27000 Programmer24000 PROJ PNO PNAME BUDGET ENOENAMETITLE E1J. DoeElect. Eng. E2M. SmithSyst. Anal. E3A. LeeMech. Eng. E4J. MillerProgrammer E5B. CaseySyst. Anal. E6L. ChuElect. Eng. E7R. DavisMech. Eng. E8J. JonesSyst. Anal. EMP ENOPNORESP E1P1Manager12 DUR E2P1Analyst24 E2P2Analyst 6 E3P3Consultant10 E3P4Engineer48 E4P2Programmer18 E5P2Manager24 E6P4Manager48 E7P3Engineer36 E8P3Manager40 ASG P1 Instrumentation 150000 P3CAD/CAM250000 P2Database Develop.135000 P4Maintenance310000 E7P5Engineer23

32. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.32 Transparent Access SELECTENAME,SAL FROMEMP,ASG,PAY WHEREDUR > 12 ANDEMP.ENO = ASG.ENO ANDPAY.TITLE = EMP.TITLE Paris projects Paris employees Paris assignments Boston employees Montreal projects Paris projects New York projects with budget > 200000 Montreal employees Montreal assignments Boston Communication Network Montreal Paris New York Boston projects Boston employees Boston assignments Boston projects New York employees New York projects New York assignments Tokyo

33. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.33 Distributed Database - User View Distributed Database

34. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.34 Distributed DBMS - Reality Communication Subsystem User Query DBMS Software DBMS Software User Application DBMS Software User Application User Query DBMS Software User Query DBMS Software

35. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.35 Potentially Improved Performance Proximity of data to its points of use à Requires some support for fragmentation and replication Parallelism in execution à Inter-query parallelism à Intra-query parallelism

36. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.36 Parallelism Requirements  Have as much of the data required by each application at the site where the application executes à Full replication How about updates? à Updates to replicated data requires implementation of distributed concurrency control and commit protocols

37. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.37 System Expansion Issue is database scaling Emergence of microprocessor and workstation technologies à Demise of Grosh's law à Client-server model of computing Data communication cost vs telecommunication cost

38. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.38 Distributed DBMS Issues Distributed Database Design à how to distribute the database à replicated & non-replicated database distribution à a related problem in directory management  Query Processing à convert user transactions to data manipulation instructions à optimization problem à min{cost = data transmission + local processing} à general formulation is NP-hard

39. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.39 Distributed DBMS Issues  Concurrency Control à synchronization of concurrent accesses à consistency and isolation of transactions' effects à deadlock management Reliability à how to make the system resilient to failures à atomicity and durability

40. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.40 Directory Management Relationship Between Issues Reliability Deadlock Management Query Processing Concurrency Control Distribution Design

41. Distributed DBMS© 2001 M. Tamer Özsu & Patrick Valduriez Page 1.41 Operating System Support à operating system with proper support for database operations à dichotomy between general purpose processing requirements and database processing requirements Open Systems and Interoperability à Distributed Multidatabase Systems à More probable scenario à Parallel issues Related Issues