1. Advanced Database Systems September 2013 Dr. Fatemeh Ahmadi-Abkenari 1

2. 2  Data might be distributed to equalize the workload so the individual site are not overloaded to such a degree that the throughput is impaired.  Data might be placed in such a way as to minimize communication costs and / or response time.  Data might be kept at the site in order to maintain control and guarantee security.  Certain data items might be replicated at multiple site to increase their availability in the event of system crashes.

3. 3 Multiple Local Schema Global Schema Restricted Global Schema

4. 4 Definition: The distributed database looks to the application program like a collection of individual databases, each with its own schema. Heterogeneous System: If the individual DBMSs have been supplied by different vendors. Homogeneous System: If the individual DBMSs have been supplied by the same vendors. DBMS Application Program

5. 5 o The application program must explicitly set up a connection to each site that contains data items to be accessed. After a connection has been established, the program can access the database using SQL statements constructed using the site’s schema. o A single SQL statement that refers to tables at different site like a global join, is not supported. In such a this case, the application must read the tuples from each table into buffers at the application site and explicitly test the join condition for each pairs of tuples. Characteristics o Data at different site might be stored at different formats. In such a this case, the application must provide the conversion routines.

6. 6 Definition: In this approach, the application designer sees a single schema that integrates all local schema. DBMS Integration Middleware Application Program Integration is done by middleware. Middleware is a software that supports the interaction of clients and servers often in a heterogeneous systems. The Global Schema might include tables that do not appear in any local schema but can be computed from tables in local schema using appropriate SQL statements.

7. 7 o Connection to individual sites are made automatically by the middleware Hence the location of tables are hidden from the application program. This feature is called Location Transparency. o Application program execute SQL statements against the global schema. In the case of join, if two tables are stored at different sites, the middleware must translate the join into a sequence of steps executed by the individual DBMSs. Characteristics o Data at different site might be stored at different formats. The middleware provides the conversion routines to integrate the systems. An important aspect of this schema is Query Optimization. It means the middleware choose a sequence of steps that constitutes a least-cost plan for evaluating the SQL statement submitted by the application.

8. 8 Restricted Global Schema are supported by the vendors of some homogeneous systems. The databases cooperate directly, eliminating the need for middleware. The application can execute an SQL statement that refers to the tables at different sites for example a global join. The system includes a Global Query Optimizer to design efficient query plans.

9. 9 1 - Partitioning 2- Replication Common Models of Data Distribution: 1-1 Horizontal Partitioning 1-2 Vertical Partitioning 1-3 Mixed Partitioning Certain data items may be stored at particular sites for security reasons. Other items may be divided and stored. Some items may be replicated for easier access.

10. 10 The simplest approach for distributing data is to store each table in a different site. However, always the table unit is not the unit of best choice to be distributed. More often, a transaction access only a subset of the rows of a table or a view of the table rather than a table as a whole. When a table is decomposed in a way that each portion of a table is stored at a different site where the corresponding transaction is executed, the portion is referred to as partitions. One advantage of partitioning a table is that the time to process a single query over a large table could be reduced by distributing the execution over a number of sites at which partitions are stored.

11. 11 A single table of T is partitioned into several tables of T 1, T 2, …, T n where each partition contains a subset of the rows of T and each row of T is in exactly one partition. Generally, each partition satisfies T i = σ Ci (T) Where c i is a selection condition and each tuple in T satisfies c i for exactly one value of i and T= U i T i

12. 12 INVENTORY (StockNum, Amount, Price, Location) If the horizontal partitioning is performed on the relation by Location, it stores all tuples satisfying LOCATION = ‘CHICAGO’ As a partition named INVENTORY_CH (StockNum, Amount, Price) regarding the redundancy of the attribute Location. Because each tuple is stored in a partition, horizontal partitioning is lossless. Example

13. 13 A single table of T is partitioned into several tables of T 1, T 2, …, T n where each partition contains a subset of the columns of T and each column must be included in at least one partition and each partition must include the columns of a candidate key (the same for all partitions). Generally, each partition satisfies T i = π att-list i (T) Because each column is stored in at least one partition, vertical partitioning is lossless T = T 1 T 2 … T n

14. 14 EMPLOYEE (SSnum, Name, Salary, Title, Location) If the vertical partitioning is performed, so the two below partitions are constructed EMP1 (SSnum, Name, Salary) EMP2 (SSnum, Name, Title, Location) Where EMP1 could be stored at headquarter site (where the payroll is computed) and EMP2 is stored elsewhere. Example

15. 15 EMPLOYEE (SSnum, Name, Salary, Title, Location) Step 1: Vertical Partitioning EMP1 (SSnum, Name, Salary) EMP2 (SSnum, Name, Title, Location) Step 2: Horizontal Partitioning As partitions named EMP2_CH (SSnum, Name, Title) and EMP2_NY (SSnum, Name, Title) respectively regarding the redundancy of the attribute Location. Example Combination of horizontal and vertical partitioning are possible but care must be taken that the original table could be constructed from its partitions.

16. 16 If one employee transfers from Chicago warehouse to New York warehouse: EMP2_CH (SSnum, Name, Title) EMP2_NY (SSnum, Name, Title) When relations are partitioned, update operations sometimes require tuples to be moved from one partition to another and hence from one database to another.

17. 17 Replication is one of the most useful and common used mechanism in distributed databases. Data Replication at several sites causes the increase in availability since the data can still be accessed if some of the sites fail. Replication has the potential to improve performance since queries can be executed more efficiently because the data could be read from a local or nearby copy. Updates are slower since all replicas of the data must be updated.

18. 18 A multidatabase system is composed of a set of independent DBMSs. In order for the application to query information stored at multiple sites, it must decompose the query into a sequence of SQL statements that each of them is processed by a particular DBMS. System that supports the Global Schema contains a Global Query Optimizer which analyzes a query using the global schema and translates it into an appropriate sequence of steps to be executed at individual sites. Regarding the fact that the cost of I/O is so much greater than that of computation, so the measure of efficiency in a query execution plan is the number of required I/O operation. Communication costs will be measured by the number of bytes that have to be transmitted.

19. 19 Planning with joins Suppose that an application in site A wants to join tables at site B and C with the result to be returned to site A. Two ways to execute the join that could be employed by the Global Query Optimizer are:  Transmit both tables to site A and execute joins there.  Transmit the smaller of the tables (e.g. the tables at site B to site C, execute the join at site C) and then transmit the result to site A.

20. 20 Example Consider Two tables of STUDENT (Id, Major) TRANSCRIPT (StudId, CrsCode) The tables are stored at site B and C respectively. Suppose an application at site A wants to compute a join with the join condition as Id=StudId. For this example, the assumptions are: 1- The length of attributes  Id and StudId: 9 bytes  Major: 3 bytes  CrsCode: 6 bytes 2- STUDENT has 15000 tuples 3- Approximately 5000 students are registered for at least one course and on average each student is registered for four courses. Thus TRANSCRIPT has about 20000 tuples.  10000 students are not registered for any course.

21. 21 Solution STUDENT has 15000 tuples each of length 12 bytes (9+3). TRANSCRIPT has 20000 tuples each with the length of 15 bytes (9+6). The join will have 20000 tuples each of length of 18 bytes (9+6+3). Based on the above assumptions, there are three alternative plans:  If both tables are sent to site A to perform the join there, so 15000*12+20000* 15= 480000 bytes needs to be transferred.  If the STUDENT table is sent to site C and then in site C the join is computed and the result is sent to site A so, 15000*12 + 20000*18=540000 bytes need to be transferred.  If the TRANSCRIPT table is sent to site B, the join table is computed there and the result is sent to site A, so 20000*15 + 20000* 18=660000 bytes need to be transferred.

22. 22 Queries that involve joins and selections Suppose that there are only one warehouse in Internet grocer application and the EMPLOYEE relation as EMPLOYEE (SSnum, Name, Salary, Title, Location) is vertically partitioned as EMP1 (SSnum, Name, Salary) : stored at site B (Headquarter) EMP2 (SSnum, Title, Location): stored at site C (Warehouse) Suppose a query at the third site, A, requests the names of all employees with the title of “manager” whose salary is more than 20000$. So the problem aims at: 1 π Name (σ Title=‘manager’ AND Salary> ‘20000’ (EMP1 EMP2)) 1 π Name (σ Title=‘manager’ AND Salary> ‘20000’ (EMP1 EMP2)) 2 π Name ((σ Salary> ‘20000’ (EMP1)) (σ Title=‘manager’ (EMP2))) √ 2 π Name ((σ Salary> ‘20000’ (EMP1)) (σ Title=‘manager’ (EMP2))) √

23. 23 So 1- At site B select all tuples from EMP1 for which the Salary is more than 20000$ and call the result R 1. 2- At site C select all tuples from EMP2 for which the Title is manager and call the result R 2. 3- Perform the join of R 1 and R 2 in a site and project on the result using Name attribute. Call it R 3. If this site is not A, send R 3 to site A. The length of attributes are: SSnum: 9bytes; Salary: 6bytes; Title: 7bytes; Location: 10 bytes; Name: 15 bytes;

24. 24 Plans 1- Sending R 2 to site B and do the join there. Then send the names to A. 2- Sending R 1 to site C and do the join there. Then send the names to A. 3- Sending R 1 and R 2 to site A and do the join there.  The length of each tuple in EMP1 is 30 bytes and in EMP2 is 26 bytes.  EMP1 and hence EMP2 has about 100000 tuples.  About 5000 employees have a salary of more than 20000$. So R 1 has 5000 tuples (each of 30 bytes) for a total of 150000 bytes.  There are about 50 managers. Therefore R 2 has about 50 tuples (each of length 26 bytes), for a total of 1300 bytes.  About 90% of the managers have salary of more than 20000$. Therefore R 3 has about 45 tuples each of length 15 bytes for a total of 675 bytes.

25. 25 Evaluating the cost of each plan: 1- Upon doing the join at site B, so 1300 bytes should be sent from site C to B and then 675 bytes from site B to site A. So totally 1975 bytes should be transferred. 2- Upon doing the join at site C, so 150000 bytes should be sent from site B to C and then 675 bytes from site C to site A. So totally 150675 bytes should be transferred. 3- Upon doing the join at site A, so 150000 bytes should be sent from site B to A and then 1300 bytes from site C to site A. So totally 151300 bytes should be transferred.