Presentation is loading. Please wait.

Presentation is loading. Please wait.

Institut für Scientific Computing – Universität WienP.Brezany Distributed Query Processing –An Overview Univ.-Prof. Dr. Peter Brezany Institut für Scientific.

Similar presentations


Presentation on theme: "Institut für Scientific Computing – Universität WienP.Brezany Distributed Query Processing –An Overview Univ.-Prof. Dr. Peter Brezany Institut für Scientific."— Presentation transcript:

1 Institut für Scientific Computing – Universität WienP.Brezany Distributed Query Processing –An Overview Univ.-Prof. Dr. Peter Brezany Institut für Scientific Computing Universität Wien Tel Sprechstunde: Di, LV-Portal:

2 Institut für Scientific Computing – Universität WienP.Brezany 2 Introduction To construct the answer to the query, the user does not precisely specify the procedure to follow  this procedure is devised by a DBMS module called query processor, which performs query optimization. The query processing problem is much more difficult in distributed environments  a larger number of parameters affect the performance: relations involved in a distributed query may be fragmented and/or replicated, thereby inducing communication costs. Furthermore, with many sites to access, query response time may become very high. The role of a distributed query processor is to map a high-level query (expressed in relational calculus) on a distributed DB (i.e., a set of global relations) into a sequence of DB operations (of relational algebra) on relation fragments. –The calculus query must be decomposed into a sequence of relational operations called an algebraic query. –The data accessed by the query must be localized so that the operations on relations are translated to bear on local data (fragments). –The algebraic query on fragments must be extended with communication operations and optimized with respect to a cost function (based on features of disk I/Os, CPUs, and communication networks) to be minimized.

3 Institut für Scientific Computing – Universität WienP.Brezany 3 Query Processing Problem Example 1: Consider EMP(ENO, ENAME, TITLE) ASG (ENO, PNO, RESP, DUR) and the following simple user query: „Find the names of employees who are managing a project“ In relational calculus using SQL: SELECTENAME FROMEMP, ASG WHEREEMP.ENO = ASG.ENO AND ASG.RESP = “Manager“ Two equivalent rel. algebra queries:  ENAME (  ASG.RESP=“Manager“  EMP.ENO=ASG.ENO (EMP  ASG) and  ENAME (ENAME ⋈ ENO (  ASG.RESP=“Manager“ (ASG))) consumes less comp. resources – it is intuitively obvious

4 Institut für Scientific Computing – Universität WienP.Brezany 4 Query Processing Problem (cont.) In a distributed system, rel. algebra is not enough to express execution strategy. It must be suplemented with operations for exchanging data between sites. The best sites to process data must also be selected. This increases the solution space. Example 2:  ENAME (ENAME ⋈ ENO (  RESP=“Manager“ (ASG))) We assume that EMP and ASG are horizontally fragmented: EMP 1 =  ENO  “E3“ (EMP) EMP 2 =  ENO  “E3“ (EMP) ASG 1 =  ENO  “E3“ (ASG) ASG 2 =  ENO  “E3“ (ASG) Fragments ASG 1, ASG 2, EMP 1, and EMP 2 are stored at sites 1, 2, 3, and 4, respectively, and the result is expected at site 5.

5 Institut für Scientific Computing – Universität WienP.Brezany 5 Equivalent Distributed Execution Strategies For the sake of pedagogical simplicity, the project operation is ignored. (b) Strategy B Strategy A exploits the fact that EMP and ASG are fragmented the same way in order to perform the select and join in parallel. Strategy B centralizes all the operand data at the result site before processing the query.

6 Institut für Scientific Computing – Universität WienP.Brezany 6 Simple Cost Model and Statistics tuple access: 1 unit (which we leave unspecified) tuple transfer: tuptrans: 10 units relations EMP and ASG have 400 and 1000 tuples, respectively. there are 20 managers in relation ASG. data is is uniformly distributed (fragmentation + allocation) among the sites. relations ASG and EMP are locally clustered on attributes RESP and ENO, respectively. Therefore, there is direct access to tuples of ASG (respectively, EMP) based on the value of attribute RESP (respectively, ENO)

7 Institut für Scientific Computing – Universität WienP.Brezany 7 Cost Estimation

8 Institut für Scientific Computing – Universität WienP.Brezany 8 Complexity of Relational Algebra Operations n denotes the relation cardinality

9 Institut für Scientific Computing – Universität WienP.Brezany 9 Layers of Query Processing

10 Institut für Scientific Computing – Universität WienP.Brezany 10 Query Decomposition – An Overview The distributed calculus query is decomposed into an algebraic query on global relations. The information about data distribution is not used.  The techniques used by this layer are those of a centralized DBMS. Query is rewritten in a normalized form that is suitable for subsequent manipulation. The normalized query is analyzed semantically so that incorrect queries are rejected as soon as possible. The correct query is simplified (e.g., eliminating redundant predicates). The calculus query is restructured as an algebraic query.  Several alg. queries can be derived from the same calc. query, but some alg. queries are “better“ than others. The quality is defined in terms of expected performance.

11 Institut für Scientific Computing – Universität WienP.Brezany 11 Data Localization – An Overview The main role is to localize the query‘s data using data distribution information. Repetition: Relations are fragmented; each being stored at a different site. Fragmentation is defined through fragmentation rules (fragmentation scheme). This layer determines which fragments are involved in the query and transforms the distributed query into a fragment query. 1.The distributed query is mapped into a fragment query by substituting each distributed relation by its recontructing program (materialization program). 2.The fragment query is simplified and restructured to produce another “good“ query (applying the same rules used in the decomposition layer).

12 Institut für Scientific Computing – Universität WienP.Brezany 12 Global and Local Query Optimization Global optimization: the goal is to find an execution strategy for the query which is close to optimal Local optimization: it is performed by all the sites having fragments involved in the query. Each subquery executing at one site, called a local query, is then optimized using the local schema of the site. At this time, the algorithms to perform the relational operations may be chosen. Local optimization uses the algorithms of centralized systems.

13 Institut für Scientific Computing – Universität WienP.Brezany 13 Query Decomposition – 1. Normalization The most important transformation is that of the query qualification (the WHERE clause), which may be arbitrarily complex, quantifier-free predicate or preceded by all necessary quantifiers (  or  ). Conjunctive and disjunctive normal forms. Rules for the transformation of the quantifier-free predicates.

14 Institut für Scientific Computing – Universität WienP.Brezany 14 Normalization (cont.) „Find the names of employees who have been working on project P1 for 12 or 24 months“ SELECTENAME FROMEMP, ASG WHEREEMP.ENO = ASG.ENO ANDASG.PNO = “P1“ AND DUR = 12 OR DUR = The qualification in conjunctive normal form is EMP.ENO = ASG.ENO  ASG.PNO = “P1“  (DUR = 12  DUR = 24) and in disjunctive normal form: (EMP.ENO = ASG.ENO  ASG.PNO = “P1“  DUR = 12)  (EMP.ENO = ASG.ENO  ASG.PNO = “P1“  DUR = 24) In the latter form, treating the two conjunctions may lead to redundant work if common subexpressions are not eliminated.

15 Institut für Scientific Computing – Universität WienP.Brezany 15 Query Decomposition - 2.Analysis The main reasons for query rejection (The query is simply returned to the user with an explanation.) are that the query is type incorrect or semantically incorrect. Example 1: The following query is type incorrect SELECTE# FROMEMP WHEREENAME > 200 for 2 reasons: (1) Attribute E# is not declared in the schema; and (2) Operation “>200“ is incompatible with the type string of ENAME.

16 Institut für Scientific Computing – Universität WienP.Brezany 16 Analysis (cont.) A query is semantically incorrect if components of it do not contribute in any way to the generation of the result. In the context of relational calculus, it is not possible to determine the semantic correctness of general queries. However, it is possible to do so for large class of relational queries, those which do not contain disjunction and negation. This is based on the representation of the query as a query graph or connection graph – one node indicates the result relation, and any other node indicates an operand relation. An edge between two nodes that are not results represents a join, whereas an edge whose destination node is the result represents a project. Furthermore, a nonresult node may be labeled by a select or a self-join predicate. An important subgraph of the relation connection graph is the join graph, in which only the joins are considered. The join graph is particularly useful in the query optimization phase.

17 Institut für Scientific Computing – Universität WienP.Brezany 17 Analysis (cont.) Example 1: “Find the names and responsibility of programmers who have been working on the CAD/CAM project for more than 3 years“ SELECTENAME, RESP FROMEMP, ASG, PROJ WHEREEMP.ENO = ASG.ENO ANDASG.PNO = PROJ.PNO ANDPNAME = “CAD/CAM“ ANDDUR >= 36 ANDTITLE = “Programmer“ The query graph and the corresponding join graph are shown in the next slide.

18 Institut für Scientific Computing – Universität WienP.Brezany 18 Analysis (cont.) Fig. 8.1

19 Institut für Scientific Computing – Universität WienP.Brezany 19 Analysis (cont.) The query graph is useful to determine the semantic correctness of a conjunctive multivariable query without negation. Such a query is semantically incorrect if its query graph is not connected. In this case one or more subgraphs (corresponding to subqueries) are disconnected from the graph that contains the result relation. The query could be considered correct (which some systems do) by considering the missing connection as a Cartesian product.

20 Institut für Scientific Computing – Universität WienP.Brezany 20 Analysis (cont.) Example 2: Let us consider the following query: SELECTENAME, RESP FROMEMP, ASG, PROJ WHEREEMP.ENO = ASG.PROJ ANDPNAME = “CAD/CAM“ ANDDUR >= 36 ANDTITLE = “Programmer“ Its query graph is disconnected, which tells us that the query is semantically incorrect.There are basically 3 solutions to the problem: (1) reject the query (2) assume that there is an implicit Cartesian product between relations ASG and PROJ, or (3) infer (using the schema) the missing join predicate ASG.PNO = PROJ.PNO which transforms the query into that of Example 1.

21 Institut für Scientific Computing – Universität WienP.Brezany 21 Query Decomposition - 3.Elimination of Redundancy The query qualification may contain redundant predicates.  A naive evaluation can well lead to duplicated work.  This can be eliminated by simplifying the qualification with the following well-known idempotency rules: Example:

22 Institut für Scientific Computing – Universität WienP.Brezany 22 Elimination of Redundancy (cont.) Example (cont.) Slide 12

23 Institut für Scientific Computing – Universität WienP.Brezany 23 Query Decomposition - 4.Rewriting The last step of query decomposition rewrites the query in relational algebra – in two substeps: (1) straighforward transformation of the query from relational calculus into relational algebra (2) restructuring of the relational algebra query to improve performance. It is customary to represent the relational algebra query graphically by an operator tree. Example 1: The query can be mapped in a straightforward way in the tree in the following slide.

24 Institut für Scientific Computing – Universität WienP.Brezany 24 Rewriting (cont.) By applying transformation rules, many different equivalent trees may be found  Vorlesung Datenbanksysteme (Prof. Schikuta)

25 Institut für Scientific Computing – Universität WienP.Brezany 25 Rewriting (cont.) Example 2: The restructuring of the tree in previous slide leads to the tree below. The resulting tree is good in the sense that repeated access to the same relation (as in the previous figure is avoided and that the most selective operations are done first. However, this tree is far from optimal.  The select operation on EMP is not very useful before the join because it does not greatly reduce the size of the operand relation.

26 Institut für Scientific Computing – Universität WienP.Brezany 26 Localization of Distributed Data This layer translates an algebraic query on global relations into an algebraic query expressed on physical fragments. Localization uses information stored in the fragment schema. Fragmentation is defined through fragmentation rules. Reduction techniques are a way how to localize a distributed query.

27 Institut für Scientific Computing – Universität WienP.Brezany 27 Reduction for Primary Horizontal Fragmentation The following example is used in subsequent discussions. Example: Relation EMP(ENO, ENAME, TITLE) can be split into three horizontal fragments EMP 1, EMP 2, and EMP 3, defined as follows: EMP 1 =  ENO  “E3“ (EMP) EMP 2 =  “E3“ < ENO  “E6“ (EMP) EMP 3 =  ENO > “E6“ (EMP) The localization program for an horizontally fragmented relation is the union of the fragments. EMP = EMP 1  EMP 2  EMP 3 Thus the generic form of any query specified on EMP is obtained by replacing it by (EMP 1  EMP 2  EMP 3 ). The reduction of queries consists primarily of detecting those subtrees that will produce empty relations, and removing them.

28 Institut für Scientific Computing – Universität WienP.Brezany 28 Reduction with Selection Selections on fragments that have a qualification contradicting the qualifications of the fragmentation rule generate empty relations. Given a relation R that has been horizontally fragmented as R 1, R 2,..., R w, where R j =  pj (R), the rule can be stated formally as follows: Rule:  pi (R j ) =  if  x in R :  (p i (x)  p j (x)) where p i and p j are selection predicates, x denotes a tuple, and p(x) denotes “predicate p holds for x.“ Determining the contradicting predicates requires theorem-proving techniques if the predicates are quite general. However, DBMSs generally simplify predicate comparison by supporting only simple predicates for defining fragmentation rules (by the DB administrator).

29 Institut für Scientific Computing – Universität WienP.Brezany 29 Reduction with Selection (cont.) Example: SELECT * FROM EMP WHERE ENO = “E5“ Applying the naive approach to localize EMP from EMP 1, EMP 2, and EMP 3 gives the generic query of Figure (a) below. It is easy to detect that the selection predicate contradicts the predicate of EMP 1 and EMP 3, thereby producing empty relations. The reduced query is simply applied to EMP 2 as shown in Figure (b).


Download ppt "Institut für Scientific Computing – Universität WienP.Brezany Distributed Query Processing –An Overview Univ.-Prof. Dr. Peter Brezany Institut für Scientific."

Similar presentations


Ads by Google