Understanding Recursion /1 Powerful computing/problem-solving techniques Examples Factorial: f(n) = 1, if n = 1 f(n) = f(n-1) * n, if n ≥ 1 Quick sort: Sort([x]) = [x] Sort([x1, …, pivot, … xn]) = sort[ys] ++ sort[zs]), where ys = [ x | x in xi, x ≤ pivot ] zs = [ x | x pivot ] 9/06/ f(0) = 0! = ??? List comprehension in Haskell or python

Understanding Recursion /2 Let C(n, m) be the number of ways to select m balls from n numbered balls Show that C(n, m) = C(n-1, m-1) + C(n-1, m) Example: m = 3, n = 5 Consider any ball in the 5 balls, e.g., ‘d’ 9/06/ entire solution space ?1?2?3 d?1?2 ?1 (not d)?2?3 abcde ?i in {}

Key Points Sub-problems need to be “smaller”, so that a simple/trivial boundary case can be reached Divide-and-conquer There may be multiple ways the entire solution space can be divided into disjoint sub-spaces, each of which can be conquered recursively. 9/06/2015 3

Outline We offer two ways to derive/interpret a BUC-like algorithm Slides 5-8: Slides 9-11: geometric derivation Slides 12-17: Simplified pseudo-code of BUC Misc stuffs Example 9/06/2015 4

Naïve Relational Cubing Method /1 All tuples in the cube has the format (a i, b j, c k, …, M l ) a i must be one of the values on dim A observed from the base cuboid + * (i.e. ALL)  collectively denoted as Dom*(A) Cubing R ABC = foreach (a i, b j, c k ) in Dom*(A) X Dom*(B) X Dom*(C) return ( a i, b j, c k, aggregate(R, (a i, b j, c k )) ) 9/06/ i.e., Cubing(R, ABC). We omit () for clarity. No confusion as Cubing(.,.) always takes 2 parameters

Naïve Relational Cubing Method /2 Rather than hardcode 3 nested loops, use recursion Cubing R ABC = [Cubing R a i BC | a i in Dom*(A)] Cubing R a i BC = [a i ] Cubing R BC is the Cartesian product; effectively prepending a i to every tuple from the recursion call Assertion (which is easy to prove): Cubing(.,.) returns the (almost) correct cube from R wrt the given set of dimensions 9/06/ Boundary case omitted. Try to write it by yourself.

Improved Relational Cubing Method /1 Cubing R ABC = [ [a i ] Cubing R BC | a i in Dom*(A)] Problem: may generate non- observed dimension value combinations. The choice of b j should depend on a i Fix: (1) pass tuples with A = ai to recursive calls (2) take bj values from those observed in the set of tuples passed in. 9/06/ [1] [2] [*] (1, 1, *, 0) is spurious

Improved Relational Cubing Method /2 Cubing R ABC = [ [a i ] Cubing ∏ BC σ A=ai (R) BC | a i in Dom*(R.A) ] 9/06/ [1] [2] [*]

Reduce Cube(in 2D) to Cube(in 1D) 9/06/ Geometric Interpretation /1 M11M12M13[Step 1] M21M22M23[Step 1] [Step 2] [Step 3] a1 a2 b1b2b3 M11M12M13[Step 1] b1b2b3 M21M22M23[Step 1] [Step 2] [Step 3] [a1] [a2] [a*]

Reduce Cube(in 3D) to Cube(in 2D) 9/06/ Geometric Interpretation /2

Reduce Cube(in 3D) to Cube(in 2D) 9/06/ Geometric Interpretation /3

12 Scaffolding BUC Alg: BottomUpCube(input, d)

BUC (Scaffolded) Explained Essentially the same as the improved recursive cubing algorithm Some recursion manually unfolded Computes coarse aggregation first (Line 1), mainly for iceberg cube computation Computes aggregates from cuboids interleavingly and in the order shown on the right 9/06/ ABC ABACBC AB C  Our: ABC, AB, AC, A, BC, B, C, ɸ BUC: ɸ, A, AB, ABC, AC, B, BC, C Cuboid AB = GROUP BY A, B = (a i, b j, [….])

An Alternative View of BUC’s Algorithm Divide the solution space (all tuples in the cube) in the following manner: A=a i A=*, B=b j A=*, B=*, C=c k … … ??? 9/06/ disjoint & complete (why?  write out the last bullet) Compute (d-1) dim cube Compute (d-2) dim cube … ???

15 Additional Advantage of Divide and Conquer: Locality of Access Increasingly important when dealing with large datasets, residing on disk (disk is slower than memory) in the memory (memory is slower than L2/1 cache; TLB misses) Each chunk of data is loaded once into the memory, and then we perform all the computation depending on it If (1, _, _ …) fits in the memory Compute all (1, _, _, …) without additional I/O cost Write out (*, _, _, …) No longer needed afterwards c.f., external memory sort

9/06/ BUC Example MCBA ABC ABACBC AB C  (*, *, *) = 150 AB C  (1, *, *) (2, *, *) AB (1, 1, *) (1, 2, *) (1, 3, *) AB ABC (1, 1, 1) ABC (1, 2, 1) ABC (1, 3, 1) (2, 1, *) ABC (2, 1, 1) AC (1, *, 1) (1, *, 2) AC (2, *, 1) (1, 1, 2)

9/06/ BUC Example (*, *, *) = 150 B C  (*, 1, *) BC (*, 1, 1) (*, 1, 2) MCB (*, 2, *) BC (*, 2, 1) (*, 3, *) BC (*, 3, 1) (*, *, 1) (*, *, 2) ABC ABACBC AB C  A ABC ABAC Note: strictly speaking, BUC uses depth-first traversal order and it is slightly different from what is shown in the animation here. E.g., when partitioning on B, it discovers three partitions, and will delve into the first partition (and calculate (*,1,*)); it will only access and perform computation for other partitions after all the (recursive) computation of the first partition is completed.