1. CSE 326: Data Structures Binary Search Trees

2. Today’s Outline Dictionary ADT / Search ADT Quick Tree Review Binary Search Trees

3. ADTs Seen So Far Stack –Push –Pop Queue –Enqueue –Dequeue Priority Queue –Insert –DeleteMin Then there is decreaseKey…

4. The Dictionary ADT Data: –a set of (key, value) pairs Operations: –Insert (key, value) –Find (key) –Remove (key) The Dictionary ADT is also called the “Map ADT” jfogarty James Fogarty CSE 666 phenry Peter Henry CSE 002 boqin Bo Qin CSE 002 insert(jfogarty, ….) find(boqin) boqin Bo, Qin, …

5. A Modest Few Uses Sets Dictionaries Networks: Router tables Operating systems: Page tables Compilers: Symbol tables 5 Probably the most widely used ADT!

6. Implementations Unsorted Linked-list Unsorted array Sorted array insert delete find

7. Tree Calculations Recall: height is max number of edges from root to a leaf Find the height of the tree... 7 t runtime:

8. Tree Calculations Example 8 A E B DF C G IH KJL M L N How high is this tree?

9. More Recursive Tree Calculations: Tree Traversals A traversal is an order for visiting all the nodes of a tree Three types: Pre-order:Root, left subtree, right subtree In-order:Left subtree, root, right subtree Post-order:Left subtree, right subtree, root + * 24 5 (an expression tree)

10. Inorder Traversal void traverse(BNode t){ if (t != NULL) traverse (t.left); process t.element; traverse (t.right); }

11. Binary Trees Binary tree is –a root –left subtree (maybe empty) –right subtree (maybe empty) Representation: 11 A B DE C F HG JI Data right pointer left pointer

12. Binary Tree: Representation 12 A right pointer left pointer A B DE C F B right pointer left pointer C right pointer left pointer D right pointer left pointer E right pointer left pointer F right pointer left pointer

13. Binary Tree: Special Cases A B DE C GF IH A B DE C F A B DE C GF Full Tree Complete TreePerfect Tree

14. Binary Tree: Some Numbers! For binary tree of height h: –max # of leaves: –max # of nodes: –min # of leaves: –min # of nodes: Average Depth for N nodes?

15. Binary Search Tree Data Structure Structural property –each node has  2 children –result: storage is small operations are simple average depth is small Order property –all keys in left subtree smaller than root’s key –all keys in right subtree larger than root’s key –result: easy to find any given key What must I know about what I store? 4 121062 115 8 14 13 79

16. Example and Counter-Example 3 1171 84 5 4 181062 115 8 20 21 BINARY SEARCH TREES? 7 15

17. Find in BST, Recursive Node Find(Object key, Node root) { if (root == NULL) return NULL; if (key < root.key) return Find(key, root.left); else if (key > root.key) return Find(key, root.right); else return root; } 2092 155 10 307 17 Runtime:

18. Find in BST, Iterative Node Find(Object key, Node root) { while (root != NULL && root.key != key) { if (key < root.key) root = root.left; else root = root.right; } return root; } 2092 155 10 307 17 Runtime:

19. Insert in BST 2092 155 10 307 17 Runtime: Insert(13) Insert(8) Insert(31) Insertions happen only at the leaves – easy!

20. BuildTree for BST Suppose keys 1, 2, 3, 4, 5, 6, 7, 8, 9 are inserted into an initially empty BST. Runtime depends on the order! –in given order –in reverse order –median first, then left median, right median, etc.

21. Bonus: FindMin/FindMax Find minimum Find maximum 2092 155 10 307 17

22. Deletion in BST 2092 155 10 307 17 Why might deletion be harder than insertion?

23. Lazy Deletion Instead of physically deleting nodes, just mark them as deleted +simpler +physical deletions done in batches +some adds just flip deleted flag –extra memory for “deleted” flag –many lazy deletions = slow finds –some operations may have to be modified (e.g., min and max) 2092 155 10 307 17

24. Non-lazy Deletion Removing an item disrupts the tree structure. Basic idea: find the node that is to be removed. Then “fix” the tree so that it is still a binary search tree. Three cases: –node has no children (leaf node) –node has one child –node has two children

25. Non-lazy Deletion – The Leaf Case 2092 155 10 307 17 Delete(17)

26. Deletion – The One Child Case 2092 155 10 307 Delete(15)

27. Deletion – The Two Child Case 3092 205 10 7 Delete(5) What can we replace 5 with?

28. Deletion – The Two Child Case Idea: Replace the deleted node with a value guaranteed to be between the two child subtrees Options: succ from right subtree: findMin(t.right) pred from left subtree : findMax(t.left) Now delete the original node containing succ or pred Leaf or one child case – easy!

29. Finally… 3092 207 10 7 replaces 5 Original node containing 7 gets deleted

30. Balanced BST Observation BST: the shallower the better! For a BST with n nodes –Average height is O(log n) –Worst case height is O(n) Simple cases such as insert(1, 2, 3,..., n) lead to the worst case scenario Solution: Require a Balance Condition that 1.ensures depth is O(log n) – strong enough! 2.is easy to maintain – not too strong!

31. Potential Balance Conditions 1.Left and right subtrees of the root have equal number of nodes 2.Left and right subtrees of the root have equal height

32. Potential Balance Conditions 3.Left and right subtrees of every node have equal number of nodes 4.Left and right subtrees of every node have equal height

33. CSE 326: Data Structures AVL Trees

34. Balanced BST Observation BST: the shallower the better! For a BST with n nodes –Average height is O(log n) –Worst case height is O(n) Simple cases such as insert(1, 2, 3,..., n) lead to the worst case scenario Solution: Require a Balance Condition that 1.ensures depth is O(log n) – strong enough! 2.is easy to maintain – not too strong!

35. Potential Balance Conditions 1.Left and right subtrees of the root have equal number of nodes 2.Left and right subtrees of the root have equal height 3.Left and right subtrees of every node have equal number of nodes 4.Left and right subtrees of every node have equal height

36. The AVL Balance Condition AVL balance property: Left and right subtrees of every node have heights differing by at most 1 Ensures small depth –Will prove this by showing that an AVL tree of height h must have a lot of (i.e. O(2 h )) nodes Easy to maintain –Using single and double rotations

37. The AVL Tree Data Structure Structural properties 1.Binary tree property (0,1, or 2 children) 2.Heights of left and right subtrees of every node differ by at most 1 Result: Worst case depth of any node is: O(log n) Ordering property –Same as for BST 4 121062 115 8 141379 15

38. 11 1 84 6 3 7 1 84 6 2 5 AVL trees or not? 10 12 7

39. Proving Shallowness Bound 121062 115 8 141379 15 Let S(h) be the min # of nodes in an AVL tree of height h Claim: S(h) = S(h-1) + S(h-2) + 1 Solution of recurrence: S(h) = O(2 h ) (like Fibonacci numbers) AVL tree of height h=4 with the min # of nodes (12)

40. Testing the Balance Property 2092 155 10 30177 NULL s have height -1 We need to be able to: 1. Track Balance 2. Detect Imbalance 3. Restore Balance

41. An AVL Tree 20 92 15 5 10 30 177 0 0 0 011 22 3 10 3 data height children

42. AVL trees: find, insert AVL find: –same as BST find. AVL insert: –same as BST insert, except may need to “fix” the AVL tree after inserting new value.

43. AVL tree insert Let x be the node where an imbalance occurs. Four cases to consider. The insertion is in the 1.left subtree of the left child of x. 2.right subtree of the left child of x. 3.left subtree of the right child of x. 4.right subtree of the right child of x. Idea: Cases 1 & 4 are solved by a single rotation. Cases 2 & 3 are solved by a double rotation.

44. Bad Case #1 Insert(6) Insert(3) Insert(1)

45. Fix: Apply Single Rotation 3 16 00 1 6 3 1 0 1 2 Single Rotation: 1. Rotate between x and child AVL Property violated at this node (x)

46. Single rotation in general a Z Y b X h h h h  -1 a Z Y b X h+1 hh X < b < Y < a < Z Height of tree before? Height of tree after? Effect on Ancestors?

47. Bad Case #2 Insert(1) Insert(6) Insert(3)

48. Fix: Apply Double Rotation 3 16 00 1 3 6 1 0 1 2 6 3 1 0 1 2 AVL Property violated at this node (x) Double Rotation 1. Rotate between x’s child and grandchild 2. Rotate between x and x’s new child

49. Double rotation in general a Z b W c X Y h-1 h h a Z b W c X Y hhh h  0 W < b <X < c < Y < a < Z Height of tree before? Height of tree after? Effect on Ancestors?

50. Double rotation, step 1 104 178 15 3 6 16 5 10 6 178 15 4 3 16 5

51. Double rotation, step 2 10 6 178 15 4 3 16 5 10 6 17 8 15 4 3 16 5

52. Imbalance at node X Single Rotation 1.Rotate between x and child Double Rotation 1.Rotate between x’s child and grandchild 2.Rotate between x and x’s new child

53. 9 5 2 11 7 1.single rotation? 2.double rotation? 3.no rotation? Inserting what integer values would cause the tree to need a: Single and Double Rotations: 13 30

54. Insertion into AVL tree 1.Find spot for new key 2.Hang new node there with this key 3.Search back up the path for imbalance 4.If there is an imbalance: case #1: Perform single rotation and exit case #2: Perform double rotation and exit Both rotations keep the subtree height unchanged. Hence only one rotation is sufficient!

55. Easy Insert 2092 155 10 3017 Insert(3) 12 0 0 100 12 3 0 Unbalanced?

56. Hard Insert (Bad Case #1) 2092 155 10 3017 Insert(33) 3 12 1 0 100 22 3 0 0 How to fix? Unbalanced?

57. Single Rotation 2092 155 10 30173 12 33 1 0 200 23 3 1 0 0 3092 205 10 333 15 1 0 110 22 3 0 0 1712 0

58. Hard Insert (Bad Case #2) Insert(18) 2092 155 10 30173 12 1 0 100 22 3 0 0 How to fix? Unbalanced?

59. Single Rotation (oops!) 2092 155 10 30173 12 1 1 200 23 3 0 0 3092 205 10 3 15 1 1 020 23 3 0 1712 0 18 0 0

60. Double Rotation (Step #1) 2092 155 10 30173 12 1 1 200 23 3 0 0 18 0 1792 155 10 203 12 1 200 23 3 1 0 30 0 18 0

61. Double Rotation (Step #2) 1792 155 10 203 12 1 200 23 3 1 0 30 0 18 0 2092 175 10 303 15 1 0 110 22 3 0 0 12 0 18

62. Insert into an AVL tree: 5, 8, 9, 4, 2, 7, 3, 1

63. CSE 326: Data Structures Splay Trees

64. AVL Trees Revisited Balance condition: Left and right subtrees of every node have heights differing by at most 1 –Strong enough : Worst case depth is O(log n) –Easy to maintain : one single or double rotation Guaranteed O(log n) running time for –Find ? –Insert ? –Delete ? –buildTree ?

65. Single and Double Rotations a Z Y b X h h h a Z b W c X Y h-1 h h

66. AVL Trees Revisited What extra info did we maintain in each node? Where were rotations performed? How did we locate this node?

67. Other Possibilities? Could use different balance conditions, different ways to maintain balance, different guarantees on running time, … Why aren’t AVL trees perfect? Many other balanced BST data structures –Red-Black trees –AA trees –Splay Trees –2-3 Trees –B-Trees –…

68. Splay Trees Blind adjusting version of AVL trees –Why worry about balances? Just rotate anyway! Amortized time per operations is O(log n) Worst case time per operation is O(n) –But guaranteed to happen rarely Insert/Find always rotate node to the root! SAT/GRE Analogy question: AVL is to Splay trees as ___________ is to __________

69. Recall: Amortized Complexity If a sequence of M operations takes O(M f(n)) time, we say the amortized runtime is O(f(n)). Amortized complexity is worst-case guarantee over sequences of operations. Worst case time per operation can still be large, say O(n) Worst case time for any sequence of M operations is O(M f(n)) Average time per operation for any sequence is O(f(n))

70. Recall: Amortized Complexity Is amortized guarantee any weaker than worstcase? Is amortized guarantee any stronger than averagecase? Is average case guarantee good enough in practice? Is amortized guarantee good enough in practice?

71. The Splay Tree Idea 17 10 92 5 If you’re forced to make a really deep access: Since you’re down there anyway, fix up a lot of deep nodes! 3

72. Find/Insert in Splay Trees 1.Find or insert a node k 2.Splay k to the root using: zig-zag, zig-zig, or plain old zig rotation Why could this be good?? 1.Helps the new root, k oGreat if k is accessed again 2.And helps many others! oGreat if many others on the path are accessed

73. Splaying node k to the root: Need to be careful! One option (that we won’t use) is to repeatedly use AVL single rotation until k becomes the root: (see Section 4.5.1 for details) s A k B C r D q E p F r D q E p F C s A B k

74. Splaying node k to the root: Need to be careful! What’s bad about this process? s A k B C r D q E p F r D q E p F C s A B k

75. Splay: Zig-Zag * g X p Y k Z W * Just like an… k Y g W p ZX Which nodes improve depth?

76. Splay: Zig-Zig * k Z Y p X g W g W X p Y k Z *Is this just two AVL single rotations in a row?

77. Special Case for Root: Zig p X k Y Z root k Z p Y X Relative depth of p, Y, Z?Relative depth of everyone else? Why not drop zig-zig and just zig all the way?

78. Splaying Example: Find(6) 2 1 3 4 5 6 Find(6) 2 1 3 6 5 4 ?

79. Still Splaying 6 2 1 3 6 5 4 1 6 3 25 4 ?

80. Finally… 1 6 3 25 4 6 1 3 25 4 ?

81. Another Splay: Find(4) Find(4) 6 1 3 25 4 6 1 4 35 2 ?

82. Example Splayed Out 6 1 4 35 2 61 4 35 2 ?

83. But Wait… What happened here? Didn’t two find operations take linear time instead of logarithmic? What about the amortized O(log n) guarantee?

84. Why Splaying Helps If a node n on the access path is at depth d before the splay, it’s at about depth d/2 after the splay Overall, nodes which are low on the access path tend to move closer to the root Splaying gets amortized O(log n) performance. (Maybe not now, but soon, and for the rest of the operations.)

85. Practical Benefit of Splaying No heights to maintain, no imbalance to check for –Less storage per node, easier to code Data accessed once, is often soon accessed again –Splaying does implicit caching by bringing it to the root

86. Splay Operations: Find Find the node in normal BST manner Splay the node to the root –if node not found, splay what would have been its parent What if we didn’t splay? Amortized guarantee fails! Bad sequence: find(leaf k), find(k), find(k), …

87. Splay Operations: Insert Insert the node in normal BST manner Splay the node to the root What if we didn’t splay?

88. Splay Operations: Remove find(k) LR k LR > k< k delete k Now what?

89. Join Join(L, R): given two trees such that (stuff in L) < (stuff in R), merge them: Splay on the maximum element in L, then attach R LR R L Does this work to join any two trees? splay max

90. Delete Example 91 6 47 2 Delete(4) find(4) 9 6 7 1 4 2 1 2 9 6 7 Find max 2 1 9 6 7 2 1 9 6 7

91. Splay Tree Summary All operations are in amortized O(log n) time Splaying can be done top-down; this may be better because: –only one pass –no recursion or parent pointers necessary –we didn’t cover top-down in class Splay trees are very effective search trees –Relatively simple –No extra fields required –Excellent locality properties: frequently accessed keys are cheap to find

92. Splay E E D C F I G B A H

93. B I H C D G F E A

94. CSE 326: Data Structures B-Trees

95. B-Trees Weiss Sec. 4.7

96. CPU (has registers) Cache Main Memory Disk TIme to access (conservative) 2-10 ns 40-100 ns a few milliseconds (5-10 Million ns) SRAM 8KB - 4MB DRAM up to 10GB many GB Cache Main Memory Disk 1 ns per instruction

97. Trees so far BST AVL Splay

98. AVL trees Suppose we have 100 million items (100,000,000): Depth of AVL Tree Number of Disk Accesses

99. M-ary Search Tree Maximum branching factor of M Complete tree has height = # disk accesses for find: Runtime of find:

100. Solution: B-Trees specialized M-ary search trees Each node has (up to) M-1 keys: –subtree between two keys x and y contains leaves with values v such that x  v < y Pick branching factor M such that each node takes one full {page, block} of memory 371221 x<3 3  x<77  x<1212  x<2121  x

101. B-Trees What makes them disk-friendly? 1.Many keys stored in a node All brought to memory/cache in one access! 2.Internal nodes contain only keys; Only leaf nodes contain keys and actual data The tree structure can be loaded into memory irrespective of data object size Data actually resides in disk

102. B-Tree: Example B-Tree with M = 4 (# pointers in internal node) and L = 4 (# data items in leaf) 1 AB 2 xG 3569101112 1517 202526 303233364042 506070 1040 3 15203050 Note: All leaves at the same depth! Data objects, that I’ll ignore in slides

103. B-Tree Properties ‡ –Data is stored at the leaves –All leaves are at the same depth and contains between  L/2  and L data items –Internal nodes store up to M-1 keys –Internal nodes have between  M/2  and M children –Root (special case) has between 2 and M children (or root could be a leaf) ‡ These are technically B + -Trees

104. Example, Again B-Tree with M = 4 and L = 4 12 3569101112 1517 202526 303233364042 506070 1040 3 15203050 (Only showing keys, but leaves also have data!)

105. Building a B-Tree The empty B-Tree M = 3 L = 2 3 Insert(3) 314 Insert(14) Now, Insert(1)?

106. Splitting the Root And create a new root 13 14 13 13 3 Insert(1) Too many keys in a leaf! So, split the leaf. M = 3 L = 2

107. Overflowing leaves Insert(59) 14 13 59 14 13 Insert(26) 14 13 2659 1459 13142659 And add a new child Too many keys in a leaf! So, split the leaf. M = 3 L = 2

108. Propagating Splits 1459 13142659 1459 142659 135 Insert(5) 514 2659 135 5 135 142659 14 Add new child Create a new root Split the leaf, but no space in parent! So, split the node. M = 3 L = 2

109. Insertion Algorithm 1.Insert the key in its leaf 2.If the leaf ends up with L+1 items, overflow! –Split the leaf into two nodes: original with  (L+1)/2  items new one with  (L+1)/2  items –Add the new child to the parent –If the parent ends up with M+1 items, overflow! 3.If an internal node ends up with M+1 items, overflow! –Split the node into two nodes: original with  (M+1)/2  items new one with  (M+1)/2  items –Add the new child to the parent –If the parent ends up with M+1 items, overflow! 4.Split an overflowed root in two and hang the new nodes under a new root This makes the tree deeper!

110. After More Routine Inserts 5 135 142659 14 5 135 265979 5989 14 89 Insert(89) Insert(79) M = 3 L = 2

111. Deletion 5 135 14265979 5989 14 89 5 135 142679 89 14 89 Delete(59) What could go wrong? 1.Delete item from leaf 2.Update keys of ancestors if necessary M = 3 L = 2

112. Deletion and Adoption 5 135 142679 8914 89 Delete(5) ? 13 142679 8914 89 3 133 142679 89 14 89 A leaf has too few keys! So, borrow from a sibling M = 3 L = 2

113. Does Adoption Always Work? What if the sibling doesn’t have enough for you to borrow from? e.g. you have  L/2  -1 and sibling has  L/2  ?

114. Deletion and Merging 3 1 3 142679 8914 89 Delete(3) ? 1 142679 8914 89 1 142679 8914 89 A leaf has too few keys! And no sibling with surplus! So, delete the leaf But now an internal node has too few subtrees! M = 3 L = 2

115. Adopt a neighbor 1 142679 89 14 89 14 1 2679 89 79 89 Deletion with Propagation (More Adoption) M = 3 L = 2

116. Delete(1) (adopt a sibling) 14 1 2679 89 79 89 A Bit More Adoption 26 14 26 79 89 79 89 M = 3 L = 2

117. Delete(26) 26 14 26 79 89 79 89 Pulling out the Root 14 79 89 79 89 A leaf has too few keys! And no sibling with surplus! 14 79 89 79 89 So, delete the leaf; merge A node has too few subtrees and no neighbor with surplus! 14 79 89 Delete the node But now the root has just one subtree! M = 3 L = 2

118. Pulling out the Root (continued) 14 79 89 The root has just one subtree! 14 79 89 Simply make the one child the new root! M = 3 L = 2

119. Deletion Algorithm 1.Remove the key from its leaf 2.If the leaf ends up with fewer than  L/2  items, underflow! –Adopt data from a sibling; update the parent –If adopting won’t work, delete node and merge with neighbor –If the parent ends up with fewer than  M/2  items, underflow!

120. Deletion Slide Two 3.If an internal node ends up with fewer than  M/2  items, underflow! –Adopt from a neighbor; update the parent –If adoption won’t work, merge with neighbor –If the parent ends up with fewer than  M/2  items, underflow! 4.If the root ends up with only one child, make the child the new root of the tree This reduces the height of the tree!

121. Thinking about B-Trees B-Tree insertion can cause (expensive) splitting and propagation B-Tree deletion can cause (cheap) adoption or (expensive) deletion, merging and propagation Propagation is rare if M and L are large (Why?) If M = L = 128, then a B-Tree of height 4 will store at least 30,000,000 items

122. Tree Names You Might Encounter FYI: –B-Trees with M = 3, L = x are called 2-3 trees Nodes can have 2 or 3 keys –B-Trees with M = 4, L = x are called 2-3-4 trees Nodes can have 2, 3, or 4 keys

123. K-D Trees and Quad Trees

124. Range Queries Think of a range query. –“Give me all customers aged 45-55.” –“Give me all accounts worth $5m to $15m” Can be done in time ________. What if we want both: –“Give me all customers aged 45-55 with accounts worth between $5m and $15m.”

125. Geometric Data Structures Organization of points, lines, planes, etc in support of faster processing Applications –Map information –Graphics - computing object intersections –Data compression - nearest neighbor search –Decision Trees - machine learning

126. k-d Trees Jon Bentley, 1975, while an undergraduate Tree used to store spatial data. –Nearest neighbor search. –Range queries. –Fast look-up k-d tree are guaranteed log 2 n depth where n is the number of points in the set. –Traditionally, k-d trees store points in d-dimensional space which are equivalent to vectors in d-dimensional space.

127. Range Queries x y a b f c g h e d i x y a b f c g h e d i Rectangular query Circular query

128. x y a b f c g h e d i Nearest Neighbor Search query Nearest neighbor is e.

129. k-d Tree Construction If there is just one point, form a leaf with that point. Otherwise, divide the points in half by a line perpendicular to one of the axes. Recursively construct k-d trees for the two sets of points. Division strategies –divide points perpendicular to the axis with widest spread. –divide in a round-robin fashion (book does it this way)

130. x y k-d Tree Construction a b f c g h e d i divide perpendicular to the widest spread.

131. y k-d Tree Construction (18) x a b c g h e d i s1 s2 yy s6 s3 x s4 y s7 y s8 y s5 x s1 s2 s3 s4 s5 s6 s7 s8 ab de gcfhi x f k-d tree cell

132. 2-d Tree Decomposition 1 2 3

133. x y k-d Tree Splitting a b c g h e d i adgbeichf acbdfehgi x y sorted points in each dimension f max spread is the max of f x -a x and i y - a y. In the selected dimension the middle point in the list splits the data. To build the sorted lists for the other dimensions scan the sorted list adding each point to one of two sorted lists. 1 2 3 4 5 6 7 8 9

134. x y k-d Tree Splitting a b c g h e d i adgbeichf acbdfehgi x y sorted points in each dimension f 1 2 3 4 5 6 7 8 9 abcdefghi 001001011 indicator for each set scan sorted points in y dimension and add to correct set abdegcfhi y

135. k-d Tree Construction Complexity First sort the points in each dimension. –O(dn log n) time and dn storage. –These are stored in A[1..d,1..n] Finding the widest spread and equally divide into two subsets can be done in O(dn) time. We have the recurrence –T(n,d) < 2T(n/2,d) + O(dn) Constructing the k-d tree can be done in O(dn log n) and dn storage

136. Node Structure for k-d Trees A node has 5 fields –axis (splitting axis) –value (splitting value) –left (left subtree) –right (right subtree) –point (holds a point if left and right children are null)

137. Rectangular Range Query Recursively search every cell that intersects the rectangle.

138. Rectangular Range Query (1) y x a b c g h e d i s1 s2 yy s6 s3 x s4 y s7 y s8 y s5 x s1 s2 s3 s4 s5 s6 s7 s8 ab de gcfh x f i

139. Rectangular Range Query (8) y x a b c g h e d i s1 s2 yy s6 s3 x s4 y s7 y s8 y s5 x s1 s2 s3 s4 s5 s6 s7 s8 ab d e g cfh x f i

140. Rectangular Range Query print_range(xlow, xhigh, ylow, yhigh :integer, root: node pointer) { Case { root = null: return; root.left = null: if xlow < root.point.x and root.point.x < xhigh and ylow < root.point.y and root.point.y < yhigh then print(root); else if(root.axis = “ x ” and xlow < root.value ) or (root.axis = “ y ” and ylow < root.value ) then print_range(xlow, xhigh, ylow, yhigh, root.left); if (root.axis = “ x ” and xlow > root.value ) or (root.axis = “ y ” and ylow > root.value ) then print_range(xlow, xhigh, ylow, yhigh, root.right); }}

141. k-d Tree Nearest Neighbor Search Search recursively to find the point in the same cell as the query. On the return search each subtree where a closer point than the one you already know about might be found.

142. y k-d Tree NNS (1) x a b c g h e d i s1 s2 yy s6 s3 x s4 y s7 y s8 y s5 x s1 s2 s3 s4 s5 s6 s7 s8 ab de gcfhi x f query point

143. y k-d Tree NNS x a b c g h e d i s1 s2 yy s6 s3 x s4 y s7 y s8 y s5 x s1 s2 s3 s4 s5 s6 s7 s8 ab d c fh i x f query point w e g

144. Notes on k-d NNS Has been shown to run in O(log n) average time per search in a reasonable model. Storage for the k-d tree is O(n). Preprocessing time is O(n log n) assuming d is a constant.

145. y Worst-Case for Nearest Neighbor Search query point Half of the points visited for a query Worst case O(n) But: on average (and in practice) nearest neighbor queries are O(log N) x

146. Quad Trees Space Partitioning x y a b c g e d f ge dbacf

147. A Bad Case x y a b

148. Notes on Quad Trees Number of nodes is O(n(1+ log(  /n))) where n is the number of points and  is the ratio of the width (or height) of the key space and the smallest distance between two points Height of the tree is O(log n + log  )

149. K-D vs Quad k-D Trees –Density balanced trees –Height of the tree is O(log n) with batch insertion –Good choice for high dimension –Supports insert, find, nearest neighbor, range queries Quad Trees –Space partitioning tree –May not be balanced –Not a good choice for high dimension –Supports insert, delete, find, nearest neighbor, range queries

150. Geometric Data Structures Geometric data structures are common. The k-d tree is one of the simplest. –Nearest neighbor search –Range queries Other data structures used for –3-d graphics models –Physical simulations