Data Structures Hanoch Levi and Uri Zwick March 2011 Lecture 3 Dynamic Sets / Dictionaries Binary Search Trees

2 Dictionaries/Dynamic sets Maintain a set of items. Each item has a key associated with it. Keys belong to a totally ordered universe, and can be compared with each other Support the following operations: Insert, Delete, Search, Min, Max, … Extremely useful data structure!

Abstract Data Type: Dictionaries Dic-Item(i,k) – Create a dictionary item containing object i with key k Item(a), Key(a) – The item and key contained in Dec-Item a. Dictionary() – Create an empty dictionary Insert(D,a) – Insert a into D Delete(D,a) – Delete a from D (assuming a is in D) Search(D,k) – Find a dic-item with key k in D, if any. Min(D) – Return the dic-item with the minimum key in D. Max(D) – Return the dic-item with the maximum key in D. Successor(D,a) – Return the successor of a in D. Predecessor(D,a) – Return the predecessor of a in D. Assume that dic-items have distinct keys

4 Implementing dictionaries using lists Store the dic-items in a list (in no particular order). Insert a new dic-item to an arbitrary position of the list, e.g., the first or last position. Delete a dic-item by either using a supplied pointer to it, or by first locating it in the list. Search, and other operations, are implemented by scanning the list.

5 Implementing dictionaries using doubly linked lists (ver. 1) Store the dic-items in a list, in no particular order. Insert a new dic-item to an arbitrary position of the list, e.g., the first or last position. Delete a dic-item using a supplied pointer to it. Search, and other operations, are implemented by scanning the list. Insert, Delete – O(1) time Other operations – O(n) time

6 Implementing dictionaries using doubly linked lists (ver. 2) Store the dic-items in a list, in increasing order of keys. Insert a new dic-item to the appropriate position Delete a dic-item by using a supplied pointer to it. Search is implemented by scanning the list, stopping when the key of the current item is larger than the key sought. Insert,Search – O(n) time (or O(n/2) “on average”) Delete – O(1) time Min, Max, Successor, Predecessor – O(1) time

7 Implementing dictionaries using (circular) arrays Store the dic-items in a list, in increasing order of keys. Insert a new dic-item to the appropriate position Delete a dic-item by using a supplied pointer to it. Search implemented using binary search. Insert, Delete – O(n) time (or O(n/2) ) Min, Max, Successor, Predecessor – O(1) time Search – O(log n)

8 Binary search Successful search: Search(38) high mid low

9 Binary search Unsuccessful search: Search(39) high mid low

10 Binary search Key k was found in position mid Key k should be inserted in position mid or mid+1 Key(Retrieve(L,mid))

11 Can we implement all operations in O(log n) time? Yes! Using Binary Search Trees

12 Binary search trees A binary tree in which each node contains a dic-item. Satisfies the binary-search-tree property: If y is in the left subtree of x, then y.key < x.key. If y is in the right subtree of x, then y.key > x.key right key left x parentitem

13 Binary search trees right key left x parentitem Dic-Item ≡ Tree-Node We assume that each Dic-Item now has room for the three additional pointers left, right, parent, which are initially set to null D.root

14 A set can be represented using several different trees We assume that all keys are distinct In most figures, we show the keys and ignore the items Height – length of the longest path from a root to a leaf height=2 height=4

15 Tree-Search(x,k) – Look for k in the subtree of x x Tree-Search(x,5) x x Search(D,k)  Tree-Search(D.root,k) We usually start the search at the root of the tree:

16 Tree-Position(x,k) – Look for k in the subtree of x Return the last node encountered x Tree-Position(x,6) x x y y y Returns the node containing 5 Tree-Position(x,k) is used to find insertion points

17 Printing the elements of a BST in sorted order (In-order walk) Printing, of course, is just an example…

18 Finding the minimum “keep going left”

19 Successor(x) If x has a right child, the successor of x is the minimal element in x.right. x What if x.right=null ? “Go right once, and then left all the way”

20 x Successor(x) If x.right=null, the successor of x is the lowest ancestor y of x such that x is in its right subtree y “Go up from x until the first turn right’’

21 x Successor(x) y If x has the largest key, then Successor(x)=null. Predecessor is symmetric

22 Insertions and deletions

Insertions Insert(6) Insert(9) 69

24 Binary Search Tree Animations For the time being, turn all buttons on the right off Warning: There are some differences with what we learn

Deletion: easy cases first Delete(6) – 6 is a leaf; simply remove it. Delete(8) – 8 has only one child; bypass it. 69 Delete(2) – more complicated… Delete(10) – 10 has only one child; bypass it.

26 Deletion more challenging case z y If z has two children, let y be the successor of z y has no left child Remove y from the tree Replace z by y Binary-search-tree property preserved! Is it enough to let z.key  y.key?

27 Analysis Each operation takes O(h+1) time, where h is the height of the tree In general h may be as large as n Want to keep the tree with small h

28 Balanced trees h = log 2 (n+1)−1 How do we keep the tree more or less balanced? A full tree of height h contains n=2 h+1 − 1 nodes

29 Randomly built BSTs Maybe balancing will take care of itself? Not if we insert the elements in sorted order. We get a path of length n Things are usually ok if we insert the elements in random order Theorem: If n distinct keys are inserted into a BST in random order, then the expected depth of each element is O(log n). Theorem: If n distinct keys are inserted into a BST in random order, then the expected height of the tree is O(log n). We want worst-case results…

30 Variation: Items only at the leaves Keep elements only at the leaves Each internal node contains a number to direct the search Costs space Internal part similar to The previous tree

31 Analysis Each operation takes O(h) time, where h is the height of the tree In general h may be as large as n Want to keep the tree with small h

32 Balance  h = O(log n) How do we keep the tree balanced through insertions and deletions ?

33 1) O(n) : Worst Case 2) כמה זמן לוקח בממוצע להכניס n איברים אקראיים? עץ מושלם (הכי טוב): עץ קווי (הכי גרוע): כל איבר לוקח לכל היותר log n ממוצע של O(n) לצומת Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי כל האיברים יחד לוקחים

34 זמן הכנסה ממוצע של עלה בין כל העצים: - רק INSERT -כל הסידורים של אלמנטים בעץ מתוך n! הסידורים האפשריים הם שווי הסתברות Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי הוכחה: מספר הנחות - עלות איבר = אורך המסלול אליו עלות ממוצעת = אורך המסלול הממוצע

35 איך נחשב אורך מסלול ממוצע? יהי L(n) אורך ממוצע של מסלול של צומת שרירותי בעץ בגודל n (אורך מסלול כאן = מס’ צמתים במסלול) בהנתן i מסוים, מה יהיה אורך המסלול הממוצע? Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי נשתמש בנוסחה רקורסיבית: a i (n-i-1) לכל עץ בעל n איברים, יש איבר ראשון שנסמן a ושני תתי עצים אם בתת העץ הראשון ישנם i איברים, בתת העץ השני ישנם (n-i-1) איברים אם האיבר שאנו מחפשים הוא a, האורך הוא 1 אם האיבר שאנו מחפשים קטן מ a האורך הוא 1 + L(i) אם האיבר שאנו מחפשים גדול מ a האורך הוא 1 + L(n-i-1)

36 שמאל ימין מרכז Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי ההסתברות לחיפוש כל איבר היא שווה, ולכן: ההסתברות שהאיבר שאנו מחפשים הוא a היא ההסתברות שהאיבר שאנו מחפשים קטן מ a היא ההסתברות שהאיבר שאנו מחפשים גדול מ a היא לכן בהנתן i אורך המסלול הממוצע הוא:

נגדיר את P i כהסתברות ש i איברים קטנים מ a Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי נסכום עבור כל ערך אפשרי של i: הסתברות זו שווה להסתברות ש a הוא האיבר ה i + 1, ו a יכול להיות כל אחד מהאיברים בהסתברות שווה, לכן 37

ולכן: Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי נשים לב ש: 38 כעת אפשר להוכיח באינדוקציה כי: נניח והתנאי מתקיים עבור כל i < n:

Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי 39

Data Structures, CS, TAU אנליזת זמן של עץ חיפוש בינארי 40

41 Balance  h = O(log n) How do we keep the tree balanced through insertions and deletions ? - Next chapter!

42 Rotations x y B C A y x B A C Right rotate Left rotate