1. E.G.M. PetrakisTrees1  Definition (recursive): finite set of elements (nodes) which is either empty or it is partitioned into m + 1 disjoint subsets T 0,T 1,…T m where T 0 contains one node (root) and T 1, T 2 …T m are trees T0T0 T2T2 T1T1 TmTm … T:

2. E.G.M. PetrakisTrees2 Definitions  Depth of tree node: length of the path from the root to the node  the root has depth 0  Height of tree: depth of the deepest node  Leaf: no child nodes  Full tree: all levels are full  Complete tree: all levels except perhaps level d are full

3. E.G.M. PetrakisTrees3 root (tree) = A descendents (B) = {F,G}  sons (B) ancestors (H) = {E}  parent (H) degree (tree) = 4: max number of sons degree (E) = 3 level (H) = 2 level (A) = 0 depth (tree) = 3  max level root A DB E C H F JI K G sub-tree leaf

4. E.G.M. PetrakisTrees4 Binary Trees  Finite set of nodes which is either empty or it is partitioned into sets T 0, T left, T right where T 0 is the root and T left, T right are binary trees  A binary tree with n nodes at level l has at most2n nodes at level l+1 Τ0Τ0 Τ left Τ right

5. E.G.M. PetrakisTrees5 Full Binary Tree  d: depth  A binary tree has exactly 2 l nodes at level l <= d  Total number of nodes: N = 2 d+1 –1  N = B A C EF D G all leaf nodes at the same level

6. E.G.M. PetrakisTrees6 Binary Tree Traversal  Traversal: list all nodes in order a)preorder (depth first order): root traverse T left preorder traverse T right preorder b)inorder (symmetric order): traverse T left inorder root traverse T right inorder c)postorder: traverse T left traverse T right root

7. E.G.M. PetrakisTrees7 A C E B F H G D I preorder: ABDGCEHIF inorder: DGBAHEICF postorder: GDBHIEFCA

8. E.G.M. PetrakisTrees8 B A F C G J L I K E D H preorder: ABCEIFJDGHKL inorder: EICFJBGDKHLA postorder: IEJFCGKLHDBA

9. E.G.M. PetrakisTrees9 Traversal of Trees of Degree >= 2 T0T0 T2T2 T1T1 TΜTΜ … preorder T 0 (root) T 1 preorder T 2 preorder … T M preorder inorder T 1 inorder T 0 (root) T 2 inorder … T M inorder postorder T 1 postorder T 2 postorder … T M postorder T 0 (root)

10. E.G.M. PetrakisTrees10 B CED A G H F preorder: ABCDHEFG inorder: CBHDEAGF postorder: CHDEBGFA

11. E.G.M. PetrakisTrees11 Binary Tree Node ADT interface BinNode { public Object element(); // Return and set the element value public Object setElement(Object v); public BinNode left(); // Return and set the left child public BinNode setLeft(BinNode p); public BinNode right(); // Return and set the right child public BinNode setRight(BinNode p); public boolean isLeaf(); // Return true if this is a leaf node } // interface BinNode

12. E.G.M. PetrakisTrees12 Binary Tree Node Class class BinNodePtr implements BinNode { private Object element; // Object for this node private BinNode left; // Pointer to left child private BinNode right; // Pointer to right child public BinNodePtr( ) {left = right = null; } // Constructor 1 public BinNodePtr(Object val) { // Constructor 2 left = right = null; element = val; } public BinNodePtr(Object val, BinNode l, BinNode r) // Construct 3 { left = l; right = r; element = val; }

13. E.G.M. PetrakisTrees13 Binary Tree Node Class (cont.) // Return and set the element value public Object element() { return element; } public Object setElement(Object v) { return element = v; } // Return and set the left child public BinNode left() { return left; } public BinNode setLeft(BinNode p) { return left = p; } // Return and set the right child public BinNode right() { return right; } public BinNode setRight(BinNode p) { return right = p; } public boolean isLeaf() // Return true if this is a leaf node { return (left == null) && (right == null); } } // class BinNodePtr

14. E.G.M. PetrakisTrees14 Binary Tree Traversal void preorder (BinNode tree) { if (tree == NULL) return; else { System.out.print(tree.element() + " "); preorder ( tree.left( ) ); preorder ( tree.right( ) ); }

15. E.G.M. PetrakisTrees15 Binary Search Tree (BST)  Each node stores key K  The nodes of T left have keys < K  The nodes of T right have keys >= K 10 3 3015 20 1 25

16. E.G.M. PetrakisTrees16 Search in BST 1.Compare with root 2.If x == root(key) => key found !! 3.If x < key(root) search the T left recursively 4.If x >= key(root) search T right recursively 30 50 35 60 20 65

17. E.G.M. PetrakisTrees17 Find Min/Max Key  Find the maximum key in a BST: follow nodes on T right until key is found or NULL  Find the minimum key in a BST: follow nodes on T left 50 30 35 60 20 65 max min

18. E.G.M. PetrakisTrees18 Insert Key in BST  Search until a leaf is found  insert it as left child of leaf if key(leaf) > x  insert it as right child of leaf if key(leaf) <= x 50 35 40 37 60 30 20 65 ρ

19. E.G.M. PetrakisTrees19 Shape of BST  Depends on insertion order  For random insertion sequence the BST is more or less balanced  e.g., 10 20 5 7 1  For ordered insertion sequence the BST becomes list  e.g., 1 5 7 10 20 10 5 71 20 1 5 7 10

20. E.G.M. PetrakisTrees20 Delete key x from BST  Three cases: a) X is a leaf: simply delete it b) X is not a leaf & it has exactly one sub-tree  the father node points to the node next to x  delete node(x) c) X is not a leaf & it has two sub-trees  find p: minimum of T right or maximum of T left  this has at most one sub-tree!!  delete it as in (a) or (b)  substitute x with p

21. E.G.M. PetrakisTrees21 a) The key is a Leaf

22. E.G.M. PetrakisTrees22 b) The key has one Sub-Tree

23. E.G.M. PetrakisTrees23 c) The key has Two Sub-Trees

24. E.G.M. PetrakisTrees24 BST Sorting  Insert keys in BST  Insertion sequence: 50 30 60 20 35 40 37 65  Output keys inorder  Average case: O(nlogn)  Worst case: O(n 2 ) 30 50 37 60 35 20 40 65

25. E.G.M. PetrakisTrees25 Implementation of BSTs  Each node stores the key and pointers to roots of T left, T right  Array-based implementation:  known maximum number of nodes  fast  good for heaps  Dynamic memory allocation:  no restriction on the number of nodes  slower but general

26. E.G.M. PetrakisTrees26 Dynamic Memory Allocation  Two C++ classes  BinNode: node class (page 12)  node operations: left, right, setValue, isleaf, father …  BST: tree class  composite operations: find, insert, remove, traverse …  Use of “help” operations:  easier interface to class  No particular reason for this

27. E.G.M. PetrakisTrees27 Binary Tree Node Class class BinNodePtr implements BinNode { private Object element; // Object for this node private BinNode left; // Pointer to left child private BinNode right; // Pointer to right child public BinNodePtr( ) {left = right = null; } // Constructor 1 public BinNodePtr(Object val) { // Constructor 2 left = right = null; element = val; } public BinNodePtr(Object val, BinNode l, BinNode r) // Construct 3 { left = l; right = r; element = val; }

28. E.G.M. PetrakisTrees28 Binary Tree Node Class (cont.) // Return and set the element value public Object element() { return element; } public Object setElement(Object v) { return element = v; } // Return and set the left child public BinNode left() { return left; } public BinNode setLeft(BinNode p) { return left = p; } // Return and set the right child public BinNode right() { return right; } public BinNode setRight(BinNode p) { return right = p; } public boolean isLeaf() // Return true if this is a leaf node { return (left == null) && (right == null); } } // class BinNodePtr

29. E.G.M. PetrakisTrees29 Elem Interface Definition of an Object with support for a key field: interface Elem {// Interface for generic // element type public abstract int key(); // Key used for search // and ordering //more data here … (eg. Name, address …) } // interface Elem

30. E.G.M. PetrakisTrees30 BST Class class BST { // Binary Search Tree implementation private BinNode root; // The root of the tree public BST() { root = null; } // Initialize root to null public void clear() { root = null; } public void insert(Elem val) { root = inserthelp(root, val); } public void remove(int key) { root = removehelp(root, key); } public Elem find(int key) { return findhelp(root, key); } public boolean isEmpty() { return root == null; }

31. E.G.M. PetrakisTrees31 BST Class (cont.) public void print() { // Print out the BST if (root == null) System.out.println("The BST is empty."); else { printhelp(root, 0); System.out.println(); }

32. E.G.M. PetrakisTrees32 BST Class (cont.) private Elem findhelp(BinNode rt, int key) { if (rt == null) return null; Elem it = (Elem)rt.element(); if (it.key() > key) return findhelp(rt.left(), key); else if (it.key() == key) return it; else return findhelp(rt.right(), key); }

33. E.G.M. PetrakisTrees33 BST Class (cont.) private BinNode inserthelp(BinNode rt, Elem val) { if (rt == null) return new BinNodePtr(val); Elem it = (Elem)rt.element(); if (it.key() > val.key()) rt.setLeft(inserthelp(rt.left(), val)); else rt.setRight(inserthelp(rt.right(), val)); }

34. E.G.M. PetrakisTrees34 BST Class (cont.) private BinNode deletemin(BinNode rt) { if (rt.left() != null) rt.left = deletemin(rt.left()); else { BinNode temp = rt; rt = rt.right(); return temp; }

35. E.G.M. PetrakisTrees35 Array Implementation  General for trees with degree d >= 2  Many alternatives: 1.array with pointers to father nodes 2.array with pointers to children nodes 3.array with lists of children nodes 4.binary tree array (good for heaps)

36. E.G.M. PetrakisTrees36 a) Pointers to Father Nodes  Easy to find ancestors  Difficult to find children  Minimum space Α ΒC DEF G 1 A 0 2 B 1 3 C 1 4 D 2 5 E 2 6 F 2 7 G 3

37. E.G.M. PetrakisTrees37 b) Pointers to Children Nodes  Easy to find children nodes  Difficult to find ancestor nodes 0 A 1 2 1 B 3 4 5 2 C 6 3 D 4 E 5 F 6 G Α ΒC DEF G

38. E.G.M. PetrakisTrees38 c) Lists of Children Nodes 0 Α 1 2 1 B 3 4 5 2 C 6 3 D 4 E 5 F 6 G  Easy to find children nodes  Difficult to find ancestor nodes Α ΒC DEF G

39. E.G.M. PetrakisTrees39 d) Binary Tree Array  parent ( r ) = (r-1)/2 if 0 < r < n  leftchild ( r ) = 2r+1 if 2r + 1 < n  rightchild ( r ) = 2r+2 if 2r + 2 < n  leftsibling ( r ) = r-1 if r even & 0 < r < n  rightsibling ( r ) = r+1 if r odd & 0 < r+1 < n 7 46 1235 7 4 6 1 2 3 5 r:r: 0 1 2 3 4 5 6 7 4 6 1 2 3 5

40. E.G.M. PetrakisTrees40 Heap  Complete binary tree (not BST)  Min heap: every node has a value >= children  Max heap: every node has a value <= children 7 64 3 max element 3 64 9 min element

41. E.G.M. PetrakisTrees41 Heap Operations  insert new element in heap  build heap from n elements  delete max (min) element from heap  shift-down: move an element from the root to a leaf  In the following assume min heap

42. E.G.M. PetrakisTrees42 Insertion 1.Insert last in the array (as leaf) and 2.shift it to its proper position: swap it with parent elements as long as it has greater value  Complexity O(logn): at most d shift operations in array (d: depth of tree) new element 57 25 65 48 => 57 65 25 48 => 65 57 25 48

43. E.G.M. PetrakisTrees43 Build Head  Insert n elements in an empty heap  complexity: O(nlogn)  Better bottom-up method:  Initially data at random positions in array  Assume H 1, H 2 heaps and shift R down to its proper position (depending on which root from H 1, H 2 has greater value)  The same (recursively) within H 1, H 2 R H2H2 H1H1

44. E.G.M. PetrakisTrees44 Example  Starts at d-1 level: the leafs need not be accessed  Shift down elements 1 5 2 7 63 => 7 51 4263 7 56 4213 R H1H1 H2H2 R H1H1 H2H2 4

45. E.G.M. PetrakisTrees45 Complexity  Up to n/2 1 elements at d level => 0 moves/elem  Up to n/2 2 elements at d-1 level => 1 moves/elem …  Up to n/2 i elements at level i => i-1 moves/elem total number of moves

46. E.G.M. PetrakisTrees46 Deletion  The max element is removed  swap with last element in array  delete last element  shift-down root to its proper position  Complexity: O(logn) 65 57 25 48 => 25 57 65 48 => 57 2548

47. E.G.M. PetrakisTrees47 Class Heap public class Heap { // Heap class private Elem[] Heap; // Pointer to the heap array private int size; // Maximum size of the heap private int n; // Number of elements now in the heap public Heap(Elem[] h, int num, int max) // Constructor { Heap = h; n = num; size = max; buildheap(); } public int heapsize() // Return current size of the heap { return n; } public boolean isLeaf(int pos) // TRUE if pos is a leaf position { return (pos >= n/2) && (pos < n); }

48. E.G.M. PetrakisTrees48 Class Heap (cont.) // Return position for left child of pos public int leftchild(int pos) { Assert.notFalse(pos < n/2, "Position has no left child"); return 2*pos + 1; } // Return position for right child of pos public int rightchild(int pos) { Assert.notFalse(pos < (n-1)/2, "Position has no right child"); return 2*pos + 2; }

49. E.G.M. PetrakisTrees49 Class Heap (cont.) public int parent(int pos) { // Return position for parent Assert.notFalse(pos > 0, "Position has no parent"); return (pos-1)/2; } public void buildheap() // Heapify contents of Heap { for (int i=n/2-1; i>=0; i--) siftdown(i); }

50. E.G.M. PetrakisTrees50 Class Heap (cont.) // Put element in its correct place private void shiftdown(int pos) { Assert.notFalse((pos >= 0) && (pos < n), "Illegal heap position"); while (!isLeaf(pos)) { int j = leftchild(pos); if ((j Heap[j+1].key())) j++; // j is now index of child with greater value if (Heap[pos].key() <= Heap[j].key()) return; // Done DSutil.swap(Heap, pos, j); pos = j; // Move down }

51. E.G.M. PetrakisTrees51 Class Heap (cont.) public void insert(Elem val) { // Insert value into heap Assert.notFalse(n < size, "Heap is full"); int curr = n++; Heap[curr] = val; // Start at end of heap // Now sift up until curr's parent's > curr's key while ((curr!=0) && (Heap[curr].key()<Heap[parent(curr)].key())) { DSutil.swap(Heap, curr, parent(curr)); curr = parent(curr); }

52. E.G.M. PetrakisTrees52 Class Heap (cont.) public Elem removemin() { // Remove minimum value Assert.notFalse(n > 0, "Removing from empty heap"); DSutil.swap(Heap, 0, --n); // Swap minimum with last value if (n != 0) // Not on last element siftdown(0); // Put new heap root val in correct place return Heap[n]; }

53. E.G.M. PetrakisTrees53 Class Heap (cont.) // Remove value at specified position public Elem remove(int pos) { Assert.notFalse((pos > 0) && (pos < n), "Illegal heap position"); DSutil.swap(Heap, pos, --n); // Swap with last value if (n != 0) // Not on last element siftdown(pos); // Put new heap root val in correct place return Heap[n]; } } // Heap class

54. E.G.M. PetrakisTrees54 Heap Sort  Insert all elements in new heap array  At step i delete max element  delete: put max element last in array  Store this element at position i+1  After n steps the array is sorted!!  Complexity: O(nlogn) why??

55. E.G.M. PetrakisTrees55 92 3786 33124857 25 (a) initial maxheap 86 3757 33124825 92 57 3748 33122586 92 x[0] x[2] x[4] x[6] x[5] x[7] (b) x[7] = delete(92) (c) x[6] = delete(86) x[3] x[1]

56. E.G.M. PetrakisTrees56 48 3725 3312 92 5786 37 3325 12485786 92 (d) x[5] = delete(57) (e) x[4] = delete(48)

57. E.G.M. PetrakisTrees57 33 1225 37485786 92 25 1233 37485786 92 12 2533 37485786 92 (ζ) x[3] = delete(37) (f) x[2] = delete(33) (g) x[1] = delete(25) x[0] x[2] x[1] x[3] X[7] x[4]x[5] x[6]

58. E.G.M. PetrakisTrees58 Huffman Coding  Goal: improvement in space requirements in exchange of a penalty in running time  Assigns codes to symbols  code: binary number  Main idea: the length of a code depends on its frequency  shorter codes to more frequent symbols

59. E.G.M. PetrakisTrees59 Huffman Coding (cont.)  Input: sequence of symbols (letters)  typically 8 bits/symbol  Output: a binary representation  the codes of all symbols are concatenated in the same order they appear in the input  a code cannot be prefix of another  less than 8 bits/symbol on the average

60. E.G.M. PetrakisTrees60 Huffman Coding (cont.)  The code of each symbol is derived from a binary tree  each leaf corresponds to a letter  goal: build a tree with the minimum external path weight (epw)  epw: sum. of weighted path lengths  min epw min n  a letter with high weight (frequency) should have low depth => short code  a letter with low weight may be pushed deeper in the Huffman tree => longer code

61. E.G.M. PetrakisTrees61 Building the Huffman Tree  The tree is build bottom-up  Order the letters by ascending frequency  The first two letters become leaves of the Huffman tree  Substitute the two letters with one with weight equal to the sum of their weights  Put this new element back on the ordered list (in its correct place)  Repeat until only one element (root of Huffman tree) remains in the list

62. E.G.M. PetrakisTrees62 Assigning Codes to Letters  Beginning at the root assign 0 to left branches and 1 to right branches  The Huffman code of a letter is the binary number determined by the path from the root to the leaf of that letter  Longer paths correspond to less frequent letters (and the reverse)  Replace each letter in the input with its code

63. E.G.M. PetrakisTrees63 0 Weights:.12.40.15.08.25 Symbols: a b c d e Weights:.20.40.15.25 Symbols: b c e 4 2 da Weights:.35.40.25 Symbols: b e 1 c d a 3 Weights:.60.40 Symbols: b e c d a Weights: 1.0 Symbols: Ηuffman Tree b e c d a Huffman Example

64. E.G.M. PetrakisTrees64 Huffman codes a: 1111 b: 0 c:110 d:1110 e:10 Ηuffman Tree b e c d a => input: acde => Huffman code: 1111 110 1110 10 1 1 1 1 0 0 0 0 Huffman Encoding

65. E.G.M. PetrakisTrees65 Huffman Decoding  Repeat until end of code  beginning at the root, take right branch for each 1 and left branch for each 0 until we reach letter  output the letter 1 1 1 1 ¦ 1 1 1 0 ¦ 1 0 ¦ 0 ¦ 1 1 0 a ¦ d ¦ e ¦ b ¦ c Ηuffman Tree b e c d a

66. E.G.M. PetrakisTrees66 Critique  Advantages:  compression (40-80%)  cheaper transmission  some security (encryption)  locally optimal code  Disadvantages:  CPU cost  difficult error correction  we need the Huffman tree for the decoding  code not globally optimal