E.G.M. PetrakisTrees1 Definition (recursive): finite set of elements (nodes) which is either empty or it is partitioned into m + 1 disjoint subsets T 0,T 1,…T m where T 0 contains one node (root) and T 1, T 2 …T m are trees T0T0 T2T2 T1T1 TmTm … T:

E.G.M. PetrakisTrees2 Depth of Tree node: length of the path from the root to the node –The root has depth 0 Height of Tree: depth of the deepest node Leaf: node with no children Full Tree: each internal node has either two children or it is a leaf Complete Tree: all levels except perhaps level d-1 are full

E.G.M. PetrakisTrees3 A root root (T) = A descendents (B) = {F,G}  sons (B) ancestors (H) = {E}  parent (H) degree (T) = 4 : max number of sons degree (E) = 3 level (H) = 2 level (A) = 0 depth (T) = 3  max level DB E C H F JI K G sub-tree leaf

E.G.M. PetrakisTrees4 Full binary tree: binary tree of level d with exactly 2 l nodes at level l <= d –Total number of nodes N = 2 d+1 –1 –N = B A C EF D G All leaf nodes at the same level

E.G.M. PetrakisTrees5 Binary trees: finite set of nodes which is either empty or it is partitioned into sets T 0, T left, T right where T 0 is the root and T left, T right are binary trees If a binary tree has n nodes at level l then it has at most 2 n nodes at level l+1 Τ0Τ0 Τ left Τ right

E.G.M. PetrakisTrees6 Traversal: list all nodes in order a)Preorder (depth first order): root traverse T left preorder traverse T right preorder b)Inorder (symmetric order): traverse T left inorder root traverse T right inorder c)Postorder: traverse T left traverse T right root

E.G.M. PetrakisTrees7 A C E B F H G D I Preorder: ABDGCEHIF Inorder: DGBAHEICF Postorder: GDBHIEFCA

E.G.M. PetrakisTrees8 B A F C G J L I K E D H Preorder: ABCEIFJDGHKL Inorder: EICFJBGDKHLA Postorder: IEJFCGKLHDBA

E.G.M. PetrakisTrees9 Traversal of Trees of Degree >= 2 T0T0 T2T2 T1T1 TΜTΜ … preorder T 0 (root) T 1 preorder T 2 preorder.... T M preorder inorder T 1 inorder T 0 (root) T 2 inorder …. T M inorder postorder T 1 postorder T 2 postorder …. T M postorder T 0 (root)

E.G.M. PetrakisTrees10 B CED A G H F preorder: ABCDHEFG inorder: CBHDEAGF postorder: CHDEBGFA

E.G.M. PetrakisTrees11 Binary Tree Node ADT: class BinNode {//Bin.Tree node class public: BinNode* leftchild( ) const; //pointer to left child BinNode* rightchild( ) const //pointer to right child BELEM value( ) const; //node value void setValue(BELEM val); //set node value bool isLeaf( ) const; //check if node is a leaf };

E.G.M. PetrakisTrees12 enum bool {true = ‘1’, false=‘0’}; typedef int BELEM; class BinNode { public: BELEM element; BinNode * left, * right; BinNode ( ) {left = right = NULL;} //constructors BinNode (BELEM e,BinNode * l = NULL,BinNode * r = NULL) {element = e; left = l; right = r;} ~BinNode ( ) { } //destructor BinNode *leftchild ( ) const { return left;} …..

E.G.M. PetrakisTrees13 BinNode * rightchild ( ) const {return right ;} BELEM value ( ) const { return element;} void setValue (BELEM val) {element = val;} bool isLeaf ( ) const { return (left == NULL ) && ( right == NULL);} bool isLeft (BinNode * p) {….} }

E.G.M. PetrakisTrees14 void preorder (BinNode * tree) { if ( tree = = NULL) return ; else { …………….. cout << ‘’node:‘’ << tree  element << ‘’\n‘’ ; preorder ( tree  leftchild ( ) ); preorder (tree  rightchild ( ) ); }

E.G.M. PetrakisTrees15 Binary Search Trees (BST): binary tree –Each node stores key K –The nodes of T left have keys < K –The nodes of T right have keys >= K

E.G.M. PetrakisTrees16 Search a BST to find key x: –Compare with root –If root(key) == x  key found!! –If key(root) < x search the T left recursively –If key(root) >= x search T right recursively

E.G.M. PetrakisTrees17 Find the maximum key in a BST: follow nodes on T right until key is found or NULL Find the minimum key in a BST: follow nodes on T left max min

E.G.M. PetrakisTrees18 Insert key x in a BST: search for x in the BST until a leaf is found –Insert it as left child of leaf if key(leaf) < x –Insert it as right child of leaf if key(leaf) >= x ρ

E.G.M. PetrakisTrees19 Shape of BST depends on insertion order –Example: –Example: for ordered insertion sequences the BST becomes list

E.G.M. PetrakisTrees20 Delete key x from BST: three cases a)X is a leaf: simply delete it b)X is not a leaf; it has exactly one sub-tree The father node points to the node next to x Delete node(x) c)X is not a leaf; it has two sub-trees Find p: minimum of T right or maximum of T left This has at most one sub-tree!! Delete it as in (a) or (b) Substitute x with p

E.G.M. PetrakisTrees21 Delete key from BST: the key is a leaf

E.G.M. PetrakisTrees22 Delete key from BST: the key has exactly one sub-tree

E.G.M. PetrakisTrees23 Delete key from BST: the key has two sub-trees

E.G.M. PetrakisTrees24 Application of BST: sorting –Insert keys in a BST –Output inorder sequence of keys –insertion sequence: Complexity: Average case: O(nlogn) Worst case: O(n 2 )

E.G.M. PetrakisTrees25 Implementation of BSTs: each node stores the key and pointers to roots of T left, T right –Array-based implementation: known maximum number of nodes fast good for heaps –Dynamic memory allocation: no restriction on the number of nodes slower but more general

E.G.M. PetrakisTrees26 Implementation with dynamic memory: –Two C++ classes –BinNode: node class The same as in general binary trees Elementary Operations on node left, right, setValue, isleaf, father … –BST: tree class Composite operations on trees Find, insert, remove, traverse …. –Use of help operations: Easier interface to class Encapsulation: implementation becomes private

E.G.M. PetrakisTrees27 class BST { private: BinNode *root; void inserthelp (BinNode *&, const BELEM); void removehelp(BinNode *&,const BELEM); BinNode * findhelp (BinNode *,const BELEM) const; void printhelp (const BinNode *,const int)const; void clearhelp (BinNode *); public: BST ( ) { root = NULL; } ~BST ( ) { clearhelp (root); } void clear ( ) { clearhelp (root); root = NULL;} void insert (const BELEM & val) { inserthelp (root, val); } continued...

E.G.M. PetrakisTrees28 continued... void remove (const BELEM &val ) {removehelp (root, val); } BinNode *deletemin (BinNode *&); BinNode *find (const BELEM & val) const { return findhelp (root,val); bool isEmpty ( ) const {return root == NULL } void print ( ) const { if ( root == NULL ) cout < < ‘’BST empty \n ‘’ ; else printhelp (root, 0); }

E.G.M. PetrakisTrees29 BinNode*BST::findhelp(BinNode *rt, const BELEM val) const { if (rt == NULL) return NULL ; else if ( val < rt  value ( ) ) return findhelp (rt  leftchild ( ), val ); else if ( val == rt  value ( ) ) return rt; else return findhelp (rt  rightchild ( ), val ); } void BST :: inserthelp (BinNode *& rt, const BELEM val) { if (rt = = NULL) rt = new BinNode (val, NULL, NULL ); else if (val < rt  value ( ) ) inserthelp (rt  left, val ); else inserthelp (rt  right, val ); }

E.G.M. PetrakisTrees30 BinNode *BST:: deletemin (BinNode *& rt ) { assert (rt ! = NULL ); if (rt  left ! = NULL return deletemin (rt  left ); else { BinNode * temp = rt; rt = rt  right; return temp; }

E.G.M. PetrakisTrees31 Array-based implementation of trees –General for trees with degree d >= 2 –Many alternative implementations Α ΒC DEF G

E.G.M. PetrakisTrees32 Α ΒC DEF G Array with pointers to father nodes : 1 A 0 2 B 1 3 C 1 4 D 2 5 E 2 6 F 2 7 G 3 Easy to find ancestors Difficult to find children Minimum space

E.G.M. PetrakisTrees33 Array with pointers to children nodes 1 A B C 7 4 D 5 E 6 F 7 G –Easy to find children nodes –Difficult to find ancestor nodes Α ΒC DEF G

E.G.M. PetrakisTrees34 Array with lists to children nodes 1 Α B C 7 4 D 5 E 6 F 7 G –Easy to find children nodes –Difficult to find ancestor nodes Α ΒC DEF G

E.G.M. PetrakisTrees35 General array implementation r parent ( r ) = (r-1)/2 if 0 < r < n leftchild ( r ) = 2r+1 if 2r + 1 < n rightchild ( r ) = 2r+2 if 2r + 2 < n leftsibling ( r ) = r-1 if r even  0 < r < n rightsibling ( r ) = r+1 if r odd  0 < r+1 < n

E.G.M. PetrakisTrees36 Heap Binary tree (not BST), complete –Max heap: every node has a value >= children Max element: root

E.G.M. PetrakisTrees37 –Min heap: every node has a value <= children Min element: root

E.G.M. PetrakisTrees38 Operations on heaps: –Insert new element in heap –Build heap from n elements –Delete max (min) element from heap –Shift-down: move an element from the root to a leaf (elementary operation) Array-Based implementation Assume max heap

E.G.M. PetrakisTrees39 Insert: the new element is inserted as a leaf –Last in the array –Then shifted it to its proper position –Swap the new elements with parent elements as long as it has greater value –Complexity O(logn): at most d shift operations in array d: depth of tree New element => =>

E.G.M. PetrakisTrees40 Build a new heap from n elements: insert elements in an initially empty heap –Complexity: O(nlogn) –Better method: Build heap bottom-up Assume that H 1, H 2 are heaps and shift R down to its proper position in H 1 or H 2 (depending on which root from H 1, H 2 has greater value) Do the same (recursively) within H 1, H 2 R H2H2 H1H1

E.G.M. PetrakisTrees => R H2H2 H1H1 R H2H2 H1H1 4 Starts from d-2 level, the leafs need not be accessed

E.G.M. PetrakisTrees42 Complexity: –Up to n/2 elements at d-1 level  0 moves –Up to n/4 elements at d-2 level  1 moves/elem … –Up to n/2 i elements at level i  i-1 moves/elem Complexity (total number of moves) leafs

E.G.M. PetrakisTrees43 Delete: the max element is removed –Swap with last element in array –Delete last elements –Shift-down new root to find its proper position –Complexity: O(logn) => =>

E.G.M. PetrakisTrees44 class heap { private: ΕLEM* Heap ; int size ; int n ; void siftdown(int) ; public: heap(ELEM*,int, int) ; int heapsize( ) const ; bool isLeaf(int) const ; int leftchild(int) const ; int rightchild(int) const ; int parent(int) const ; void insert(const ELEM) ; ELEM removemax( ) ; ELEM remove(int) ; void buildheap( ) ; } ; // Pointer to the heap array // Maximum size of the heap // Number of elements now in the heap // Put an element in its correct place // Constructor // Return current size of the heap // TRUE if pos is a leaf position // Return position for left child // Return position for right child // Return position for parent of pos // Insert value into heap // Remove maximum value // Remove value at specified position // Heapify contents of Heap Class Heap

E.G.M. PetrakisTrees45 heap :: heap(ELEM* h, int num, int max) // Constructor { heap = h; n = num; size = max; buildheap( ); } int heap :: heapsize( ) const { return n; } // Size of heap bool heap :: isLeaf(int pos) const // TRUE if leaf { return (pos >= n/2) && (pos < n); } int heap :: leftchild(int pos) const {// position of left child assert(pos < n/2); // Must be a position with a left child return 2*pos + 1; } int heap :: rightchild(int pos) const {// position of right child assert(pos < (n-1)/2); // Must be a position with a left child return 2*pos + 2; }

E.G.M. PetrakisTrees46 int heap :: parent(int pos) const { // position for parent assert(pos > 0); // Posmust have parent return (pos-1)/2; } void heap :: insert(const ELEM val) { // Insert value assert(n < size); int curr = n++; Heap[curr] = val; // Start at end of heap, then // sift it up until curr’s parent > curr while ((curr != 0) && (key(Heap[curr]) > key(Heap[parent(curr)]))) { swap(Heap[curr], Heap[parent(curr)]); curr = parent(curr); }

E.G.M. PetrakisTrees47 ELEM heap :: removemax( ) { // Remove maximum value assert(n > 0); swap(Heap[0], Heap[--n]) ; // Swap maximum with last value if (n !=0) // Not on last element shiftdown(0); // Put new root in correct place return Heap[n]; }

E.G.M. PetrakisTrees48 // Remove value at specified position ELEM heap :: remove(int pos) { assert((pos > 0) && (pos < n)); swap(Heap[pos], Heap[--n]); // Swap with last value if (n != 0) // Not on last element shiftdown(pos); // Put new heap root val in correct place return Heap[n]; } // starts from elements with higher index values void heap :: buildheap( ) // Heapify contents of Heap { for (int i = n/2-1; i>= 0; i--) siftdown(i); }

E.G.M. PetrakisTrees49 void heap :: shiftdown(int pos) {//put elem in proper position assert((pos >= 0) && (pos < n)); while (!isLeaf(pos)) { int j = leftchild(pos); if ((j < (n-1)) && (key(Heap[j]) < key(Heap[j+1]))) j++; // j is now index of child with greater value if (key(Heap[pos]) >= key(Heap[j])) return; // Done swap(Heap[pos], Heap[j]); pos = j; // Move down } rightchild

E.G.M. PetrakisTrees50 Application: heap sort –Insert all elements in new heap –At step i delete max element Delete: Put max element last in array –Store this element at position i+1 –After n steps the array is sorted!! –Complexity: O(nlogn) why??

E.G.M. PetrakisTrees (a) Initial maxheap x[1] x[3] x[5] x[7] x[6] x[8] (b) x[8] = delete(92) (c) x[7] = delete(86) x[4] x[2]

E.G.M. PetrakisTrees (d) x[6] = delete(57) (e) x[5] = delete(48)

E.G.M. PetrakisTrees (ζ) x[4] = delete(37) (f) x[3] = delete(33) (g) x[2] = delete(25) x[1] x[3] x[2] x[4] x[8] x[5]x[6] x[7]

E.G.M. PetrakisTrees54 Huffman Coding Trees Goal: improvement in space requirements –In exchange of a penalty in running time Huffman coding assigns codes to symbols Main idea: the length of a code depends on its frequency –Code: binary number –Assign short codes to the more frequent symbols

E.G.M. PetrakisTrees55 Input: sequence of symbols (letters) –Typically 8 bits/symbol Output: a binary representation –The codes of all symbols concatenated in the same order –Requirement: A code cannot be prefix of another –n: Less than 8 bits/symbol on the average

E.G.M. PetrakisTrees56 The Huffman code of each letter is derived from a binary tree –Each leaf corresponds to a letter –Goal: build a tree with the min. external path weight (epw) –epw: sum. of weighted path lengths –Min epw  min. n –A letter with high weight (frequency) should have low depth  short code –A letter with low weight may be pushed deeper in the Huffman tree  longer code

E.G.M. PetrakisTrees57 Building the Huffman tree: –The tree is build bottom-up –Order the letters by ascending weight (frequency) –Remove the first two letters and assign them to leaves of the Huffman tree –Substitute the two letters with one with weight equal to the sum of their weights –Put this new element back on the ordered list (in its correct place) –Repeat until only one element (root of Huffman tree) remains in the list

E.G.M. PetrakisTrees58 Assigning codes to letters: beginning at the root: –Assign 0 to left branches and 1 to right branches –The Huffman code of a letter is the binary number determined by the path from the root to the leaf of that letter –Longer paths correspond to less frequent letters (and the reverse) – Replace each letter in the input with its code.

E.G.M. PetrakisTrees59 0 Weights: Symbols: a b c d e Weights: Symbols: b c e 4 2 da Weights: Symbols: b e 1 c d a 3 Weights: Symbols: b e c d a Weights: 1.0 Symbols: Ηuffman Tree b e c d a

E.G.M. PetrakisTrees60 Huffman codes a: 1111 b: 0 c:110 d:1110 e:10 Ηuffman Tree b e c d a  Input: acde  Huffman code:

E.G.M. PetrakisTrees61 Decoding: repeat until end of code –Beginning at the root, take right branch for each 1 and left branch for each 0 until we reach letter –Output the letter We need the Huffman tree for the decoding Example: ¦ ¦ 1 0 ¦ 0 ¦ a ¦ d ¦ e ¦ b ¦ c Ηuffman Tree b e c d a

E.G.M. PetrakisTrees62 Advantages: –Compression (40-80%) –Cheaper transmission –Some security (encryption) –Locally optimal code Disadvantages: –CPU cost –Difficult error correction –We need the Huffman tree for the decoding –Code not globally optimal