Chapter 5 : Trees

Basic Tree Concepts A tree consists of finite set of elements, called nodes, and a finite set of directed lines called branches, that connect the nodes. The number of branches associated with a node is the degree of the node.

Basic Tree Concepts When the branch is directed toward the node, it is indegree branch. When the branch is directed away from the node, it is an outdegree branch. The sum of the indegree and outdegree branches is the degree of the node. If the tree is not empty, the first node is called the root.

Basic Tree Concepts The indegree of the root is, by definition, zero. With the exception of the root, all of the nodes in a tree must have an indegree of exactly one; that is, they may have only one predecessor. All nodes in the tree can have zero, one, or more branches leaving them; that is, they may have outdegree of zero, one, or more.

Basic Tree Concepts A leaf is any node with an outdegree of zero, that is, a node with no successors. A node that is not a root or a leaf is known as an internal node. A node is a parent if it has successor nodes; that is, if it has outdegree greater than zero. A node with a predecessor is called a child.

Basic Tree Concepts Two or more nodes with the same parents are called siblings. An ancestor is any node in the path from the root to the node. A descendant is any node in the path below the parent node; that is, all nodes in the paths from a given node to a leaf are descendants of that node.

Basic Tree Concepts A path is a sequence of nodes in which each node is adjacent to the next node. The level of a node is its distance from the root. The root is at level 0, its children are at level 1, etc. …

Basic Tree Concepts The height of the tree is the level of the leaf in the longest path from the root plus 1. By definition the height of any empty tree is -1. A subtree is any connected structure below the root. The first node in the subtree is known is the root of the subtree.

Recursive definition of a tree A tree is a set of nodes that either: is empty or has a designated node, called the root, from which hierarchically descend zero or more subtrees, which are also trees.

Binary Trees A binary tree can have no more than two descendents. In this section we discuss the properties of binary trees, four different binary tree traversals

Binary Trees A binary tree is a tree in which no node can have more than two subtrees; the maximum outdegree for a node is two. In other words, a node can have zero, one, or two subtrees. These subtrees are designated as the left subtree and the right subtree.

A null tree is a tree with no nodes

Some Properties of Binary Trees The height of binary trees can be mathematically predicted Given that we need to store N nodes in a binary tree, the maximum height is A tree with a maximum height is rare. It occurs when all of the nodes in the entire tree have only one successor.

Some Properties of Binary Trees The minimum height of a binary tree is determined as follows: For instance, if there are three nodes to be stored in the binary tree (N=3) then Hmin=2.

Some Properties of Binary Trees Given a height of the binary tree, H, the minimum number of nodes in the tree is given as follows:

Some Properties of Binary Trees The formula for the maximum number of nodes is derived from the fact that each node can have only two descendents. Given a height of the binary tree, H, the maximum number of nodes in the tree is given as follows:

Some Properties of Binary Trees The children of any node in a tree can be accessed by following only one branch path, the one that leads to the desired node. The nodes at level 1, which are children of the root, can be accessed by following only one branch; the nodes of level 2 of a tree can be accessed by following only two branches from the root, etc. The balance factor of a binary tree is the difference in height between its left and right subtrees:

Balance of the tree B=0 B=0 B=1 B=-1 B=0 B=1 B=2 B=-2

Binary Search Tree

Binary Search Tree BST Property All elements stored in the left subtree of a node whose value is K have values less than K. All elements stored in the right subtree of a node whose value is K have values greater than or equal to K. That is, a node’s left child must have a key less than its parent, and a node’s right child must have a key greater than its parent.

Binary Search Tree Example:

Operations on BST ADT Binary Search Tree Create a BST Insert an element Remove an element Find an element Clear (remove all elements) Display all elements in a sorted order

If there is a left child visit it Visit the node itself Binary Search Tree Inorder traversal Visit the left subtree, then the node, then the right subtree. Algorithm: If there is a left child visit it Visit the node itself If there is a right child visit it

Binary Search Tree If there is a left child visit it Postorder traversal Visit each node after visiting its children. Algorithm: If there is a left child visit it If there is a right child visit it Visit the node itself

Binary Search Tree Visit the node itself Preorder traversal Visit each node then visit its children. Algorithm: Visit the node itself If there is a left child visit it If there is a right child visit it

Binary Search Tree Insert Algorithm If value we want to insert < key of current node, we have to go to the left subtree Otherwise we have to go to the right subtree If the current node is empty (not existing) create a node with the value we are inserting and place it here.

Binary Search Tree For example, inserting ’15’ into the BST?

Binary Search Tree Delete Algorithm How do we delete a node form BST? Similar to the insert function, after deletion of a node, the property of the BST must be maintained.

Binary Search Tree There are 3 possible cases Node to be deleted has no children  We just delete the node. Node to be deleted has only one child  Replace the node with its child and make the parent of the deleted node to be a parent of the child of the deleted node Node to be deleted has two children

Binary Search Tree Node to be deleted has two children

Binary Search Tree Node to be deleted has two children Steps: Find minimum value of right subtree Delete minimum node of right subtree but keep its value Replace the value of the node to be deleted by the minimum value whose node was deleted earlier.

Binary Search Tree

Expression Trees 9/10/2018

Expression Trees 9/10/2018

Coding Problem

Coding Problem Coding: assignment of bit strings to alphabet characters Codewords: bit strings assigned for characters of alphabet Two types of codes: fixed-length encoding (e.g., ASCII) variable-length encoding (e,g., Morse code) Prefix-free codes (or prefix-codes): no codeword is a prefix of another codeword Problem: If frequencies of the character occurrences are known, what is the best binary prefix-free code? E.g. We can code {a,b,c,d} as {00,01,10,11} or {0,10,110,111} or {0,01,10,101}. E.g. if P(a) = 0.4, P(b) = 0.3, P(c) = 0.2, P(d) = 0.1, then the average length of code #2 is 0.4 + 2*0.3 + 3*0.2 + 3*0.1 = 1.9 bits It allows for efficient (online) decoding! E.g. consider the encoded string (msg) 10010110… The one with the shortest average code length. The average code length represents on the average how many bits are required to transmit or store a character.

Huffman codes represents {00, 011, 1} Any binary tree with edges labeled with 0’s and 1’s yields a prefix-free code of characters assigned to its leaves Optimal binary tree minimizing the average length of a codeword can be constructed as follows: Huffman’s algorithm Initialize n one-node trees with alphabet characters and the tree weights with their frequencies. Repeat the following step n-1 times: join two binary trees with smallest weights into one (as left and right subtrees) and make its weight equal the sum of the weights of the two trees. Mark edges leading to left and right subtrees with 0’s and 1’s, respectively. 1 represents {00, 011, 1}

Example character A B C D _ frequency 0.35 0.1 0.2 0.2 0.15 codeword 11 100 00 01 101 average bits per character: 2.25 for fixed-length encoding: 3 compression ratio: (3-2.25)/3*100% = 25%

Minimum Spanning Tree

Minimum Spanning Tree (MST) Spanning tree of a connected graph G: a connected acyclic subgraph of G that includes all of G’s vertices Minimum spanning tree of a weighted, connected graph G: a spanning tree of G of the minimum total weight Example: c d b a 6 2 4 3 1 c d b a 6 4 1 c d b a 2 3 1

Prim’s MST algorithm Start with tree T1 consisting of one (any) vertex and “grow” tree one vertex at a time to produce MST through a series of expanding subtrees T1, T2, …, Tn On each iteration, construct Ti+1 from Ti by adding vertex not in Ti that is closest to those already in Ti (this is a “greedy” step!) Stop when all vertices are included

Example c d b a c d b a c d b a c d b a c d b a 4 2 6 1 3 4 2 6 1 3 4

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 4 Prim’s algorithm is an example of greedy algorithm. At every iteration, it chooses an edge with minimum cost that does not form a cycle. 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Prim’s Algorithm 2 1 3 1 3 2 2 1 4 1 3

Notes about Prim’s algorithm Proof by induction that this construction actually yields an MST (CLRS, Ch. 23.1). Main property is given in the next page. Needs priority queue for locating closest fringe vertex. The Detailed algorithm can be found in Levitin, P. 310. Efficiency O(n2) for weight matrix representation of graph and array implementation of priority queue O(m log n) for adjacency lists representation of graph with n vertices and m edges and min-heap implementation of the priority queue

The Crucial Property behind Prim’s Algorithm Claim: Let G = (V,E) be a weighted graph and (X,Y) be a partition of V (called a cut). Suppose e = (x,y) is an edge of E across the cut, where x is in X and y is in Y, and e has the minimum weight among all such crossing edges (called a light edge). Then there is an MST containing e. Y y x X

Another greedy algorithm for MST: Kruskal’s Sort the edges in nondecreasing order of lengths “Grow” tree one edge at a time to produce MST through a series of expanding forests F1, F2, …, Fn-1 On each iteration, add the next edge on the sorted list unless this would create a cycle. (If it would, skip the edge.)

Example c c c a a a d d d b b b c c c a a a d d d b b b 4 4 4 1 1 1 6 2 6 1 3 c d b a 4 2 6 1 3 c d b a 4 2 6 1 3 c d b a 4 2 6 1 3 c d b a 2 6 1 3 c d b a 2 1 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 Another algorithm for MST is Kruskal. Kruskal’s algorithm is also greedy – it repeatedly finds edges with minimum costs that does not form a cycle. Note that Prim’s algorithm “grows” a MST from a root. Kruskal’s algorithm forms a MST by connected forests. 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 4 3 1 3 2 4 4 2 3 3 4 1 5 3 4 1 4 3

Kruskal’s Algorithm 3 2 1 1 3 2 2 1 4 1 3

Notes about Kruskal’s algorithm Algorithm looks easier than Prim’s but is harder to implement (checking for cycles!) Cycle checking: a cycle is created iff added edge connects vertices in the same connected component Union-find algorithms – see section 9.2 Runs in O(m log m) time, with m = |E|. The time is mostly spent on sorting.

An example of MST A graph and one of its minimum costs spanning tree

An example for Prim’s algorithm

An example of Kruskal’s algorithm

Cycles and Cuts Cycle. Set of edges the form a-b, b-c, c-d, …, y-z, z-a. Cutset. A cut is a subset of nodes S. The corresponding cutset D is the subset of edges with exactly one endpoint in S. 2 3 1 6 4 Cycle C = 1-2, 2-3, 3-4, 4-5, 5-6, 6-1 5 7 8 2 3 1 Cut S = { 4, 5, 8 } Cutset D = 5-6, 5-7, 3-4, 3-5, 7-8 6 4 5 7 8

Definitions Chord (link) : It is an edge of graph that is not in tree. Complement of tree : The set of chord Cut – set : It is a (minimal) set of edges in a graph such that removal of the set will increase the no. of connected components in the remaining subgraph fundamental system of cut-sets : For a given spanning tree, the set v-1 cut-sets corresponding to v-1 branches of spanning tree fundamental cut-set : A cut set in fundamental system of cut-sets

Theorem Acircuit and the complement of any spanning tree must have at least one edge in common A cut-set and any spanning tree must have at least one edge in common Every circuit has, an even number of edges in common with every cut-set

Theorem For a given spanning tree, let D = {e1,e2,e3…..ek} be a fundamental cut-set in which e1 is a branch and e1,e2…ek are chords of the spanning tree. Then e1 is contained in the fundamental circuits corresponding to ei for i=2,3…k. Moreover e1 is not contained in any other fundamental circuits For a given spanning tree C = {e1,e2,e3…..ek} be a fundamental circuit in which e1 is a chord and e1,e2…ek are branches of the spanning tree. Then e1, is contained in the fundamental cut-sets corresponding to ei for i=2,3…k. Moreover e1 is not contained in any other fundamental cut-set