Lecture 12: Revision Lecture Dr John Levine Algorithms and Complexity March 27th 2006

Lecture List Lecture 1: Why algorithms are so important Lecture 2: Big-Oh notation, a first look at different complexity classes (log, linear, log-linear, quadratic, polynomial, exponential, factorial) Lecture 3: simple search in a list of items is O(n) with unordered data but O(log n) with ordered data, and can be even faster with a good indexing scheme and parallel processors Lecture 4: sorting: random sort is O(n!), naïve sort is O(n 2 ), bubblesort is O(n 2 ), quicksort is O(n log n)

Lecture List Lecture 5: more on sorting: why comparison sort is O(n log n), doing better than O(n log n) by not doing comparisons (e.g. bucket sort) Lecture 6: harder search: how to represent a problem in terms of states and moves Lecture 7: uninformed search through states using an agenda: depth-first search and breadth-first search Lecture 8: making it smart: informed search using heuristics; how to use heuristic search without losing optimality – the A* algorithm

Lecture List Lecture 9: game tree search (Mark Dunlop’s notes), minimax and alpha-beta pruning Lecture 10: class review, ADT for lookup table, hash tables Lecture 11: Dijkstra’s algorithm Lecture 12: revision, NP-hard problems

Syllabus: Algorithmic Complexity Introduction to Algorithmic Complexity: basic algorithmic classification, with examples; the order notation (Big-oh); elementary complexity and estimation of run times; the tyranny of growth. Covered in specifically in Lectures 1 and 2 and Practical 1, also covered throughout the course Examinable Typical exam question: say what Big-oh is, give an analysis of some code fragments

Syllabus: Algorithmic Complexity Big-Oh notation (see Wikipedia article) The major classes of complexity: O(1), O(log n), O(n), O(n log n), O(n 2 ), O(n 3 ),…,O(a n ), O(n!) Analysis of nested loops and operations on arrays Why complexity is important: the tyranny of growth Easy problems (P) are solvable in polynomial time Hard problems (NP) are only solvable in exponential time It is thought that P ≠ NP but no-one has proved this

Syllabus: Searching and Sorting Searching and Sorting: the complexity of a range of techniques, including the divide and conquer approach; the relative complexity of searching and sorting algorithms; the sorting algorithms covered will include bubble sort, insertion sort, merge sort and quick sort; searching, including sequential search and the binary chop; hashing. Partially covered by Lectures 3 and 4 We’ll do a quick recap of the four sorting algorithms We’ll look at hash tables today Examinable

Syllabus: Sorting Insertion sort: insert items one at a time into the correct position in a sorted list, aka naïve sort, O(n 2 ) Bubble sort: pass through list swapping adjacent pairs, repeat until sorted, O(n 2 ) Quicksort: choose pivot as estimate of middle item, partition into two sets (> pivot, < pivot), then partition each set using the same process and repeat until the list is sorted, O(n log n) Merge sort: divide into two sets, sort each set, then merge the two sorted sets, O(n log n)

Syllabus: Sorting The best we can do by comparisons is O(n log n): n items implies n! possible unsorted sequences, each comparison divides this by 2, hence we need to make log(n!) comparisons to identify the sequence, and log(n!) ≈ n log n We don’t have to use comparisons: if we are sorting items which have a natural finite sequence, we can use something like bucket sort, which is O(n) There are numerous search algorithms: see the Wikipedia article for many more

Syllabus: Searching Linear search through an (unordered) list of items is O(n), and if n is very large we want to do better Binary search is O(log n); search using a binary tree to store the data is O(log n) Using a hash table to hold the data, the search is O(1), but finding a really good hash function is hard!

Syllabus: Game Trees Binary Trees revisited: implementations by array; expression trees; binary tree implementation of sorted list; access times; algorithms covered include traversal, searching, balancing and deletion. Already covered in Programming Techniques We did state space search and game trees instead Game trees, minimax search and alpha-beta pruning are examinable, the rest is not (because last year’s students didn’t do it) Typical exam question: map out a simple game tree and say what move the computer should make next

Syllabus: Game Trees Game trees: mapping out of states for each possible move by each player in turn Colouring the game tree: finding out what move the computer should make Minimax search: assume your opponent chooses the move to minimise the evaluation function, and you choose the move which forces the evaluation function to be maximised Alpha-beta pruning: don’t search branches of the game tree when you already know they are poor

Syllabus: Graph Algorithms Graphs revisited: directed and undirected graphs; representations of graphs; basic graph algorithms; applications of graphs to real world problems (for example telecommunications, transportation systems, dependencies between objects). To be covered next week Representation of graphs, Dijkstra’s algorithm Application to the tube map problem Examinable Typical exam question: show application of Dijkstra’s algorithm to a real world problem

Syllabus: Graph Algorithms Dijkstra’s algorithm: from initial vertex A, find all vertices reachable in a single step. Add the shortest reachable vertex and the path taken to get from A to the vertex to the set S. From this new vertex, find all vertices reachable in a single step. Now add the shortest reachable vertex which is not already in S to S. Repeat until all vertices are in S. Simple analysis: O(V 2 )

Syllabus: NP-Hard Problems Permutations and Combinations: branch and bound; greedy algorithms; backtracking search; typical problems, for example the TSP and related problems, the knapsack problem. Mostly covered by our consideration of state space search and games, and also in Topics 1 and 2 I will touch briefly on this area in my summary and revision lecture in Week 10 Not directly examinable – see the Week 10 lecture for what you need to know in this area

Syllabus: NP-Hard Problems What do you do with an NP-hard problem? Many of these are optimisation problems, so many solutions are feasible, very few are optimal Apply a polynomial time algorithm and make do with a less than perfect solution Run an improvement algorithm and wait for as long as you possibly can (always have a feasible solution) Branch-and-bound: similar to alpha-beta pruning Most instances are not hard: you may be able to find some structure in the instance which you can exploit