1. Algorithms and their Applications CS2004 (2012-2013) Dr Stephen Swift 1.2 Introduction to Algorithms

2. Introduction to AlgorithmsSlide 2 Introduction In this topic we will look at: What is a computer? What is computer science? What is an algorithm? Other concepts we should already know will also be discussed: Software Programming Language Implementation

3. Introduction to AlgorithmsSlide 3 Where Did it all Start? – Part 1 How did the computer come about? David Hilbert In ~1900, a very famous German Mathematician called David Hilbert (1862-1943) listed 23 unsolved mathematical problems He later refined problem number ten to ask “is there a way of determining if an arbitrary first- order symbolic logic statement is true or false?” (Do not worry what this means!)

4. Introduction to AlgorithmsSlide 4 Where Did it all Start? – Part 2 This became known as the “Entscheidungsproblem” or “Decision Problem” In 1936 Alan Turing (1912-1954) created two concepts called a “Turing Machine” and a “Universal Turing Machine” These were used to show that the “Decision Problem” was impossible to solve

5. Introduction to AlgorithmsSlide 5 Where Did it all Start? – Part 3 In 1939 Alan Turing used his ideas to create a machine called “The Bombe” which helped crack the German enigma code In 1948, the first computer (called the Baby) and associated computer program were developed at Manchester University, UK, based on Turing’s ideas

6. Introduction to AlgorithmsSlide 6 Computer Science Computer science is the study of computation A discipline that studies computable problems and computational structures A discipline that involves the understanding and design of computers and computational processes Is it a mathematical, scientific or engineering discipline? Interdisciplinary

7. Introduction to AlgorithmsSlide 7 What is Computation Computation is the procedure of calculating i.e. determining something by mathematical or logical methods “In natural science, Nature has given us a world and we’re just to discover its laws. In computers, we can stuff laws into it and create a world”, by A. Kay Computability is about what computers can do and cannot do Algorithmically Characterises problems that can be solved Algorithmically Exhibit problems that cannot be solved (e.g. the proof of the insolvability of the “halting problem”)

8. Introduction to AlgorithmsSlide 8 So What is a Computer and Computer Program? computer A computer is a device for carrying out certain types of algorithms algorithm An algorithm (in our context) is a set of steps for solving a problem computer programming language A computer programming language is a “language” for describing algorithms to computers computer program A computer program is (usually) a description of a single algorithm and/or problem Computer software Computer software is a collection of programs that carry out a similar task or related E.g. Windows

9. Introduction to AlgorithmsSlide 9 Why use a Computer? A computer can perform many mundane tasks much faster than people, E.g. adding up a large number of figures Some tasks can only be done by computers E.g. high precision robotics However there are many tasks that computer’s are no good at E.g. critically appraising a film

10. Introduction to AlgorithmsSlide 10 What is an Algorithm? InputOutput Algorithm An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output An algorithm can be seen as a tool for solving a well-specified computational problem

11. Introduction to AlgorithmsSlide 11 What is Implementation? Implementation is the task of turning an algorithm into a computer program Depending on the level of abstraction of the description of the algorithm, implementation can be anything between a straightforward translating task and a lengthy, difficult job

12. Introduction to AlgorithmsSlide 12 Programming Languages – Part 1 Computers really just understand binary, which is just sequences of zeros and ones A computer language is a method of allowing us (the user) to describe to a computer what we want it to do I.e. The specification of an Algorithm It is then translated (by the computer!) into binary so the computer can then understand it

13. Introduction to AlgorithmsSlide 13 Programming Languages – Part 2 Computer programming languages are similar to spoken languages They have structure and grammar They have words and phrases A incorrectly written program will not be understood by the computer and will result in an error Programmers usually use previously written programs if possible and combine them to solve new problems I.e. They try not to “reinvent the wheel”!

14. Introduction to AlgorithmsSlide 14 Why Study Algorithms? – Part 1 So why do we want to study algorithms? Some algorithms that solve the same problem can differ massively in their efficiency Computers may be getting faster but they are not infinitely fast! For example, sorting a list of billions of numbers could take a computer a very long time If we do not make the algorithms we write efficient then they may take too long to run If we do not analyse our algorithms we do not know how long they will take 4 years two weeks I am currently trying to make a program more efficient (by changing the algorithms and developing new algorithms) which will take about 4 years to run in its current state! I am hoping to get this down to around two weeks!!!

15. Introduction to AlgorithmsSlide 15 Why Study Algorithms? – Part 2 What makes the difference between a truly skilled programmer and an average one? Strong algorithmic knowledge and technique It might be that you can perform certain programming tasks without knowing much about algorithms However with a good grounding in algorithms, you can accomplish much more

16. Introduction to AlgorithmsSlide 16 What is the Analysis of an Algorithm? This is an indication of the time a computer will take to solve a problem given the size of the problem Time is abstract, and is usually measured in operations Multiplications, additions, swaps This allows us to compare two algorithms that try and solve the same problem Independent of the speed of a computer Also known as the computational complexity of an algorithm

17. Introduction to AlgorithmsSlide 17 Algorithm Example – Part 1 You have 70 coins (N=70) which are all “supposed” to be gold coins of the same weight, although you know that one coin is fake and weighs less than the others You have a balance scale; you can put any number of coins on each side of the scale at one time, and it will tell you if the two sides weigh the same, or which side is lighter if they don’t weigh the same

18. Introduction to AlgorithmsSlide 18 Algorithm Example – Part 2 Outline an algorithm for finding the fake coin How many times do you have to weigh? Easiest approach is not necessarily the best approach! Ideally, we want to design an algorithm that will take the shortest period of time We want an algorithm that is efficient, for example, what if we are then given 10,000 (N=10,000) coins and told that one is fake?

19. Introduction to AlgorithmsSlide 19 Example Solution One – Part 1 Take two coins at a time, one on each side of the scales: If weight(Left) = weight(Right) Then Take the next two coins Else Coin at the lighter side is fake End If Repeat the above until either the fake coin is found or you have checked all the coins

20. Introduction to AlgorithmsSlide 20 Example Solution One – Part 2 In the worst case, e.g. the fake coin is the last or the last but one, the IF Then-Else statement will have to run N/2 (35) times In the best case, e.g. the fake coin is the first or second, the same statement will only have to run once

21. Introduction to AlgorithmsSlide 21 Example Solution Two – Part 1 Divide-and-Conquer: Split the set of coins into two subsets of equal size; N/2 go to the left side of the scale, N/2 go to the right side If weight(left) ≠ weight(right) Then The lighter side contains the fake coin Split the lighter N/2 coins … Else Neither side contains the fake coin End If

22. Introduction to AlgorithmsSlide 22 Example Solution Two – Part 2 Binary Search Essentially this is a form of Binary Search This will be discussed later in the module The number of splits is equivalent to the number of levels (k) in the corresponding complete binary tree (we will cover this later) Since there is only one comparison on each level, the worst case is to conduct log 2 (N+1) comparisons Note: 2 x =y, log 2 (y) = x, e.g. log 2 (8) = 3 So when N=70 this will be less than 7 in total

23. Introduction to AlgorithmsSlide 23 Example Solution Comparison Solution 1: F1(N) = N/2, F1 ∈ O(N) Solution 2: F2(N) = log 2 (N+1), F2 ∈ O(log 2 (N))

24. Introduction to AlgorithmsSlide 24 The Analysis of Algorithms So there can be many competing algorithms for solving a given problem This module will enable you to gain experience in analysing and comparing different algorithms Later on in the module we will discuss that there are some problems that are impossible to solve computationally

25. Introduction to AlgorithmsSlide 25 Next Topic Lecture We will next look at algorithm analysis in more detail Laboratory There are no formal laboratories this week This weeks laboratory is a worksheet for you to attempt in your own time (practice)