1. An Introduction to Programming Concepts and OI-programming …from abstract theory to dirty tricks…

2. Objectives Today Introduction to the concept of “ Algorithms ” Introduction to the concept of “ Algorithms ” Introduction to common algorithms in OI competitions Introduction to common algorithms in OI competitions Introduction to graph theory Introduction to graph theory Introduction to complexity Introduction to complexity “ Philosophy ” of OI competitions “ Philosophy ” of OI competitions OI-style programming OI-style programming

3. What is an Algorithm? From Wikipedia: An algorithm is a finite set of well- defined instructions for accomplishing some task which, given an initial state, will terminate in a corresponding recognizable end-state. From Wikipedia: An algorithm is a finite set of well- defined instructions for accomplishing some task which, given an initial state, will terminate in a corresponding recognizable end-state. (what does that mean?) (what does that mean?) Usually, an algorithm solves a “ problem ”. Usually, an algorithm solves a “ problem ”. Examples Examples Insertion sort Insertion sort Binary Search Binary Search An algorithm does not have to be a computer program! Think about other possible algorithms in real life An algorithm does not have to be a computer program! Think about other possible algorithms in real life

4. “ Problem ” s Usually a set of well defined inputs and corresponding outputs Usually a set of well defined inputs and corresponding outputs Example: the sorting problem: Example: the sorting problem: Input: a list of numbers Input: a list of numbers Output: a sorted list of numbers Output: a sorted list of numbers We can use a number of different sorting algorithms to solve the sorting problem We can use a number of different sorting algorithms to solve the sorting problem

5. Data Structures Supplementary objects that help store data in an algorithm Supplementary objects that help store data in an algorithm Different data structures have different properties, and can store different types of data, and access them in different ways Different data structures have different properties, and can store different types of data, and access them in different ways Selecting the right data structure can be very important, as you will learn later Selecting the right data structure can be very important, as you will learn later Examples: arrays, queues, stacks … more will be introduced later Examples: arrays, queues, stacks … more will be introduced later

6. Examples of algorithms Sorting algorithms Sorting algorithms Graph algorithms – Djikstra, Warshall-floyd, Bellman-Ford, Prims, Kruskal Graph algorithms – Djikstra, Warshall-floyd, Bellman-Ford, Prims, Kruskal Tree-Search algorithms – BFS, DFS Tree-Search algorithms – BFS, DFS Linear Searching Algorithms Linear Searching Algorithms

7. Examples of Data Structures Array – random access Array – random access Queue – First in First Out Queue – First in First Out Stack – First in Last Out Stack – First in Last Out Heap – extract min/max number Heap – extract min/max number Binary Search Tree – Efficient insert, search, delete, etc. Binary Search Tree – Efficient insert, search, delete, etc. Hash Table – fast lookup Hash Table – fast lookup Other graph data structures discussed below Other graph data structures discussed below

8. Examples of Techniques in Designing Algorithms Recursion Recursion Dynamic programming Dynamic programming Greedy Greedy Divide and conquer Divide and conquer Branch and bound Branch and bound (the above may have overlaps) (the above may have overlaps)

9. Using and Creating Algorithms “ It is science. You can derive them. ” “ It is art. We have no way to teach you! ” – Alan Tam Why study algorithms? Why study algorithms? To solve problems that can be directly solved by existing algorithms To solve problems that can be directly solved by existing algorithms To solve problems that can be solved by combining algorithms To solve problems that can be solved by combining algorithms To get feelings and inspirations on how to design new algorithms To get feelings and inspirations on how to design new algorithms

10. Related Issues Proving correctness of algorithms (why? why not?) Proving correctness of algorithms (why? why not?) Other methods: finding counter examples, “ Unu ’ s Conjecture of Competition ” Other methods: finding counter examples, “ Unu ’ s Conjecture of Competition ” Questions? Questions?

11. Graphs What is a graph? What is a graph? Informally, a set of relationships between things Informally, a set of relationships between things A graph is defined as G=(V,E), where A graph is defined as G=(V,E), where V is the set of vertices (singular: vertex) V is the set of vertices (singular: vertex) E is the set of edges that connect some of the vertices E is the set of edges that connect some of the vertices A path is a sequence of vertices which are connected by edge(s) A path is a sequence of vertices which are connected by edge(s)

12. Example Map of Australia Map of Australia WA NT SA Q NSW V T

13. Common Types of Graphs Directed/Undirected Graph Directed/Undirected Graph Weighted/Unweighted Graph Weighted/Unweighted Graph Connectivity Connectivity WA NT SA Q NSW V T

14. Trees A few common definitions (equivalent): A few common definitions (equivalent): Connected graph with no cycles Connected graph with no cycles There is a unique path between any two vertices There is a unique path between any two vertices Connected graph with v – 1 edges (v = num of vertices) Connected graph with v – 1 edges (v = num of vertices) Rooted/Unrooted Trees Rooted/Unrooted Trees Heap, Binary Search Trees Heap, Binary Search Trees

15. Representing a graph Adjacency Matrix Adjacency Matrix Adjacency List Adjacency List

16. Complexity What is complexity? What is complexity? We are not (yet!) concerned with the exact runtime or memory used We are not (yet!) concerned with the exact runtime or memory used We want to know how well an algorithm “ scales up ” (i.e. when there is a large input). Why? We want to know how well an algorithm “ scales up ” (i.e. when there is a large input). Why?

17. Complexity (cont ’ d) Here ’ s why: Here ’ s why:

18. Quasi-Formal Definition of Big-O (you need not remember these) (you need not remember these) We say f(x) is in O(g(x)) f(x) is in O(g(x)) if and only if there exist numbers x 0 and M such that there exist numbers x 0 and M such that |f(x)| ≤ M |g(x)| for x > x 0

19. Example 1 – Bubble sort For i := 1 to n do For j := i downto 2 do if a[j] > a[j-1] then swap(a[j], a[j-1]); For i := 1 to n do For j := i downto 2 do if a[j] > a[j-1] then swap(a[j], a[j-1]); Time Complexity? O(n 2 ) Time Complexity? O(n 2 ) “ Swap Complexity ” ? “ Swap Complexity ” ? How about memory? How about memory?

20. Example 2 – Insertion Sort Quick introduction to insertion sort (you will learn more in the searching and sorting training): Quick introduction to insertion sort (you will learn more in the searching and sorting training): [] 4 3 1 5 2 [] 4 3 1 5 2 [4] 3 1 5 2 [4] 3 1 5 2 [3 4] 1 5 2 [3 4] 1 5 2 [1 3 4] 5 2 [1 3 4] 5 2 [1 3 4 5] 2 [1 3 4 5] 2 [1 2 3 4 5] [1 2 3 4 5] Time Complexity = ? Time Complexity = ?

21. Applications Usually, the time complexity of the algorithm gives us a rough estimation of the actual run time. Usually, the time complexity of the algorithm gives us a rough estimation of the actual run time. O(n) for very large N O(n) for very large N O(n 2 ) for n ~ 1000-3000 O(n 2 ) for n ~ 1000-3000 O(n 3 ) for n ~ 100-200 O(n 3 ) for n ~ 100-200 O(n 4 ) for n ~ 50 O(n 4 ) for n ~ 50 O(k n ) for O(n!) for very small n, usually < 20 O(k n ) for O(n!) for very small n, usually < 20 Keep in mind Keep in mind The constant of the algorithms (including the implementation) The constant of the algorithms (including the implementation) Computers vary in speeds, so the time needed will be different Computers vary in speeds, so the time needed will be different Therefore remember to test the program/computer before making assumptions! Therefore remember to test the program/computer before making assumptions!

22. Problem I have implemented bubble sort for an Array A[N] and applied binary search on it. I have implemented bubble sort for an Array A[N] and applied binary search on it. Time complexity of bubble sort? Time complexity of bubble sort? O(N 2 ). No doubt. O(N 2 ). No doubt. Time complexity of binary search? Time complexity of binary search? O(lg N) O(lg N) Well, what is the time complexity of my algorithm? Well, what is the time complexity of my algorithm?

23. Properties O(f) + O(g) = max(O(f), O(g)) O(f) + O(g) = max(O(f), O(g)) O(f) * O(g) = O(fg) O(f) * O(g) = O(fg) So, what is the answer regarding to previous question? So, what is the answer regarding to previous question?

24. Some other notations (optional) f(N) is Θ(g(N)) f(N) is Θ(g(N)) iff f(N) is O(g(N)) and g(N) is O(f(N)) iff f(N) is O(g(N)) and g(N) is O(f(N)) f(N) is o(g(N)) f(N) is o(g(N)) For all C, there exists N 0 such that For all C, there exists N 0 such that |f(N)| N 0 f(N) is Ω(g(N)) f(N) is Ω(g(N)) iff g(N) is O(f(N)) iff g(N) is O(f(N)) Again no need to remember them Again no need to remember them

25. Computational Theory Topics P (Polynomical) P (Polynomical) Can be solved in polynomical time Can be solved in polynomical time NP (Non-deterministic Polynomical) NP (Non-deterministic Polynomical) Can be checked in polynomial time Can be checked in polynomial time NP does NOT stand for “ not-polynomial ” !! NP does NOT stand for “ not-polynomial ” !! NP-Complete NP-Complete The “ hardest ” NP problems The “ hardest ” NP problems

26. “ Philosophy ” of OI Competitions Objective of Competition … Objective of Competition … The winner is determined by: The winner is determined by: Fastest Program? Fastest Program? Amount of time used in coding? Amount of time used in coding? Number of Tasks Solved? Number of Tasks Solved? Use of the most difficult algorithm? Use of the most difficult algorithm? Highest Score Highest Score Therefore, during a competition, aim to get highest score, at all costs – “ All is fair in love and war. ” Therefore, during a competition, aim to get highest score, at all costs – “ All is fair in love and war. ”

27. Scoring A “ black box ” judging system A “ black box ” judging system Test data is fed into the program Test data is fed into the program Output is checked for correctness Output is checked for correctness No source code is manually inspected No source code is manually inspected How to take advantage (without cheating of course!) of the system? How to take advantage (without cheating of course!) of the system?

28. The OI Programming Process Reading the problems Reading the problems Choosing a problem Choosing a problem Reading the problem Reading the problem Thinking Thinking Coding Coding Testing Testing Finalizing the program Finalizing the program

29. Reading the Problem Usually, a task consists of Usually, a task consists of Title Title Problem Description Problem Description Constraints Constraints Input/Output Specification Input/Output Specification Sample Input/Output Sample Input/Output Scoring Scoring

30. Reading the Problem Constraints Constraints Range of variables Range of variables Execution Time Execution Time NEVER make assumptions yourself NEVER make assumptions yourself Ask whenever you are not sure Ask whenever you are not sure (Do not be afraid to ask questions!) (Do not be afraid to ask questions!) Read every word carefully Read every word carefully Make sure you understand before going on Make sure you understand before going on

31. Thinking Classify the problem Classify the problem Graph? Mathematics? Data Processing? Dynamic Programming? etc …. Graph? Mathematics? Data Processing? Dynamic Programming? etc …. Some complicated problems may be a combination of the above Some complicated problems may be a combination of the above Draw diagrams, use rough work, scribble … Draw diagrams, use rough work, scribble … Consider special cases (smallest, largest, etc) Consider special cases (smallest, largest, etc) Is the problem too simple? Is the problem too simple? Usually the problem setters have something they want to test the contestants, maybe an algorithm, some specific observations, carefulness etc. Usually the problem setters have something they want to test the contestants, maybe an algorithm, some specific observations, carefulness etc. Still no idea? Give up. Time is precious. Still no idea? Give up. Time is precious.

32. Designing the Solution Remember, before coding, you MUST have an idea what you are doing. If you don ’ t know what you are doing, do not begin coding. Remember, before coding, you MUST have an idea what you are doing. If you don ’ t know what you are doing, do not begin coding. Some points to consider: Some points to consider: Execution time (Time complexity) Execution time (Time complexity) Memory usage (Space complexity) Memory usage (Space complexity) Difficulty in coding Difficulty in coding Remember, during competition, use the algorithm that gains you most score, not the fastest/hardest algorithm! Remember, during competition, use the algorithm that gains you most score, not the fastest/hardest algorithm!

33. Coding Optimized for ease of coding, not for reading Optimized for ease of coding, not for reading Ignore all the “ coding practices ” outside, unless you find them particularly useful in OI competitions Ignore all the “ coding practices ” outside, unless you find them particularly useful in OI competitions No Comments needed No Comments needed Short variable names Short variable names Use less functions Use less functions NEVER use 16 bit integers (unless memory is limited) NEVER use 16 bit integers (unless memory is limited) 16 bit integer may be slower! (PC ’ s are usually 32-bit, even 64 bit architectures should be somewhat-optimized for 32 bit) 16 bit integer may be slower! (PC ’ s are usually 32-bit, even 64 bit architectures should be somewhat-optimized for 32 bit)

34. Coding (2) Use goto, break, etc in the appropriate situations Use goto, break, etc in the appropriate situations Never mind what Djikstra has to say Never mind what Djikstra has to say Avoid using floating point variables if possible (eg. real, double, etc) Avoid using floating point variables if possible (eg. real, double, etc) Do not do small (aka useless) “ optimizations ” to your code Do not do small (aka useless) “ optimizations ” to your code Save and compile frequently Save and compile frequently See example program code … See example program code …

35. Testing Sample Input/Output “ A problem has sample output for two reasons: Sample Input/Output “ A problem has sample output for two reasons: 1. To make you understand what the correct output format is 2. To make you believe that your incorrect solution has solved the problem correctly ” Manual Test Data Manual Test Data Generated Test Data (if time allows) Generated Test Data (if time allows) Boundary Cases (0, 1, other smallest cases) Boundary Cases (0, 1, other smallest cases) Large Cases (to check for TLE, overflows, etc) Large Cases (to check for TLE, overflows, etc) Tricky Cases Tricky Cases

36. Debugging Debugging – find out the bug, and remove it Debugging – find out the bug, and remove it Easiest method: writeln/printf/cout Easiest method: writeln/printf/cout It is so-called “ Debug message ” It is so-called “ Debug message ” Use of debuggers: Use of debuggers: FreePascal IDE debugger FreePascal IDE debugger gdb debugger gdb debugger

37. Finalizing Check output format Check output format Any trailing spaces? Missing end-of-lines? (for printf users, this is quite common) Any trailing spaces? Missing end-of-lines? (for printf users, this is quite common) better test once more with sample output better test once more with sample output Remember to clear those debug messages Remember to clear those debug messages Check I/O – filename? stdio? Check I/O – filename? stdio? Check exe/source file name Check exe/source file name Is the executable updated? Is the executable updated? Method of submission? Method of submission? Try to allocate ~5 mins at the end of competition for finalizing Try to allocate ~5 mins at the end of competition for finalizing

38. Interactive Tasks Traditional Tasks Traditional Tasks Give input in one go Give input in one go Give output in one go Give output in one go Interactive Tasks Interactive Tasks Your program is given some input Your program is given some input Your program gives some output Your program gives some output Your program is given some more input Your program is given some more input Your program gives more output Your program gives more output … etc … etc

39. Example “ Guess the number ” “ Guess the number ” Sample Run: Sample Run: Judge: I have a number between 1 and 5, can you guess? Judge: I have a number between 1 and 5, can you guess? Program: is it 1? Program: is it 1? J: Too small J: Too small P: 2? P: 2? J: Too small J: Too small P: 3? P: 3? J: Too small J: Too small P: 4? P: 4? J: Correct J: Correct P: 5? P: 5? J: Too big J: Too big P: Your number is 4! P: Your number is 4!

40. Open Test Data Test data is known Test data is known Usually quite difficult to solve Usually quite difficult to solve Some need time consuming algorithms, therefore you are given a few hours (i.e. competition time) to run the program Some need time consuming algorithms, therefore you are given a few hours (i.e. competition time) to run the program Tricks: Tricks: ALWAYS look at all the test data first ALWAYS look at all the test data first Solve by hand, manually Solve by hand, manually Solve partially by program, partially by hand Solve partially by program, partially by hand Some with different programs Some with different programs Solve all with one program (sometimes impossible!) Solve all with one program (sometimes impossible!) Make good use of existing tools – you do not have to write all the programs if some are already available! (eg. sort, other languages, etc) Make good use of existing tools – you do not have to write all the programs if some are already available! (eg. sort, other languages, etc)

41. Tricks “ No solution ” “ No solution ” Solve for simple cases Solve for simple cases 50% 50% Special cases (smallest, largest, etc) Special cases (smallest, largest, etc) Incorrect greedy algorithms Incorrect greedy algorithms Hard Code Hard Code Stupid Hardcode: begin writeln(random(100)); end. Stupid Hardcode: begin writeln(random(100)); end. Na ï ve hardcode: “ if input is x, output hc(x) ” Na ï ve hardcode: “ if input is x, output hc(x) ” More “ intelligent ” hardcode (sometimes not possible): pre-compute the values, and only save some of them More “ intelligent ” hardcode (sometimes not possible): pre-compute the values, and only save some of them Brute force Brute force Other Weird Tricks (not always useful … ) Other Weird Tricks (not always useful … ) Do nothing (e.g.. Toggle, IODM) Do nothing (e.g.. Toggle, IODM)

42. Pitfalls / Common Mistakes Misunderstanding the problem Misunderstanding the problem Not familiar with competition environment Not familiar with competition environment Output format Output format Using complex algorithms unnecessarily Using complex algorithms unnecessarily Choosing the hardest problem first Choosing the hardest problem first

43. Advertisement (targeted ad) NOI/IOI use Linux as competition environment exclusively NOI/IOI use Linux as competition environment exclusively We are thinking of providing Linux only environments for upcoming team formation test(s) We are thinking of providing Linux only environments for upcoming team formation test(s) Linux, when used properly, can be more powerful than Microsoft Windows TM for contests, because it has more powerful tools Linux, when used properly, can be more powerful than Microsoft Windows TM for contests, because it has more powerful tools Eg. Command Line tools, Powerful Editors (vim, emacs), etc. Eg. Command Line tools, Powerful Editors (vim, emacs), etc.

44. The End Note: most of the contents are introductions only. You may want to find more in-depth materials Note: most of the contents are introductions only. You may want to find more in-depth materials Books – Introduction to Algorithms Books – Introduction to Algorithms Online – Google, Wikipedia Online – Google, Wikipedia HKOI – Newsgroup, training websites of previous years, discuss with trainers/trainees. HKOI – Newsgroup, training websites of previous years, discuss with trainers/trainees. Training – Many topics are further covered in later trainings Training – Many topics are further covered in later trainings Experience! Experience! Any Questions? Any Questions?