1. Winter 2006CISC121 - Prof. McLeod1 Stuff Assn 5 is available and due Monday, 7pm. Pre-exam tutorial time. (Exam is April 27). April 18, room TBA. Office hours Wed. 26 th, time TBA.

2. Winter 2006CISC121 - Prof. McLeod2 Last Time Complexity of Mergesort. Started Quicksort.

3. Winter 2006CISC121 - Prof. McLeod3 Today Finish up Quicksort. How numbers are stored in Java. Sources and Effects of Roundoff Error.

4. Winter 2006CISC121 - Prof. McLeod4 Code for “preparation” method: public static void quickSort(int[] A) { int max = 0; for (int i = 1; i < A.length; i++) if (A[i] > A[max]) max = i; swap(A, A.length-1, max); quickSort(A, 0, A.length-2); } // end quickSort(array) Quicksort – Cont.

5. Winter 2006CISC121 - Prof. McLeod5 public static void quickSort (int[] A, int first, int last) { int lower = first + 1; int upper = last; swap(A, first, (first+last)/2); int pivot = A[first]; while (lower <= upper) { while (A[lower] < pivot) lower++; while (A[upper] > pivot) upper--; if (lower < upper) swap(A, lower++, upper--); else lower++; } swap(A, upper, first); if (first < upper - 1) quickSort(A, first, upper-1); if (upper + 1 < last) quickSort(A, upper+1, last); } // end quickSort(subarrays)

6. Winter 2006CISC121 - Prof. McLeod6 Example - sorting array: 8 5 4 7 6 1 6 3 8 12 10 After “preparation”: 8 5 4 7 6 1 6 3 8 10 12 After selection of pivot value and initial assignment of lower and upper: 6 5 4 7 8 1 6 3 8 10 12 Upper and lower move in until a swap has to be made: 6 5 4 7 8 1 6 3 8 10 12 Make the swap and move to next swap to be made: 6 5 4 3 8 1 6 7 8 10 12 Max value pivot lowerupper pivot lowerupper pivot lowerupper

7. Winter 2006CISC121 - Prof. McLeod7 Make the swap, and then upper and lower pass each other: 6 5 4 3 6 1 8 7 8 10 12 Put pivot value back in, and divide into two subarrays: 1 5 4 3 6 6 8 7 8 10 12 (Note how old pivot and max value are already in correct positions and do not need to be included in next recursive call). pivot lowerupper

8. Winter 2006CISC121 - Prof. McLeod8 Two recursive calls with subarrays: 4 5 1 3 667 8 8 10 12 After upper and lower have moved through arrays and pivots have been replaced: 13 4 5 667 8 8 10 12 Last set of recursive calls with subarrays: 1 3 4 5 66 7 8 8 10 12 Array is sorted: 1 3 4 5 6 6 7 8 8 10 12 pivot

9. Winter 2006CISC121 - Prof. McLeod9 Quicksort – Cont. Note that new arrays are not created, only the bounds of the subarrays in the array A are changed. The recursive calls will only contain different bounding values. And the subarrays get smaller for each call. The anchor case is when the size of the subarray is one.

10. Winter 2006CISC121 - Prof. McLeod10 Quicksort - Cont. The worst case is when a near-median value is not chosen – the pivot value is always a maximum or a minimum value. Now the algorithm is O(n 2 ). However, if the pivot values are always near the median value of the arrays, the algorithm is O(nlog(n)) – which is the best case. (See the derivation of this complexity for merge sort). The average case also turns out to be O(nlog(n)).

11. Winter 2006CISC121 - Prof. McLeod11 Choice of Pivot Value Several techniques can be used: –Choose first value –Choose middle value –Calculate “pseudo-median”. This is how quicksort is implemented in Arrays.sort(). –Why not calculate the actual median? The technique you use will depend how random the data set is.

12. Winter 2006CISC121 - Prof. McLeod12 Quicksort - Cont. So, the choice of the pivot value can be critical. Knowledge of the nature of the data to be sorted might suggest another algorithm to use in choosing the pivot value. Experiment has shown that Quicksort is faster than any other efficient sort. It has also been suggested that Quicksort can be made even faster by choosing to use a simple sort like insertion sort for subarray sizes less than about 30.

13. Winter 2006CISC121 - Prof. McLeod13 Comparison of Quicksort and Mergesort Mergesort is slower than Quicksort primarily because it does not do any preliminary shuffling of the data when it divides it. It also requires more memory because of the creation of temporary arrays (or lists) to complete the merge operation.

14. Winter 2006CISC121 - Prof. McLeod14 Comparison of Quicksort and Mergesort However, merge sort is very useful when sorting data that does not have all be in memory at once (“in place”). For example, if the data must be saved on disk because it will not all fit in memory, then mergesort will make the minimum number of file reads and writes. If you have to sort lists, it is very easy to code mergesort for them.

15. Winter 2006CISC121 - Prof. McLeod15 Comparison of Sorts Sorting 50,000 random int ’s (average case) on a PIII using Ready To Program. Selection sort is not influenced by the state of the data, where bubble and insertion are. SortTime (msec)# Comparisons# Swaps Insertion11000~623,000,00049,999 Selection18000~1,250,000,00050,000 Bubble40000~623,000,000 simple O(n 2 ) O(n)

16. Winter 2006CISC121 - Prof. McLeod16 Comparison of Sorts – Cont. SortTime (msec)# Comparisons# Swaps Shell50~1,300,000~1,784,000 Quicksort40360,000218,000 Merge (array)90718,100784,464 Merge (lists)1810718,100784,464 Radix (queues)960N/A1,000,000 Arrays.sort40?? Collections.sort (Vectors) 250?? Java Efficient O(n 1.25 ) O(nlog(n)) O(n)

17. Winter 2006CISC121 - Prof. McLeod17 Number Representation Binary numbers or “base 2” is a natural representation of numbers to a computer. As a transition, hexadecimal (or “hex”) numbers are also used. Octal (base 8) numbers are used to a lesser degree. Decimal (base 10) numbers are *not* naturally represented in computers.

18. Winter 2006CISC121 - Prof. McLeod18 Number Representation - Cont. If the number is in base “r” (the “radix”!) then the number is represented as: A n-1 ×r n-1 +A n-2 ×r n-2 + … A 1 ×r 1 + A 0 ×r 0 + +A -1 ×r -1 + A -2 ×r -2 + …A -m+1 ×r -m+1 + A -m ×r -m

19. Winter 2006CISC121 - Prof. McLeod19 Number Representation - Cont. For example, in base 10: when r=10, a decimal number: 365.24219878 is: 3×10 2 +6 ×10 1 +5×10 0 +2×10 -1 +4×10 -2 +2×10 -3 + +1×10 -4 +9×10 -5 +8×10 -6 +7×10 -7 +8×10 -8 = 365.24219878 (in base 10) 300 60 5 0.2 0.04 0.002 0.0001 0.00009 0.000008 0.0000007 + 0.00000008 _______________ 365.24219878

20. Winter 2006CISC121 - Prof. McLeod20 Number Representation - Cont. In base 2 (digits either 0 or 1): r=2, a binary number: (110101.11) 2 = 1×2 5 +1×2 4 +0×2 3 +1×2 2 +0×2 1 +1×2 0 +1×2 -1 +1×2 -2 = =53.75 (in base 10)

21. Winter 2006CISC121 - Prof. McLeod21 Number Representation - Cont. Octal Numbers: a base-8 system with 8 digits: 0, 1, 2, 3, 4, 5, 6 and 7: For example: (127.4) 8 = 1×8 2 +2×8 1 +7×8 0 +4×8 -1 =87.5.

22. Winter 2006CISC121 - Prof. McLeod22 Number Representation - Cont. Hexadecimal Numbers: a base-16 system with 16 digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, and F: For example: (B65F) 16 = 11×16 3 +6×16 2 +5×16 1 +15×16 0 = 46687.

23. Winter 2006CISC121 - Prof. McLeod23 Number Representation - Cont. The above series show how you can convert from binary, octal or hex to decimal. How to convert from decimal to one of the other bases?: integral part: divide by r and keep the remainder. decimal part: multiply by r and keep the carry “r” is the base - either 2, 8 or 16

24. Winter 2006CISC121 - Prof. McLeod24 Number Representation - Cont. For example, convert 625.76 10 to binary: So, 625 10 is 1001110001 2 Divisor(r)DividendRemainder 2 625 2 312 (quotient) 1 2 156 0 2 78 0 2 39 0 2 19 1 2 9 1 2 4 1 2 2 0 2 1 0 0 1 most significant digit least significant digit

25. Winter 2006CISC121 - Prof. McLeod25 Number Representation - Cont. For the “0.76 10 ” part: So, 0.76 10 is 0.110000101… 2 625.76 is: (1001110001.110000101...) 2 Multiplier(r)MultiplicandCarry 2 0.76 2 1.52(product) 1 2 1.04 1 2 0.08 0 2 0.16 0 2 0.32 0 2 0.64 0 2 1.28 1 2 0.56 0 2 1.02 1...

26. Winter 2006CISC121 - Prof. McLeod26 Aside - Roundoff Error From the previous example, you can see that exact base 10 decimals cannot always be exactly represented in binary. For example 0.1 10 is: 0.0001100110011001100110011… in base 2. Since numbers are stored in a finite amount of computer memory, some of this number will be lost - this is the source of “Roundoff Error”.

27. Winter 2006CISC121 - Prof. McLeod27 Number Representation - Cont. Converting between binary, octal and hex is much easier - done by “grouping” the numbers: For example: (010110001101011.111100000110) 2 =(?) 8 010 110 001 101 011. 111 100 000 110 (2 6 1 5 3. 7 4 0 6) 8

28. Winter 2006CISC121 - Prof. McLeod28 Number Representation - Cont. Another example: (2C6B.F06) 16 =(?) 2 (2 C 6 B. F 0 6) 16 ( 0010 1100 0110 1011. 1111 0000 0110) 2

29. Winter 2006CISC121 - Prof. McLeod29 From Before: Integer Primitive Types in Java For byte, from -128 to 127, inclusive (1 byte). For short, from -32768 to 32767, inclusive (2 bytes). For int, from -2147483648 to 2147483647, inclusive (4 bytes). For long, from -9223372036854775808 to 9223372036854775807, inclusive (8 bytes). A “byte” is 8 bits, where a “bit” is either 1 or 0.

30. Winter 2006CISC121 - Prof. McLeod30 Storage of Integers An “un-signed” 8 digit binary number can range from 00000000 to 11111111 00000000 is 0 in base 10. 11111111 is 1x2 0 + 1x2 1 + 1x2 2 + … + 1x2 7 = 255, base 10.

31. Winter 2006CISC121 - Prof. McLeod31 Storage of Integers - Cont. So, how can a negative binary number be stored? One way is to use the “two’s complement” system of storage. Make the most significant bit a negative number: So, the lowest “signed” binary 8 digit number is now: 10000000, which is -1x2 7, or -128 base 10.

32. Winter 2006CISC121 - Prof. McLeod32 Storage of Integers - Cont. Two’s Compliment System: binarybase 10 10000000-128 10000001-127 11111111 000000000 000000011 01111111127

33. Winter 2006CISC121 - Prof. McLeod33 Storage of Integers - Cont. For example, the binary number 10010101 is 1x2 0 + 1x2 2 + 1x2 4 - 1x2 7 = 1 + 4 + 16 - 128 = -107 base 10 Now you can see how the primitive integer type, byte, ranges from -128 to 127.

34. Winter 2006CISC121 - Prof. McLeod34 Storage of Integers - Cont. Suppose we wish to add 1 to the largest byte value: 01111111 +00000001 This would be equivalent to adding 1 to 127 in base 10 - the result would normally be 128. In base 2, using two’s compliment, the result of the addition is 10000000, which is -128 in base 10! So integer numbers wrap around, in the case of overflow - no warning is given in Java!

35. Winter 2006CISC121 - Prof. McLeod35 Storage of Integers - Cont. An int is stored in 4 bytes using “two’s complement”. An int ranges from: 10000000 00000000 00000000 00000000 to 01111111 11111111 11111111 11111111 or -2147483648 to 2147483647 in base 10

36. Winter 2006CISC121 - Prof. McLeod36 Real Primitive Types For float, (4 bytes) roughly ±1.4 x 10 -38 to ±3.4 x 10 38 to 7 significant digits. For double, (8 bytes) roughly ±4.9 x 10 -308 to ±1.7 x 10 308 to 15 significant digits.

37. Winter 2006CISC121 - Prof. McLeod37 Storage of Real Numbers The system used to store real numbers in Java complies with the IEEE standard number 754. Like an int, a float is stored in 4 bytes or 32 bits. These bits consist of 24 bits for the mantissa and 8 bits for the exponent: 00000000 00000000 mantissaexponent

38. Winter 2006CISC121 - Prof. McLeod38 Storage of Real Numbers - Cont. So a value is stored as: value = mantissa  2 exponent The exponent for a float can range from -128 to 128, which is about 10 -38 to 10 38. The float mantissa must lie between -1.0 and 1.0, and will have about 7 significant digits when converted to base 10.

39. Winter 2006CISC121 - Prof. McLeod39 Storage of Real Numbers - Cont. The double type is stored using 8 bytes or 64 bits - 53 bits for the mantissa, and 11 bits for the exponent. The exponent gives numbers between 2 -1024 and 2 1024, which is about 10 -308 and 10 308. The mantissa allows for the storage of about 16 significant digits in base 10. (Double.MAX_VALUE is: 1.7976931348623157E308 )

40. Winter 2006CISC121 - Prof. McLeod40 Storage of Real Numbers - Cont. See the following web site for more info: http://grouper.ieee.org/groups/754/

41. Winter 2006CISC121 - Prof. McLeod41 Storage of Real Numbers - Cont. So, a real number can only occupy a finite amount of storage in memory. This effect is very important for two kinds of numbers: –Numbers like 0.1 that can be written exactly in base 10, but cannot be stored exactly in base 2. –Real numbers (like  or e) that have an infinite number of digits in their “real” representation can only be stored in a finite number of digits in memory. And, we will see that it has an effect on the accuracy of mathematical operations.

42. Winter 2006CISC121 - Prof. McLeod42 Storage of “Real” or “Floating-Point” Numbers - Cont. Consider 0.1: (0.1) 10 = (0.0 0011 0011 0011 0011 0011…) 2 What happens to the part of a real number that cannot be stored? It is lost - the number is either truncated or rounded (truncated in Java). The “lost part” is called the “roundoff error”.

43. Winter 2006CISC121 - Prof. McLeod43 Storage of “Real” or “Floating-Point” Numbers - Cont. Back to the storage of 0.1: Compute: And, compare to 1000. float sum = 0; for (int i = 0; i < 10000; i++) sum += 0.1; System.out.println(sum);

44. Winter 2006CISC121 - Prof. McLeod44 Storage of “Real” or “Floating-Point” Numbers - Cont. Prints a value of 999.9029 to the screen. If sum is declared to be a double then the value: 1000.0000000001588 is printed to the screen. So, the individual roundoff errors have piled up to contribute to a cumulative error in this calculation. As expected, the roundoff error is smaller for a double than for a float.

45. Winter 2006CISC121 - Prof. McLeod45 Roundoff Error This error is referred to in two different ways: The absolute error: absolute error = |x - x approx | The relative error: relative error = (absolute error)  |x|

46. Winter 2006CISC121 - Prof. McLeod46 Roundoff Error - Cont. So for the calculation of 1000 as shown above, the errors are: The relative error on the storage of 0.1 is the absolute error divided by 1000. TypeAbsoluteRelative float0.09719.71E-5 double1.588E-101.588E-13

47. Winter 2006CISC121 - Prof. McLeod47 The Effects of Roundoff Error Roundoff error can have an effect on any arithmetic operation carried out involving real numbers. For example, consider subtracting two numbers that are very close together: Use the function for example. As x approaches zero, cos(x) approaches 1.

48. Winter 2006CISC121 - Prof. McLeod48 The Effects of Roundoff Error Using double variables, and a value of x of 1.0E-12, f(x) evaluates to 0.0. But, it can be shown that the function f(x) can also be represented by f’(x): For x = 1.0E-12, f’(x) evaluates to 5.0E-25. The f’(x) function is less susceptible to roundoff error.

49. Winter 2006CISC121 - Prof. McLeod49 The Effects of Roundoff Error - Cont. Another example. Consider the smallest root of the polynomial: ax 2 +bx+c=0: What happens when ac is small, compared to b? It is known that for the two roots, x 1 and x 2 :

50. Winter 2006CISC121 - Prof. McLeod50 The Effects of Roundoff Error - Cont. Which leads to an equation for the root which is not as susceptible to roundoff error: This equation approaches –c/b instead of zero when ac << b 2.

51. Winter 2006CISC121 - Prof. McLeod51 The Effects of Roundoff Error - Cont. The examples above show what can happen when two numbers that are very close are subtracted. Remember that this effect is a direct result of these numbers being stored with finite accuracy in memory.

52. Winter 2006CISC121 - Prof. McLeod52 The Effects of Roundoff Error - Cont. A similar effect occurs when an attempt is made to add a comparatively small number to a large number: boolean aVal = ((1.0E10 + 1.0E-20)==1.0E10); System.out.println(aVal); Prints out true to the screen Since 1.0E-20 is just too small to affect any of the bit values used to store 1.0E10. The small number would have to be about 1.0E-5 or larger to affect the large number. So, keep this behaviour in mind when designing expressions!

53. Winter 2006CISC121 - Prof. McLeod53 The Effect on Summations Taylor Series are used to approximate many functions. For example: For ln(2):

54. Winter 2006CISC121 - Prof. McLeod54 The Effect on Summations Since we cannot loop to infinity, how many terms would be sufficient? Since the sum is stored in a finite memory space, at some point the terms to be added will be much smaller than the sum itself. If the sum is stored in a float, which has about 7 significant digits, a term of about 1x10 -8 would not be significant. So, i would be about 10 8 - that’s a lot of iterations!

55. Winter 2006CISC121 - Prof. McLeod55 The Effect on Summations - Cont. On testing using a float, it took 33554433 iterations and 25540 msec to compute! (sum no longer changing, value = 0.6931375) Math.log(2) = 0.6931471805599453 So, roundoff error had a significant effect and the summation did not even provide the correct value. A float could only provide about 5 correct significant digits, tops. For double, about 10 15 iterations would be required! (I didn’t try this one…) So, this series does not converge quickly!

56. Winter 2006CISC121 - Prof. McLeod56 The Effect on Summations - Cont. Here is another way to compute natural logs: Using x = 1/3 will provide ln(2).

57. Winter 2006CISC121 - Prof. McLeod57 The Effect on Summations - Cont. For float, this took 8 iterations and <1msec (value = 0.6931472). Math.log(2) = 0.6931471805599453 For double, it took 17 iterations, <1 msec to give the value = 0.6931471805599451 Using the Windows calculator ln(2) = 0.693147180559945309417232121458177 (!!) So, the use of the 17 iterations still introduced a slight roundoff error. Note both loops are O(n) - but one is sure faster than the other!

58. Winter 2006CISC121 - Prof. McLeod58 Numeric Calculations Error is introduced into a calculation through two sources (assuming the formulae are correct!): –The inherent error in the numbers used in the calculation. –Error resulting from roundoff error. Often the inherent error dominates the roundoff error. But, watch for conditions of slow convergence or ill-conditioned matrices, where roundoff error will accumulate and end up swamping out the inherent error.

59. Winter 2006CISC121 - Prof. McLeod59 Numeric Calculations - Cont. Once a number is calculated, it is very important to be able to estimate the error using both sources, if necessary. The error must be known in order that the number produced by your program can be reported in a valid manner. This is a non-trivial topic in numeric calculation that we will not discuss in this course.