1. Number Representation Part 2 Little-Endian vs. Big-Endian Representations Floating Point Representations Rounding Representation of the Galois Field elements ECE 645: Lecture 6

2. Required Reading Endianness, from Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki/Endianness Behrooz Parhami, Computer Arithmetic: Algorithms and Hardware Design Chapter 17, Floating-Point Representations

3. Little-Endian vs. Big-Endian Representation of Integers

4. Little-Endian vs. Big-Endian Representation A0 B1 C2 D3 E4 F5 67 89 16 LSB MSB MSB = A0 B1 C2 D3 E4 F5 67 LSB = 89 Big-Endian Little-Endian LSB = 89 0 MAX 67 F5 E4 D3 C2 B1 MSB = A0 address

5. Little-Endian vs. Big-Endian Camps Big-Endian Little-Endian 0 MAX address MSB LSB... LSB MSB... Motorola 68xx, 680x0 Intel IBM Hewlett-Packard DEC VAX Internet TCP/IP Sun SuperSPARC Bi-Endian Motorola Power PC Silicon Graphics MIPS RS 232 AMD

6. Origin of the terms Little-Endian vs. Big-Endian Jonathan Swift, Gulliver’s Travels A law requiring all citizens of Lilliput to break their soft-eggs at the little ends only A civil war breaking between the Little Endians and the Big-Endians, resulting in the Big Endians taking refuge on a nearby island, the kingdom of Blefuscu Satire over holy wars between Protestant Church of England and the Catholic Church of France

8. Little-Endian vs. Big-Endian Big-EndianLittle-Endian easier to determine a sign of the number easier to compare two numbers easier to divide two numbers easier to print easier addition and multiplication of multiprecision numbers Advantages and Disadvantages

9. Pointers (1) 89 67 F5 E4 D3 C2 B1 A0 Big-Endian Little-Endian 0 MAX address int * iptr; (* iptr) = 8967;(* iptr) = 6789; iptr+1

10. Pointers (2) 89 67 F5 E4 D3 C2 B1 A0 Big-Endian Little-Endian 0 MAX address long int * lptr; (* lptr) = 8967F5E4;(* lptr) = E4F56789; lptr + 1

11. Floating Point Representations

16. The ANSI/IEEE standard floating- point number representation formats Originally IEEE 754-1985. Superseded by IEEE 754- 2008 Standard.

19. Table 17.1 Some features of the ANSI/IEEE standard floatingpoint number representation formats

20. 00017FFEFF7E80 01127254255126128 –1260+127–1+1 Decimal code Hex code Exponent value f = 0: Representation of  0 f  0: Representation of denormals, 0.f  2 –126 f = 0: Representation of  f  0: Representation of NaNs Exponent encoding in 8 bits for the single/short (32-bit) ANSI/IEEE format 1.f  2 e Exponent Encoding

21. Fig. 17.4 Denormals in the IEEE single-precision format.

25. New IEEE 754-2008 Standard Basic Formats

26. New IEEE 754-2008 Standard Binary Interchange Formats

27. Requirements for Arithmetic Results of the 4 basic arithmetic operations (+, , ,  ) as well as square- rooting must match those obtained if all intermediate computations were infinitely precise That is, a floating-point arithmetic operation should introduce no more imprecision than the error attributable to the final rounding of a result that has no exact representation (this is the best possible) Example: (1 + 2  1 )  (1 + 2  23 ) Rounded result1 + 2  1 + 2  22 Error = ½ ulp Exact result1 + 2  1 + 2  23 + 2  24

28. Rounding 101

29. The IEEE 754-2008 standard includes five rounding modes: Default: Round to nearest, ties to even (rtne) Optional: Round to nearest, ties away from 0 (rtna) Round toward zero (inward) Round toward +  (upward) Round toward –  (downward) Rounding Modes

30. 30 Required Reading Parhami, Chapter 17.5, Rounding schemes Rounding Algorithms 101 http://www.diycalculator.com/popup-m-round.shtml

31. 31 Rounding Rounding occurs when we want to approximate a more precise number (i.e. more fractional bits L) with a less precise number (i.e. fewer fractional bits L') Example 1: old: 000110.11010001 (K=6, L=8) new: 000110.11 (K'=6, L'=2) Example 2: old: 000110.11010001 (K=6, L=8) new: 000111. (K'=6, L'=0) The following pages show rounding from L>0 fractional bits to L'=0 bits, but the mathematics hold true for any L' < L Usually, keep the number of integral bits the same K'=K

32. 32 Rounding Equation y = round(x) Fractional partWhole part x k–1 x k–2... x 1 x 0. x –1 x –2... x –l y k–1 y k–2... y 1 y 0 Round

33. 33 Rounding Techniques There are different rounding techniques: 1) truncation results in round towards zero in signed magnitude results in round towards -∞ in two's complement 2) round to nearest number 3) round to nearest even number (or odd number) 4) round towards +∞ Other rounding techniques 5) jamming or von Neumann 6) ROM rounding Each of these techniques will differ in their error depending on representation of numbers i.e. signed magnitude versus two's complement Error = round(x) – x

34. 34 1) Truncation Truncation in signed-magnitude results in a number chop(x) that is always of smaller magnitude than x. This is called round towards zero or inward rounding 011.10 (3.5) 10  011 (3) 10 Error = -0.5 111.10 (-3.5) 10  111 (-3) 10 Error = +0.5 Truncation in two's complement results in a number chop(x) that is always smaller than x. This is called round towards -∞ or downward-directed rounding 011.10 (3.5) 10  011 (3) 10 Error = -0.5 100.01 (-3.5) 10  100 (-4) 10 Error = -0.5 The simplest possible rounding scheme: chopping or truncation x k–1 x k–2... x 1 x 0. x –1 x –2... x –l x k–1 x k–2... x 1 x 0 trunc ulp

35. 35 Truncation Function Graph: chop(x) Fig. 17.5 Truncation or chopping of a signed-magnitude number (same as round toward 0). Fig. 17.6 Truncation or chopping of a 2’s-complement number (same as round to -∞). chop( x ) –4 –3 –2 –1 x –4 –3 –2 –1 4 3 2 1 4 3 2 1 x ) –4 –3 –2 –1 x –4 –3 –2 –1 4 3 2 1 4 3 2 1

36. 36 Bias in two's complement truncation X (binary) X (decimal) chop(x) (binary) chop(x) (decimal) Error (decimal) 011.00301130 011.013.250113-0.25 011.103.50113-0.5 011.113.750113-0.75 100.01-3.75100-4-0.25 100.10-3.5100-4-0.5 100.11-3.25100-4-0.75 101.00-3101-30 Assuming all combinations of positive and negative values of x equally possible, average error is -0.375 In general, average error = -(2 -L' -2 -L )/2, where L' = new number of fractional bits

37. 37 Implementation truncation in hardware Easy, just ignore (i.e. truncate) the fractional digits from L to L'+1 x k-1 x k-2.. x 1 x 0. x -1 x -2.. x -L =y k-1 y k-2.. y 1 y 0. ignore (i.e. truncate the rest)

38. 38 2) Round to nearest number Rounding to nearest number what we normally think of when say round 010.01 (2.25) 10  010 (2) 10 Error = -0.25 010.11 (2.75) 10  011 (3) 10 Error = +0.25 010.00 (2.00) 10  010 (2) 10 Error = +0.00 010.10 (2.5) 10  011 (3) 10 Error = +0.5 [round-half-up (arithmetic rounding)] 010.10 (2.5) 10  010 (2) 10 Error = -0.5 [round-half-down]

39. 39 Round-half-up: dealing with negative numbers Rounding to nearest number what we normally think of when say round 101.11 (-2.25) 10  110 (-2) 10 Error = +0.25 101.01 (-2.75) 10  101 (-3) 10 Error = -0.25 110.00 (-2.00) 10  110 (-2) 10 Error = +0.00 101.10 (-2.5) 10  110 (-2) 10 Error = +0.5 [asymmetric implementation] 101.10 (-2.5) 10  101 (-3) 10 Error = -0.5 [symmetric implementation]

40. 40 Round to Nearest Function Graph: rtn(x) Round-half-up version Asymmetric implementation Symmetric implementation

41. 41 Bias in two's complement round to nearest Round-half-up asymmetric implementation X (binary) X (decimal) rtn(x) (binary) rtn(x) (decimal) Error (decimal) 010.00201020 010.012.250102-0.25 010.102.50113+0.5 010.112.750113+0.25 101.01-2.75101-3-0.25 101.10-2.5110-2+0.5 101.11-2.25110-2+0.25 110.00-2110-20 Assuming all combinations of positive and negative values of x equally possible, average error is +0.125 Smaller average error than truncation, but still not symmetric error We have a problem with the midway value, i.e. exactly at 2.5 or -2.5 leads to positive error bias always Also have the problem that you can get overflow if only allocate K' = K integral bits Example: rtn(011.10)  overflow This overflow only occurs on positive numbers near the maximum positive value, not on negative numbers

42. 42 Implementing round to nearest (rtn) in hardware Round-half-up asymmetric implementation Two methods Method 1: Add '1' in position one digit right of new LSB (i.e. digit L'+1) and keep only L' fractional bits x k-1 x k-2.. x 1 x 0. x -1 x -2.. x -L + 1 =y k-1 y k-2.. y 1 y 0. y -1 Method 2: Add the value of the digit one position to right of new LSB (i.e. digit L'+1) into the new LSB digit (i.e. digit L) and keep only L' fractional bits x k-1 x k-2.. x 1 x 0. x -1 x -2.. x -L + x -1 y k-1 y k-2.. y 1 y 0. ignore (i.e. truncate the rest) ignore (i.e truncate the rest)

43. 43 Round to Nearest Even Function Graph: rtne(x) To solve the problem with the midway value we implement round to nearest-even number (or can round to nearest odd number) Fig. 17.8 Rounding to the nearest even number. Fig. 17.9 R* rounding or rounding to the nearest odd number.

44. 44 Bias in two's complement round to nearest even (rtne) average error is now 0 (ignoring the overflow) cost: more hardware X (binary) X (decimal ) rtne(x) (binary) rtne(x) (decimal) Error (decimal) 000.100.50000-0.5 001.101.50102+0.5 010.102.50102-0.5 011.103.50100 (overfl)4+0.5 100.10-3.5100-4-0.5 101.10-2.5010-2+0.5 110.10-1.5010-2-0.5 111.10-0.50000+0.5

45. 45 4) Rounding towards infinity We may need computation errors to be in a known direction Example: in computing upper bounds, larger results are acceptable, but results that are smaller than correct values could invalidate upper bound Use upward-directed rounding (round toward +∞) up(x) always larger than or equal to x Similarly for lower bounds, use downward-directed rounding (round toward -∞) down(x) always smaller than or equal to x We have already seen that round toward -∞ in two's complement can be implemented by truncation

46. 46 Rounding Toward Infinity Function Graph: up(x) and down(x) up(x)down(x) down(x) can be implemented by chop(x) in two's complement

47. 47 Two's Complement Round to Zero Two's complement round to zero (inward rounding) also exists inward( x ) –4 –3 –2 –1 x –4 –3 –2 –1 4 3 2 1 4 3 2 1

48. 48 Other Methods Note that in two's complement round to nearest (rtn) involves an addition which may have a carry propagation from LSB to MSB Rounding may take as long as an adder takes Can break the adder chain using the following two techniques: Jamming or von Neumann ROM-based

49. 49 5) Jamming or von Neumann Chop and force the LSB of the result to 1 Simplicity of chopping, with the near-symmetry or ordinary rounding Max error is comparable to chopping (double that of rounding)

50. 50 6) ROM Rounding Fig. 17.11 ROM rounding with an 8  2 table. Example: Rounding with a 32  4 table Rounding result is the same as that of the round to nearest scheme in 31 of the 32 possible cases, but a larger error is introduced when x 3 = x 2 = x 1 = x 0 = x –1 = 1 x k–1... x 4 x 3 x 2 x 1 x 0. x –1 x –2... x –l x k–1... x 4 y 3 y 2 y 1 y 0 ROM ROM dataROM address

51. Representation of the Galois Field elements

52. Evariste Galois (1811-1832)

53. Studied the problem of finding algebraic solutions for the general equations of the degree  5, e.g., f(x) = a 5 x 5 + a 4 x 4 + a 3 x 3 + a 2 x 2 + a 1 x+ a 0 = 0 Answered definitely the question which specific equations of a given degree have algebraic solutions. On the way, he developed group theory, one of the most important branches of modern mathematics.

54. Evariste Galois (1811-1832) 1829 Galois submits his results for the first time to the French Academy of Sciences Reviewer 1 Augustin-Luis Cauchy forgot or lost the communication. 1830Galois submits the revised version of his manuscript, hoping to enter the competition for the Grand Prize in mathematics Reviewer 2 Joseph Fourier – died shortly after receiving the manuscript. 1831Third submission to the French Academy of Sciences Reviewer 3 Simeon-Denis Poisson – did not understand the manuscript and rejected it.

55. Evariste Galois (1811-1832) May 1832 Galois provoked into a duel The night before the duel he wrote a letter to his friend containing the summary of his discoveries. The letter ended with a plea: “Eventually there will be, I hope, some people who will find it profitable to decipher this mess.” May 30, 1832 Galois was grievously wounded in the duel and died in the hospital the following day. 1843Galois manuscript rediscovered by Joseph Liouville 1846 Galois manuscript published for the first time in a mathematical journal.

57. Field Set F, and two operations typically denoted by (but not necessarily equivalent to) + and * Set F, and definitions of these two operations must fulfill special conditions.

58. { set Zp={0, 1, 2, …, p-1}, + (mod p): addition modulo p, * (mod p): multiplication modulo p } Examples of fields Infinite fields Finite fields { R= set of real numbers, + addition of real numbers * multiplication of real numbers }

59. Finite Fields = Galois Fields GF(p) GF(2 m ) Polynomial basis representation Normal basis representation Fast in hardware Arithmetic operations present in many libraries Fast squaring GF(p m ) p – prime p m – number of elements in the field Most significant special cases

60. Quotient and remainder Given integers a and n, n>0  ! q, r  Z such that a = q  n + r and 0  r < n q – quotient r – remainder (of a divided by n) q = a n = a div n r = a - q  n = a – a n  n = = a mod n

61. 32 mod 5 = -32 mod 5 =

62. Integers coungruent modulo n Two integers a and b are congruent modulo n (equivalent modulo n) written a  b iff a mod n = b mod n or a = b + kn, k  Z or n | a - b

63. Laws of modular arithmetic

64. Rules of addition, subtraction and multiplication modulo n a + b mod n = ((a mod n) + (b mod n)) mod n a - b mod n = ((a mod n) - (b mod n)) mod n a  b mod n = ((a mod n)  (b mod n)) mod n

65. 9 · 13 mod 5 = 25 · 25 mod 26 =

66. Laws of modular arithmetic Modular addition Modular multiplication Regular addition Regular multiplication a+b = a+c iff b=c a+b  a+c (mod n) iff b  c (mod n) If a  b = a  c and a  0 then b = c If a  b  a  c (mod n) and gcd (a, n) = 1 then b  c (mod n)

67. Modular Multiplication: Example 18  42 (mod 8) 6  3  6  7 (mod 8) 3  7 (mod 8) x 6  x mod 8 0123456701234567 0642064206420642 x 5  x mod 8 0123456701234567 0527416305274163

68. Finite Fields = Galois Fields GF(p) GF(2 m ) Polynomial basis representation Normal basis representation Fast in hardware Arithmetic operations present in many libraries Fast squaring GF(p m ) p – prime p m – number of elements in the field Most significant special cases

69. Elements of the Galois Field GF(2 m ) Binary representation (used for storing and processing in computer systems): Polynomial representation (used for the definition of basic arithmetic operations): A = (a m-1, a m-2, …, a 2, a 1, a 0 ) a i  {0, 1} A(x) =  a i  x i = a m-1  x m-1 + a m-2  x m-2 + …+ a 2  x 2 + a 1  x+a 0  multiplication + addition modulo 2 (XOR) i=0 m-1

70. Addition and Multiplication in the Galois Field GF(2 m ) Inputs A = (a m-1, a m-2, …, a 2, a 1, a 0 ) B = (b m-1, b m-2, …, b 2, b 1, b 0 ) a i, b i  {0, 1} Output C = (c m-1, c m-2, …, c 2, c 1, c 0 ) c i  {0, 1}

71. Addition A  A(x) B  B(x) C  C(x) = A(x) + B(x) = = (a m-1 +b m-1 )  x m-1 + (a m-2 +b m-2 )  x m-2 + …+ + (a 2 +b 2 )  x 2 + (a 1 +b 1 )  x + (a 0 +b 0 ) = = c m-1  x m-1 + c m-2  x m-2 + …+ c 2  x 2 + c 1  x+c 0 Addition in the Galois Field GF(2 m )  multiplication + addition modulo 2 (XOR) c i = a i + b i = a i XOR b i C = A XOR B

72. Multiplication A  A(x) B  B(x) C  C(x) = A(x)  B(x) mod P(X) = c m-1  x m-1 + c m-2  x m-2 + …+ c 2  x 2 + c 1  x+c 0 Multiplication in the Galois Field GF(2 m ) P(x) - irreducible polynomial of the degree m P(x) = p m  x m + p m-1  x m-1 + …+ p 2  x 2 + p 1  x+p 0