Lecture 10: Knapsack Problems and Public Key Crypto Wayne Patterson SYCS 654 Spring 2010

The Classical Knapsack Problem Easily enough stated, this problem is one that turns out to be extremely difficult. First, in English: I have a knapsack, and I know it (or I) can carry W pounds. I have a bunch of things I would like to take on a trip that weigh w 1, w 2, w 3, …, w n pounds. The problem: Is there a subset of the {w 2, w 3, …, w n } that will add exactly to W, in other words, that will allow me to carry the maximum possible weight.

Knapsack = Subset Sum Sometimes this problem is also called the “subset sum” problem. Sometimes we are lucky and can find a very quick solution to the problem. For example, with knapsack weight W, and objects that weigh { 1, 2, 4, 8, 16, 32, …, 2 n }, we can answer the question very easily.

The “Easy” Knapsack Sets For the example given previously, of weights { 1, 2, 4, 8, 16, 32, …, 2 n }, the solution to the knapsack problem is unique. For every 0  W  2 n+1 – 1, there is a unique solution, and for W  2 n+1, there is no solution.

Easy Knapsacks Proof: (Binary string argument). For W  2 n+1 – 1, W has a binary representation with n+1 bits. E.g., if n+1 = 4, 2 n+1 – 1 = 15, and W is represented as 1111 (binary). For arbitrary W  2 n+1 – 1, represent W as a binary --- then all the weights corresponding to a 1-bit position can exactly fit into the knapsack.

Example Suppose the knapsack set is {1, 2, 4, 8, 16, 32, 64, 128} and W = 173. (W can go up to 255.) Express W in binary: 173 = Then the weights corresponding to the 1- bits will add to W: = 173

Super-increasing Knapsack Sets Of course, in the preceding example, there will NOT be a solution to the knapsack problem if W > 255. There is a more general class of “easy” knapsack problems, and basically the same algorithm will apply. We will call this class of problems the “super-increasing knapsack sets”.

Super-increasing … Suppose now we have a set of weights with the property that each weight is greater than the sum of the weights of all of its predecessors in order: w 2 > w 1 w 3 > w 1 + w 2 w 4 > w 1 + w 2 + w 3 and so on …

Solving the Super-increasing knapsack problem Let’s take as an example a set of weights: ◦ { 3, 7, 19, 35, 72, 155, 367, 984 } And suppose W = The algorithm for solution is: set x = W, process the weights in descending order, if the weight is less than or equal the current value of x, subtract it and remember the weight. After you have processed all the weights, if you have a remainder of 0, you have a solution. If the remainder is not zero, there is no solution.

The Computation x = 1230 (984 < x, subtract it) 984 x = 246 (367 > x, don’t subtract) (155 < x, subtract it) 155 x = 91 (72 < x, subtract it) 72 x = 19 (35 > x, don’t subtract) (19  x, subtract it) 19 x = 0, done. So the solution is: { 984, 155, 72, 19 }

General Knapsacks So we’ve looked at the easy cases, where there is a fast algorithm to determine a solution. Unfortunately, MOST knapsack sets are not nearly so nice. Consider: { 347, 356, 387, 401, 422, 461, 479, 521 } and W = 1635.

Brute Force Now for this small a knapsack set (with only 8 weights), we can solve the problem by brute force. This means one sum calculation for every subset of the knapsack set. Since a set with cardinality n has 2 n subsets, we can solve this with 2 n = 256 tries. But if the knapsack had 200 items, our brute force approach would require an estimated 803,469,022,129,495,137,770,981,046,170,58 1,301,261,101,496,891,396,417,650,688 tries.

I’m Still Working on it … Unfortunately, despite the centuries that people have thought about this problem, no better solution has been found than brute force. If you have studied complexity theory, you would know that the knapsack problem falls into the category of the most intractable problems, the category called NP-Complete.

What’s That Got to Do with PKC? Shortly after Diffie and Hellman (1976) described the concept of Public-Key Crypto with a public and private key, Merkle and Hellman proposed the use of the knapsack problem to create a Public Key Cryptosystem.

The Merkle-Hellman Knapsack PKC First, for my private key, I will define a super- increasing knapsack set. To make it interesting, the knapsack set will have n numbers, n = 100. To make sure the numbers are large enough not to be guessed, define w 1 to be chosen at random in the interval [2 100, ]; then each successive w i will be in the interval [2 100+i-1, i -1]; in this way we guarantee that the knapsack set will have the super-increasing property.

More Private Key then Public So now we have our “easy” set {w 1, …, w 100 }, and next we find a prime number p > (thus larger than the sum of all the w i ’s, and choose at random some m < p, and also compute m -1 (mod p). Now create a “hard” knapsack set {w 1 *, …, w 100 * } by computing w i * = m * w i (mod p). The public key is the “hard” knapsack set {w 1 *, …, w 100 * }

Encryption and Decryption As we well know, every user creates his or her public key and publishes it. So to send a message of length 100 bits to a user, find his or her public knapsack, and add up the numbers corresponding to the 1-bits in the message. I.e., if the message is m = b 1 …b 100, (b for bits), the encryption is: b 1 × w 1 * + b 2 × w 2 * + … + b 100 × w 100 * = c (which is just a sum of some subset …) now send c.

Decryption When I receive c, I multiply it by m -1 and reduce mod p. This gives: m -1 × (b 1 × w 1 * + b 2 × w 2 * + … + b 100 × w 100 *) = b 1 × m -1 × w 1 * + b 2 × m -1 × w 2 * + … + b 100 × m -1 × w 100 * = b 1 × w 1 + b 2 × w 2 + … + b 100 × w 100 Which is now a knapsack problem in our easy set, so solve it to get the values of the b i and therefore the message

Example Easy = { 1, 3, 7, 13, 26, 65, 119, 267} The complete sum is 501, choose p = 523 and m = 467. Then m -1 = 28. The hard knapsack set, or public key, will be 1 × 467 (mod 523), 3 × 467 (mod 523), etc. or: Hard = Public = {467, 355, 131, 318, 113, 21, 135, 215}

Encrypt the Bitstring The encryption is: c = 0 × × × × × × × × 215 = = 818 To decrypt, multiply c × m -1 (mod p) = 818 × 28= 415 (mod p).

If That was the end of the story … But unfortunately it isn’t. Within a few years, it was discovered that Merkle Hellman knapsack systems were eminently breakable. And not only the Merkle Hellman systems, but any knapsack approach that depended on numbers in the knapsack set growing very fast. So the crypto community fell out of love with knapsacks.

But there was one knapsack approach left standing … Let’s just remember good old Blaise Pascal and his triangle …

Excursions in Computation Wayne Patterson Professor of Computer Science Howard University SYCS Colloquium Series, March 26,

25

26 Pascal PK Crypto Goldbach ??????

The author is reminded of the old expression: “Something old, something new; something borrowed, something blue.” Although reluctant to suggest a presentation anything like a wedding ceremony, he will look anew at some old computational concepts involving the Pascal triangle; something new (to many) in a related application revisiting a public key crypto chestnut; borrowing some ideas from what is now usually described as “experimental mathematics”. Something blue? You’ll have to wait and see. 27

You will recall that each row in the Pascal triangle is the sequence of coefficients in the expansion of Starting with the 0 th row, (x+y) 0 = 1 And the kth element in the nth row being 29

Often the best mathematical insights come from an ability to visualize the same phenomenon from multiple perspectives. To illustrate this point, I am going to describe an example wherein the same underlying principle will have three separate expressions: one in a geometric representation, one in a combinatorial representation, and one in a binary string representation. 30

31

Consider a mouse that finds itself at the cornerstone of the parallelogram. The mouse, whose name is “One”, wishes to escape to freedom by emerging from the top. When the mouse moves up and to the right, the number “bypassed” is added to the mouse’s value (starting at One!). If the mouse moves up and to the left, nothing is added. 33

The sequence of moves: Lets the mouse escape with a value of = 70.

The sequence of moves: Lets the mouse escape with a value of = 31.

could be written more compactly by representing an “up to the right” by a “1” and “up to the left” by “0”. The result of this is a bitstring, and so the figure on the left becomes: Since each mouse move goes up by one row, all successful paths are of length 8 And to go out the top, the mouse must make an equal number of “up rights” and “up lefts” So our bitstring will be always of length 8 with 4 1-bits. 36

Clearly there is a 1-1 correspondence between paths that escape through the top and bitstrings of length 8 with 4 1- bits. How many paths? How many such bitstrings? Each such bitstring results from picking 4 positions out of 8 But this is the definition of 37

PBJ Let P = set of all paths through the parallelogram Let B = set of all bitstrings of length 2n with n 1-bits Let J = subset of N, natural numbers, = 38

39  = Use the bits to tell the mouse where to go  = Track the mouse move ments with bits  = Add the path value s      We just need , since then the last piece will be   

Track back through the parallelogram.  (52) = = 52

41

Knapsacks are dead for public key crypto, or mostly dead … As Billy Crystal said in the “Princess Bride”: “mostly dead is partly alive” … All the knapsacks previously studied were “low-density” Methods of Brickell, Lagarias and Odlyzko depended on this density So here’s a knapsack modelled on the Pascal parallelogram That can’t be attacked by the low- density methods 42

For the first row, choose a number pseudorandomly in the interval [1,2 200 ]. Second row: Each element pseudorandomly between [ ,2 201 ] For each succeeding row (i), let the kth element be chosen pseudorandomly in the interval Create 200 rows As with traditional knapsacks, find a large prime p and a multiplier m, and multiply each element in the parallelogram by m mod p. 43

The public key is the transformed parallelogram The private key is the original parallelogram, as well as m. 44

45

In recent years, a number of mathematicians have worked to develop new ways of thinking about their subject. These approaches, often described as "experimental mathematics," were simply not available to earlier generations of mathematicians, because they depend upon the ability to analyze the results of computations made feasible by appropriate mathematical software tools in order to formulate previously unthinkable hypotheses. 46

Number Theory ◦ Purest branch of mathematics ◦ Open problems can be explained to a non-mathematician ◦ Among the most difficult to solve As Jim Arthur has said: ◦ “Andrew Wiles’s proof of Fermat’s Last Theorem, in a way that we would not have expected, caught people’s imagination. Books like the one on John Nash, A Beautiful Mind, have also brought a good deal of attention to mathematics. And of course in movies, mathematics has been chic in the last five or ten years.” 47

We will look at two of the classical computational number theoretic problems: Goldbach conjecture n 2 +1-prime conjecture 48

One of the greatest remaining conjectures in elementary number theory is the Goldbach conjecture, which in its most often quoted form is: “Every even positive integer≥4 is the sum of two prime numbers” 49

“Every prime number > 11 is the sum of two composite numbers.” I have been able to prove the Goldfinger Conjecture! 50

Given any prime number p > 11, Either p  1 (mod 3) or p  2 (mod 3). If p  1 (mod 3), then p = 3k + 1 = 3(k-1) + 4 = 3*(k-1) + 2*2 If p  2 (mod 3), then p = 3k + 2 = 3(k-2) + 8 = 3*(k-2) + 2*4. Q. E. D. 51

(a) Patterson will be heading to Norway for the next Abel Prize ($980,000) or (b) The result will be formally announced on (European convention) 52

There was briefly a $1M prize for solving the Goldbach Conjecture Needless to say, it wasn’t claimed, and it’s now closing in on 200 years without solution Among the people who have recently looked at the Goldbach Conjecture is John Nash Also my Op-Ed in the Washington Post at the time of the release of “A Beautiful Mind”, regarding my interactions with him in my Princeton days, and other musings. 53

My interest has been to try to determine the difficulty of finding primes that add to a given, pseudo-randomly selected even number of varying magnitudes. For a given n = 4k+1, where k is odd, the first approach to the question involves testing the numbers 2k-1 and 2k+1 for primality. 54

maxbyexp=Table[0,{m,1,37},{n,1,2}]; Do[top=10^l; resul=Table[0,{i,1,500},{j,1,4}]; Do[od = 2*Random[Integer,{1,top/2}]+1; ev = 2*od; p = ev/2 +2;q=ev/2-2;i=1; While[(!(PrimeQ[p]&&PrimeQ[q])&&(p>0)),p=p-2;q=q+2; i=i+1]; resul[[k]]={i,ev,p,q},{k,1,500}];MatrixForm[resul];tr = Transpose[resul]; Print["For exponent ",l,", the largest i is ",Max[tr[[1]]]]; maxbyexp[[l-3]]={l,Max[tr[[1]]]},{l,4,40}]; Print[MatrixForm[maxbyexp]]; ListPlot[maxbyexp] 55

56

57

Maximum number of tests is 457 for a number of magnitude Vs. 41,177 (for 2k-1,2k+1) and 46,317 for random odd numbers There is also a result that the maximum number of tests up to is

59

Given the seeming efficiency of finding Goldbach pairs using lists of small primes, I wondered whether, given a selected interval of consecutive even numbers, the complete set of members in that interval could be covered by Goldbach pairs using the small primes And, if so, how large an interval, at what starting point, and using how large a list of small primes … 60

61

62

63

64

This number theoretic conjecture asserts that there are an infinite number of primes of the form n = = = = = = 197… 65

It might be noted that the first case where n 2 +1 is not prime for n even is = 65, and that in general, n will never be prime if n=2 (mod 10) or n=8 (mod 10), for n=8, since the last digit of n 2 will be 4, and the last digit of n 2 +1 will be 5. Thus we can limit ourselves to considerations for n, for n 2 +1 to be prime, to be n=0, 4, or 6 (mod 10). 66

As before, we selected numbers at random of varying magnitudes up to , and tested the values of n 2 +1 for primality 67

68

Numbers of the form n 4 +1 form a proper subset of those of the form n 2 +1 Since numbers of the form n 4 +1 are more spread out along the number line than those of the form n 2 +1, it would be reasonable to expect that it would be harder to find primes of the form n

1: Number of tries to find an n prime 2: Number of tries to find an n prime 3: Number of tries to find an n prime (abc): Least number of tries is a, second least b, most c. 70

Order /Test (123)(132)(213)(231)(312)(321) Total Cases

To delve further into this, I thought selecting a specific sequence of n’s would be interesting in trying to find a sequence of primes (or composites). I was led to the sequence ◦ {10 k + 1 | k= 1, 2, …} ◦ ……

Observation 1: 10 k +1 is prime if k=1 or 2. Observation 2: 10 k +1 is divisible by 11 (and therefore composite) if k is positive and odd. (Use the old trick of computing the sum of the digits in the odd and even positions; if their difference is divisible by 11, so is the number.) Observation 3: 10 k +1 is composite (k>1) if k is not a power of 2. E.g. ( ) = ( )(10 12 – – – ) 73

In studying three well-known number- theoretic outstanding conjectures, we are able to discover some unexpected phenomena, and thus shed new light on these classical problems. Furthermore, these investigations are accessible to undergraduate mathematics students. 74

75

This is Larry Bowa 76 He is the third-base coach for the Los Angeles Dodgers He wears blue He has a cryptosystem

An Important Cryptanalysis Financial share for World Series Winners (2008) = $18,417,358. Bowa communicates one of 9 signals to a runner or a batter: Plaintext = { Steal, Hold, StealOnOverflow, Take, Bunt, Swing, HitAndRun, RunAndHit, SqueezeBunt } 77

Base Coach Signals Signals have two components, typically: A number of body movements, BODY = {Belt, Clap, Hat, Leg, Nose, Shoulder, Wipe } And a “hot sign” ε [1, 10] So the key space is KEY = { (x, y) | x ε [1,10], y ε BODY } 78

Frequency of Occurrences in a Game 79 Situat ion/P lay StealHoldSteal OnOv erthr ow TakeBuntSwin g HitAn dRun RunA ndHit Sque eze 1 st, 2 nd, 1 st and 2nd rd, < 2 out , 3-0, 3-1 count s 64

Cryptanalysis Map succeeding signals in situation: runner on 1 st, 39% of time, or 116 times/game. Example: Shoul der BeltWipeBeltClapNoseBeltClapHatNose BeltWipeBeltClapShoul der HatBeltNoseWipeLeg

With One “Fixed Point”… Can exactly determine key with two readings. Number of messages = | KEY | = 7 10 = (6+1) 10 = x x x … + 10 x # of messages with 1 fixed point = 10 x 6 9 = 100,776,960 # of messages with ≥ 1 fixed point = 7 10 x 6 10 = 181,698,289 81

Probabilities Probability of exactly 1 fixed point = Probability of more than 1 fixed point = Probability of exactly 1 fixed point in 3 pitches = 1 – (0.445) 3 = Probability of exactly 1 fixed point in 4 pitches = 1 – (0.445) 6 =

References Agrawal, M., N. Kayal and N. Saxena, PRIMES in P, (August 2002), or Bailey, David H. and Jonathan M. Borwein, Experimental Mathematics: Examples, Methods, and Implications, Notices of the American Mathematics Society, vol. 52, no. 5, May 2005, pp Brickell, E. F., "Solving low density knapsacks," Advances in Cryptology-Proc. Crypto 83, Plenum Press, New York, 1984, pp Chen, J.-R. and T.-Z. Wang, On the Goldbach Problem, Acta Math. Sinica 32, 1918, pp Cooper, Rodney H., Hunter-Duvar, Ron, and Patterson, Wayne, A More Efficient Public-Key Cryptosystem Using the Pascal Triangle, ICC `89, Boston, June 1989, pp Klamkin, M. S., Problem 63—12, SIAM Review, 5 (1963) Lagarias, J. C. and A. M. Odlyzko, "Solving Low Density Subset Sum Problems," J. Assoc. Comp. Mach., vol. 32, 1985, pp Proc. 24th IEEE Symposium on Foundations of Computer Science, IEEE, 1983, pp Patterson, Wayne, An Exploration in ‘Experimental Mathematics’: Computing the Determinant Function, Proceedings of the Mid- Atlantic Consortium for Research in Mathematical Sciences, Ocean City, MD, 2004, to appear. Patterson, Wayne, “Experimentation in Computational Number Theory,” Proceedings of the Mid-Atlantic Consortium for Research in Mathematical Sciences, Orlando, FL, June Patterson, Wayne, Mathematical Cryptology, Rowman and Littlefield, 318 pp., Presidential Views: Interview with James Arthur, Notices of the American Mathematics Society, vol. 52, no. 3, March 2005, pp Richstein, J., Verifying the Goldbach Conjecture up to 4 ▪ 10 14, Math. Comp. 70:236 (2001)