The Busy Beaver, the Placid Platypus and the Universal Unicorn James Harland james.harland@rmit.edu.au Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Busy beaver problem and similar problems Machine execution techniques Overview Busy beaver problem and similar problems Machine execution techniques Searching large numbers of machines Universal Turing machines Connections with other areas Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Busy beaver problem and similar problems Machine execution techniques Overview Busy beaver problem and similar problems Machine execution techniques Searching large numbers of machines Universal Turing machines Connections with other areas Beaver, Platypus, Unicorn … Monash June 2nd, 2014

‘Organic’ theoretical computer science Overview ‘Organic’ theoretical computer science Exploration of theoretical concepts by software tools Mix of empirical and theoretical methods Beaver, Platypus, Unicorn … Monash June 2nd, 2014

a b c h What? Turing Machines of a particular type: Deterministic Symbols are only 0 (blank) and 1 (initially) Only consider blank input n states plus a halt state means size is n a b c h 11R 01R 00L 10R 01L Beaver, Platypus, Unicorn … Monash June 2nd, 2014

What? What is the largest number of 1’s that can be printed by a terminating n-state machine? n #1’s (productivity) #steps 1 2 4 6 3 21 13 107 5 ≥ 4098 ≥ 47,176,870 (??) 6 ≥ 3.51×1018,276 ≥ 7.41×1036,534 (!!) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

HERE BE DRAGONS! What? n m #1’s #steps 2 4 6 3 21 9 38 13 107 2,050 3,932,964 374,676,383 119,112,334,170,342,540 5 ≥ 4098 ≥ 47,176,870 ≥ 1.7 × 10352 ≥ 1.9 × 10704 3.51×1018276 ≥ 7.41×1036534 ≥ 1.383 × 107036 ≥ 1.025 × 1014072 ≥ 3.743 × 106518 ≥ 5.254 × 1013036 ≥ 1.903 × 104933 ≥ 2.415 × 109866 HERE BE DRAGONS! Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Various mathematical bounds known All surpassed in practice What? Busy Beaver function is non-computable; it grows faster than any computable function (!!) Various mathematical bounds known All surpassed in practice Seems hopeless for n ≥ 7 Values for n ≤ 5 seem settled (but as yet unproven) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Some enormous numbers here! `… the poetry of logical ideas’ (Einstein) Why? Some enormous numbers here! `… the poetry of logical ideas’ (Einstein) Evidence for results is somewhat lacking (“The remaining 250+ cases were checked by hand”) Should have test data and evidence available Should be able to automate entire test `Nothing capable of proof should be accepted as true without it’ (Frege) `Nothing capable of automation should be accepted as finished without it’ Beaver, Platypus, Unicorn … Monash June 2nd, 2014

An n-state (2-symbol) machine of productivity k shows that bb(n,2) ≥ k Why? An n-state (2-symbol) machine of productivity k shows that bb(n,2) ≥ k at most n states are needed to print k 1’s What is the minimum number of states (for an n- state 2-symbol machine) needed to print k 1’s? We call this the placid platypus problem [ pp(k) ] Beaver, Platypus, Unicorn … Monash June 2nd, 2014

bb(n,2) = k means that pp(k) ≤ n and pp(k+1) ≥ n+1 Why? bb(n,2) = k means that pp(k) ≤ n and pp(k+1) ≥ n+1 pp(2) = pp(3) = pp(4) = 2 pp(5) = pp(6) = 3 pp(k) = 4 for k є {7,8,9,10,11,12,13} pp(k) ≥ 5 for k ≥ 14 …??? There seem to be no 5-state 2-symbol machines with productivity 74, 85, 92-7, 100-110, 113, 114, 115, … Need complete classification to be sure Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Marxen and Buntrock (1990): macro machines Executing Machines Marxen and Buntrock (1990): macro machines Treat a sequence of k characters as a single ‘symbol’ Avoids repetition of previous sequences Significant speed up Need to choose k in advance Holkner (2004): loop prediction {b}(01)1001 -> (11)100 {b}1 State Input InDir Output OutDir NewState b 01 l 11 r Beaver, Platypus, Unicorn … Monash June 2nd, 2014

20,000+ steps in 1 Executing Machines What do the monsters do? 5x2 state champion (terminates in 47,176,870 steps) Step Configuration 12393 001111{b}(001)66100 12480 001(111)6{b}(001)63100 12657 001(111)11{b}(001)60100 … 33120 001(111)106{b}(001)3100 47176870 101{h}(001)4095100 20,000+ steps in 1 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

5x2 state champion takes 11 otter steps Executing Machines 5x2 state champion takes 11 otter steps Otter Steps Predicted 1 267 2 2,235 3 6,840 4 20,463 5 62,895 6 176,040 7 500,271 8 1,387,739 9 3,878,739 10 10,830,672 11 30,144,672 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Executing Machines Macro machines + loop prediction + observant otter can cope with all known monsters (!!) Size Ones Hops Otters Otter Steps 2x4 90 7195 2 64% 2050 3932964 8 99% 3x3 17338 127529109 17 > 99% 95524079 (16 digits) 374676383 (18 digits) 60 5x2 4098 47,176,870 11 11,798,796 10 4097 23,554,764 74 2x5 (31 digits) (62 digits) 168 (106 digits) (212 digits) 593 (353 digits) (705 digits) 1993 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Executing Machines Size Ones Hops Otters Otter Steps 6x2 (18268 digits) (36535 digits) 30345 > 99% (10567 digits) (21133 digits) 99697 (1440 digits) (2880 digits) 8167 (882 digits) (1763 digits) 2211 4x3 (7037 digits) (14073 digits) 25255 (6035 digits) (12069 digits) 34262 (4932 digits) (9864 digits) 57368 3x4 (6519 digits) (13037 digits) 13667 (2373 digits) (4745 digits) 13465 (2356 digits) (4711 digits) 107045 2x6 (4934 digits) (9867 digits) 16394 49142 (822 digits) (1644 digits) 2733 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

HERE BE DRAGONS! Executing Machines Size Ones Hops Otters Otter Steps (18268 digits) (36535 digits) 30345 > 99% (10567 digits) (21133 digits) 99697 (1440 digits) (2880 digits) 8167 (882 digits) (1763 digits) 2211 4x3 (7037 digits) (14073 digits) 25255 (6035 digits) (12069 digits) 34262 (4932 digits) (9864 digits) 57368 3x4 (6519 digits) (13037 digits) 13667 (2373 digits) (4745 digits) 13465 (2356 digits) (4711 digits) 107045 2x6 (4934 digits) (9867 digits) 16394 49142 (822 digits) (1644 digits) 2733 HERE BE DRAGONS! Beaver, Platypus, Unicorn … Monash June 2nd, 2014

HERE BE HOBBITS! Executing Machines Size Ones Hops Otters Otter Steps (18268 digits) (36535 digits) 30345 > 99% (10567 digits) (21133 digits) 99697 (1440 digits) (2880 digits) 8167 (882 digits) (1763 digits) 2211 4x3 (7037 digits) (14073 digits) 25255 (6035 digits) (12069 digits) 34262 (4932 digits) (9864 digits) 57368 3x4 (6519 digits) (13037 digits) 13667 (2373 digits) (4745 digits) 13465 (2356 digits) (4711 digits) 107045 2x6 (4934 digits) (9867 digits) 16394 49142 (822 digits) (1644 digits) 2733 HERE BE HOBBITS! Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Aberrent Albatrosses The observant otter is not a panacea … Some machines do not produce “neat, compressible” patterns … 3x3 case terminates in 2,315,619 steps with 31 non-zeroes 2x5 case terminates in 26,375,397,569,930 steps with 143 non-zeroes 1004: 012010150130170{b}134334 5000: (011)2101(111)311{a}0433135441100 10001: 01401013010160{a}331374300 ??? Beaver, Platypus, Unicorn, … Monash June 2nd, 2014

Some non-termination cases are easy Analysing Machines Some non-termination cases are easy perennial pigeon: repeats a particular configuration phlegmatic phoenix: tape returns to blank road runner: “straight to the end of the tape” meandering meerkat: terminal transition cannot be reached Hence we check for these first Beaver, Platypus, Unicorn … Monash June 2nd, 2014

How? Otherwise .. (ie all the easy cases have been checked): Run the machine for a given number of steps If it halts, done. Check for non-terminating pattern (Verify non-terminating pattern) Execute machine for a larger number of steps Eventually give up! Beaver, Platypus, Unicorn, … Monash June 2nd, 2014

Observant Otter Some surprises can be in store .... 16{D}0 → 118{D}0 → 142{D}0 (!!!) → 190{D}0 130{D}0 does not occur … So always look for three instances of a pattern Conjecture in this case is 1N{D}0 → 12N+6{D}0 or alternatively 1N{D}0 → (11)N111111{D}0 Clearly there is exponential growth here … Beaver, Platypus, Unicorn, … Monash June 2nd, 2014

Aberrent Albatrosses These occur in non-termination cases as well … 36: {a}0333 54: {a}030323 74: {a}032333 88: {a}033333 110: {a}03030323 136: {a}03232333 150: {a}03332333 168: {a}03303333 204: {a}0332330323 ?? Beaver, Platypus, Unicorn, … Monash June 2nd, 2014

Aberrent Albatrosses These occur in non-termination cases as well … 36: {a}0333 54: {a}030323 74: {a}032333 88: {a}033333 110: {a}03030323 136: {a}03232333 150: {a}03332333 168: {a}03303333 204: {a}0332330323 Beaver, Platypus, Unicorn, … Monash June 2nd, 2014

Searching for Machines To find bb(n,m): Generate all n-state m-symbol machines Classify them as terminating or non-terminating Find the terminating one which produces the largest number of 1’s (or non-zero characters) Only problem is that there are lots of traps … Beaver, Platypus, Unicorn … Monash June 2nd, 2014

HERE BE DRAGONS! Executing Machines Size Ones Hops Otters Otter Steps (18268 digits) (36535 digits) 30345 > 99% (10567 digits) (21133 digits) 99697 (1440 digits) (2880 digits) 8167 (882 digits) (1763 digits) 2211 4x3 (7037 digits) (14073 digits) 25255 (6035 digits) (12069 digits) 34262 (4932 digits) (9864 digits) 57368 3x4 (6519 digits) (13037 digits) 13667 (2373 digits) (4745 digits) 13465 (2356 digits) (4711 digits) 107045 2x6 (4934 digits) (9867 digits) 16394 49142 (822 digits) (1644 digits) 2733 HERE BE DRAGONS! Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Searching for Machines There is a finite but very large set of machines for each size … #machines of size n is O(nn) [ > O(n!) > O(2n) !!] ‘Free’ generation of machines quickly becomes cumbersome .. ‘Free’ generation creates too many trivial machines a b c h 11R 01R 00L, 10R 10R Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Searching through Machines First transition is fixed Run partial machine Add a new transition when a ‘missing’ case happens Add ‘halt’ transition when appropriate Only one transition left At least n different states for an n-state machine a b c h 11L 01R 10R 00L {a}0 -> 1{b}0 -> {a}1 -> {a}01 -> 1{b}1 -> 10{c}0 Generates machines in tree normal form (tnf) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Searching through Machines tnf free 2 36 64 3 3,522 55,296 2,765 41,472 4 507,107 ?? 501,232 25,955,301 5 99,174,375 ≥ 115,000,000 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Around 5,000 lines of SWI Prolog Versions available at my website Implementation Around 5,000 lines of SWI Prolog Versions available at my website Data available as well Documentation is a little lacking … Intention is that other researchers can use data and verify results (this is empirical science!) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Blue Bilby n-state m-symbol machines, n x m ≤ 6 Total Terminated Going Unclassified 2 36 14 22 3 3,522 1,188 2,334 2,765 839 1,926 The free and arbitrary classes have also been classified for these cases Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Ebony Elephant n-state m-symbol machines, n x m = 8 n m Total Terminated Going Unclassified 4 2 507,107 150,897 356,210 501,232 168,674 331,960 598 1's 1 2 3 4 5 6 7 8 9 10 11 12 13 # 3836 23161 37995 31023 15131 5263 1487 357 74 Beaver, Platypus, Unicorn, Zoo … Monash 29th June,2011

White Whale n-state m-symbol machines, n x m = 9 or 10 n m Total 3 25,955,301 5 2 99,174,375 ≥ 115,000,000 Need to improve the technology before taking this on ... Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Demon Duck of Doom n-state m-symbol machines, n x m = 12 (!??!!) n m Total 6 2 ?? 4 3 Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Universal Turing machines Quest for the smallest universal TM goes on … Involves searching similar (but larger) spaces Alain Colmerauer (KR’08 talk) U on code(M)code(w) simulates M on w Let M = U U on code(U)code(w) simulates U on w Let w = blank (and assume code(blank) = blank) U on code(U) simulates U on blank Hence pseudo-universality test: M is pseudo-universal if M on code(M) simulates M on blank Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Universal Turing machines What exactly is the definition of a universal Turing machine? How can such definitions be used to identify universal machines “in the wild”? What constraints are there on the coding function? Does a UTM have to terminate? Must a UTM terminate on code(M)code(w) exactly when M terminates on w? What is an appropriate “architecture” for a UTM? (code(M)code(w) vs code(w)code(M) ) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

2-D machines 2 1 1 Monash June 2nd, 2014 Beaver, Platypus, Unicorn …

2-D machines Can interpret numbers as images … Have generated some of the smaller cases here Very basic implementation of emulation Potentially very interesting loop behaviour Strict generalisation of one-dimensional case Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Learning? Ebony Elephant case suggests that `most' machines are boring … Only 2,667 are either high productivity or unclassified At most 100,000 of the non-terminators are difficult The remaining 80% are unexceptional … Can we find a criterion to identify (at least a substantial fraction) of the unexceptional machines? Need to identify possible features of machines ... Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Finite Decidability Anomaly Termination for TMs on blank input is undecidable Calculating bb(n) involves “only” a finite # machines … So there is an algorithm for computing bb(n) for each n (!!) Claim: Any decision problem over any finite set is decidable Proof Sketch: For a set of size n, there are only 2n possible decisions. Each such decision can be implemented by a Turing machine, ie. a “table” machine that outputs the decision value for each element i є {1 … n}. This is only a “table”, not an “algorithm”, but it is a Turing machine … QED Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Finite Decidability Anomaly Termination for TMs on blank input is undecidable Calculating bb(n) involves “only” a finite # machines … So there is an algorithm for computing bb(n) for each n (!!) Claim: Any decision problem over any finite set is decidable Proof Sketch: For a set of size n, there are only 2n possible decisions. Each such decision can be implemented by a Turing machine, ie. a “table” machine that outputs the decision value for each element i є {1 … n}. This is only a “table”, not an “algorithm”, but it is a Turing machine … QED Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Finite Decidability Anomaly Note: A decision procedure for a decidable problem over an infinite set is much shorter than the set itself So for a finite set, we should require that the Turing machine that decides it should be quantifiably shorter than the “table” which specifies the decision … An “algorithm” should be predictive, ie should work for more values than those specified in the table … (more thought needed!) Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Conclusions & Further Work Ebony elephant is close to capture White whale should be alert and alarmed (“Call me Ishmael’’ ) Demon Duck of Doom … [watch this space] Machine learning techniques may identify criteria which can reduce search space Duality between n-state 2-symbol and 2-state n-symbol cases Otter needs to be “remembered” Compress machines rather than fix macro size in advance Beaver, Platypus, Unicorn … Monash June 2nd, 2014

Conclusions & Further Work Weakening of conjectures for aberrent albatrosses Proofs desired for 3 instances for otter is sufficient Calculation of #steps (2nd order difference equations) Champion machines are always exhaustive Quadruple vs Quintuple machines Better “pruning” of search space Beaver, Platypus, Unicorn … Monash June 2nd, 2014