1. Self-* Programming: Run-Time Parallel Control Search for Reflection Box Olga Brukman and Shlomi Dolev Ben-Gurion University of the Negev Israel

2. Problem Ideally – Systems should anticipate every possible scenario Reality – Engineers fail to create such systems despite the effort – The number of possibilities of different interactions with dynamic environment is enormous – Relying on accumulated knowledge of human operator to deal with unexpected situations 2

3. Example 1 Airplane flying into ash cloud – Engines stopped – Pilots managed to fly the plane out of the cloud, waited till the engines cooled down, and were able to restart them 3

4. Example 2 Airplane crossing the speed of sound – Airplane control handles behave opposite to the expected behavior – Pilots increase the plane speed so it becomes higher than speed of sound, plane control is back to normal 4

5. Our Contribution: Automatic Control Search Engine for Dynamic Environment No assumptions on possible environment changes Experimentation on replicas Parallelization of experiments Polynomial search time – Parallelization – Exposing system state Observing system state (e.g., with Java reflection) Setting system state to a certain state 5

6. System Settings: Environment Environment Non-deterministic infinite automaton t Environment DA/PA(F 1 ) Environment DA/PA(F 2 ) Environment DA/PA(F 3 ) t Environment is large, sophisticated, dynamic – Non deterministic infinite automaton At every given time slot environment is deterministic automaton probabilistic automaton with a transition function F i 6

7. System Settings: Environment vs. Plant Our program (control) interacts with some machinery in environment– plant System Environment Control in 1,.., in k out 1, …,out l Plant Environment in 1 …...., in m out 1,……,out n 7

8. System Settings: Environment and Plant Environment is – Reentrant : no mutual replicas interference – History oblivious: deterministic repetition of behavior for the plants in the same initial state and with the same control System Environment Control 1 Plant in 1 …...., in m out 1,……,out n Control 2 Plant in 1 …...., in m out 1,……,out n Control 1 Plant in 1 …...., in m out 1,……,out n 8

9. System Control Plant Environment in 1 …...., in m out 1,……,out n in 1,.., in k out 1, …,out l Control Search Engine Observer plant- environment interaction Control Generator Goals= Behaviors bhv={io 1,…,io j } |A P |=N N max Recording plant-environment interaction 9

10. Settings: Periodic Control Plant automaton + = in 1 in 4 in 3 in 2 io 1 io 2 io 3 io 4 P=3 10 bhv={io 2, io 3, io 4 }, io 2, io 3, io 4 C={in 1 (in 2, in 3, in 4 )*}

11. Control Search Algorithms Plant State SetPlant State Reflection Algorithm Deterministic Environment xxI: black box xxII: set box xvIII: reflection- box vvIV: reflection- set box Probabilistic Environment vvV: reflection- set box xvVI: set box 11

12. Control Search Algorithms for Deterministic Environment 12

13. Algorithm I: black-box Complexity – Total number steps in experiments: – Longest experiment: O(PN max ) s s s s … C1C1 C2C2 CMCM s curr xP s curr s P_1 s P_2 s P_N_max+1 … 0<iN max P P(N max +1) s P_3 P 13 s start

14. Algorithm IV: reflection-set box s1s1 sNsN …. σ1σ1 σ2σ2 σΣσΣ... σ1σ1 σ2σ2 σΣσΣ ApAp Off line search of the constructed plant automaton Try all controls from every state Complexity Total number of steps in experiments: Longest experiment: O(1) 14

15. Control Search Algorithms for Probabilistic Environment

16. Probabilistic Environment Plant is unaware of the entire state of the environment – Environment can be considered to be probabilistic automaton – Plant transition function is probabilistic Control search algorithm executes all the time – Due to probabilistic transition function of plant automaton Monitoring – Recognize changes in the plant probabilistic transition function 16

17. Probabilistic Plant Automaton 0pr(s, s, σ, io) 1 pr min – minimal probability sisi sjsj 17 σ 1, io 1,1, pr 1,1 σ 1, io 1,2, pr 1,2 … σ m, io m,1, pr m,1 σ m, io m,2, pr m,2

18. Computing Probabilistic Plant Automata Graph (PPAG) sisi SF times σ sNsN …s i+1 s i-1 …s2s2 s1s1 σ, io 0.25…0.050.06…0.080.02 s1s1 0.2…0.020.6…0.070.04 s2s2 … 0.4…0.10.02…0.010.5 sNsN PPAG[s i,s j,σ,io]=pr SF=1-(1/pr min ): number of experiments required to discover the edges with the smallest probability sj?sj? io? 18

19. Preprocessing: Behavior Suffix Probability (BSP) Table For every state s and j=1,…,|bhv|: – BSP[s i,j] = [pr max, σ] pr max is the maximal probability to obtain suffix (bhv,j) starting from plant in state s i with σ as first entry in the control BSP computed from PPAG sisi σ j pr max pr 2 pr 1 pr 3 19

20. Preprocessing: Behavior Suffix Probability (BSP) Table Cont. Base step: for every s i compute BSP[s i,1] – For every s i find input σ that produces bhv[k] with the highest probability : max{PPAG[s i, *, σ,bhv[k]]} pr 1 σ 1 (io 1 ) sisi 1 pr max pr 2 pr 3 σ (bhv[k]) σ 2 (io 2 ) σ 3 (io 3 )

21. Preprocessing: Behavior Suffix Probability (BSP) Table Cont. Assume we computed all entries BSP[s i,j] for every s i and jm BSP[s i, (m+1)]? – Let σ be the one that yields the largest value of Σ s_j (PPAG[s i,s j, σ,bhv[k-m]]BSP[s,m].pr) – BSP[s i, m+1]=[Σ s_j (PPAG[s i,s, σ,bhv[k-m]]BSP[s,m].pr), σ] σ 1 (bhv[k-m]) sisi 1 pr 3 = Σ s_j (PPAG[s i,s j,σ 3,bhv[k-m]].prBSP[s j,m].pr) σ (bhv[k-m]) σ 2 (bhv[k-m]) σ 3 (bhv[k-m]) s,s, s,… s,s, s,.. s,s,s, pr max =Σ s_j (PPAG[s i,s j, σ,bhv[k-m]]BSP[s j,m].pr) pr 2 = Σ s_j (PPAG[s i,s j,σ 2,bhv[k-m]].prBSP[s j,m].pr) pr 1 = Σ s_j (PPAG[s i,s j,σ 1,bhv[k-m]].prBSP[s j,m].pr)

22. Algorithm V: reflection-set box s start s next s start =max{BSP[s,|bhv|].pr} BSP[s next,k-1]>BSP[s start,k] BSP[s start,k].σ pr=0.4 pr=0.6 BSP[s next,k-1]<BSP[s start,k]

23. Algorithm VI: set box s start BSP[s next,k-1]>BSP[s start,k] BSP[s start,k].σ S best =max{BSP[s,k].pr} s best =max{BSP[s,k].pr} s best s next 23 pr=0.7 pr=0.3 BSP[s next,k-1]<BSP[s start,k]