1. Mining Requirements from Closed Loop Control Models Jyotirmoy V. Deshmukh Xiaoqing Jin Alexander Donzé Sanjit A. Seshia Joint work with: TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA AAA A A A A A A A A A Powertrain Control/Model-Based Design/V&V

2. Traditional Requirement Driven Design Design Requirements/Targets  Aren’t you supposed to check if design satisfies requirements/specifications/properties? Mining Requirements from Closed-Loop Models 2/45

3. Challenges  Closed-loop setting very complex:  nonlinear dynamics  look-up tables  large amounts of switching  components with no models  unclear semantics of modeling language  Requirements  Design process is not always “waterfall”  Controller performance requirements need to be identified from high-level requirements Mining Requirements from Closed-Loop Models 3/45

4. What this work is all about …  How we could use formal reasoning when all we have is:  Ability to simulate and test system  Vague idea of what system should satisfy  (Possibly limited) ability to check if system satisfies property Requirement Mining! Mining Requirements from Closed-Loop Models 4/45

5. Powertrain Control Design Process Controller model Coding Integrated code System evaluation Control design Prototype modeling and implementation System evaluation Feasibility/Requirement analysis Requirement New design Legacy Code Rapid Iteration Enhance set of requirements & make amenable to V&V Use for testing Requirements and Design co-developed Requirements and Design co-developed Mining Requirements from Closed-Loop Models 5/45

6.  Ask designer if mined requirements are OK  “Settling time is 6.25 ms”  “Overshoot is 100 units”  ‘As-is’ properties of closed-loop design Mining in Action 6.25ms 100 Mining Requirements from Closed-Loop Models 6/45

7. Legacy code It’s working, but I don’t understand why! Value added by mining:  Mined Requirements become useful documentation  Use for code maintenance and revision  Use during tuning and testing Mining Requirements from Closed-Loop Models 7/45

8. Mine for one version, get many free Requirement 1 Requirement 2 Requirement 3 Version 0 Version 1Version 2 Mine Requirements Use for V & V Use for V & V Use for V & V Refine Mining Requirements from Closed-Loop Models 8/45

9. Outline  Expressing Requirements in Signal Temporal Logic  Mining Algorithm  Experimental Results  Sneak Preview: Mining Performance Bounds (if time permits) Mining Requirements from Closed-Loop Models 9/45

10. Expressing Requirements in Signal Temporal Logic Mining Requirements from Closed-Loop Models 10/45

11. Signal Temporal Logic (STL)  Extension of Metric Temporal Logic (MTL)  Allows tests over continuous-valued signal variables  Examples:  0 10050 1 3 0 100 1 -0.1 +0.1 60 Mining Requirements from Closed-Loop Models 11/45

12. Quantitative Semantics of STL  Function  that maps STL formula  to a numeric value  Quantifies “how much” a trace satisfies a property  Large positive value: trace easily satisfies   Small positive value: trace close to violating   Negative value: trace does not satisfy  Mining Requirements from Closed-Loop Models 12/45

13. Mining Algorithm Mining Requirements from Closed-Loop Models 13/45

14. CounterExample Guided Inductive Synthesis Find “Tightest” Answers Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? YES Settling Time is 5 ms Overshoot is 5 KPa Upper Bound on x is 3.6 Settling Time is 5 ms Overshoot is 5 KPa Upper Bound on x is 3.6 1. m. Mining Requirements from Closed-Loop Models 14/45

15. Settling Time is 5.3 ms Overshoot is 5.1 KPa Upper Bound on x is 3.8 Settling Time is 5.3 ms Overshoot is 5.1 KPa Upper Bound on x is 3.8 Settling Time is … ms Overshoot is … KPa Upper Bound on x is … Settling Time is … ms Overshoot is … KPa Upper Bound on x is … CounterExample Guided Inductive Synthesis Find “Tightest” Answers Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? Counterexamples 1. m. 1. n. YES Mining Requirements from Closed-Loop Models 15/45

16. CounterExample Guided Inductive Synthesis Find “Tightest” Answers Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Settling Time is ?? Overshoot is ?? Upper Bound on x is ?? Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? Settling Time is 6.3 ms Overshoot is 5.6 KPa Upper Bound on x is 4.1 Settling Time is 6.3 ms Overshoot is 5.6 KPa Upper Bound on x is 4.1 NO Settling Time is 6.3 ms Overshoot is 5.6 KPa Upper Bound on x is 4.1 Settling Time is 6.3 ms Overshoot is 5.6 KPa Upper Bound on x is 4.1 Mined Requirement 1. n. Counterexamples 1. m. Mining Requirements from Closed-Loop Models 16/45

17. Zooming in … Mining Requirements from Closed-Loop Models Find “Tightest” Answers Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? 17/45

18. Parametric STL  Constants in STL formula replaced with parameters  Scale parameters  Time parameters  Examples: Between some time and 10 seconds, x remains greater than some value After transmission shifts to gear 2, it remains in gear 2 for at least secs Mining Requirements from Closed-Loop Models Find “Tightest” Answers 18/45

19.  Validity domain: Set of values of p for which resulting formula “holds” for current set of traces PSTL formula  ( p )  p = ( )  Plug in values for p to get STL formula Mining Requirements from Closed-Loop Models Find “Tightest” Answers 19/45

20. Parameter Synthesis  Find  -tight values  Nonlinear optimization problem, multiple criteria  Solution not unique, need to find Pareto-optimal solution Mining Requirements from Closed-Loop Models 0 100 3 2.9 1 000 000 Find “Tightest” Answers 20/45

21. Parameter Synthesis  Naïve approach:  grid parameter space  evaluate satisfaction value at each point  pick valuation with smallest satisfaction value  Exponential number of points in parameter space  Could miss optimal values Mining Requirements from Closed-Loop Models Find “Tightest” Answers 21/45

22. If upper bound of all signals is 3, any number > 3 is also an upper bound  Satisfaction value monotonic in parameter value  Binary-search in monotonic parameter dimensions  Now implemented in tool B REACH Satisfaction Monotonicity 0 10050 3 4 Mining Requirements from Closed-Loop Models Find “Tightest” Answers 22/45

23. Checking Monotonicity  Checking monotonicity is undecidable  Encode monotonicity check as SMT query  F.O. Logic with quantifiers + uninterpreted functions + real arithmetic  Return “yes”/ “no” / “unknown”  If “yes” – proof of monotonicity  If “no”/“unknown” – fall back to naïve procedure Mining Requirements from Closed-Loop Models Find “Tightest” Answers 23/45

24. Falsification uS(u) Falsification Tool \  \ Mining Requirements from Closed-Loop Models Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? 24/45

25. Falsification as Optimization  Solve  If < 0, found falsifying trace!  Use stochastic optimization such as in S-T ALIRO /B REACH  “Parameterization” of input signal space  Breach: flexible parameterization  S-Taliro: interpolating Hermite polynomials Nonlinear Optimization Problem, No exact solution, Limited formal guarantees Mining Requirements from Closed-Loop Models Are there behaviors that do NOT satisfy these requirements? Are there behaviors that do NOT satisfy these requirements? 25/45

26. Mining in a nutshell B REACH Template PSTL property S-T ALIRO / B REACH falsified Requirement? S-T ALIRO / B REACH falsified Requirement? Candidate Requirement NO Mined STL Requirement 1. n. Counterexamples 1. m. YES Mining Requirements from Closed-Loop Models 26/45

27. Experimental Results Mining Requirements from Closed-Loop Models 27/45

28. Experimental Results for Mined Requirements Time Taken (seconds) Number of Simulations Parameter Synthesis B REACH Falsifier Upper bounds on speed & rpm 23 197496 Cannot reach 100mph in  seconds with rpm <  11 267709 Cannot reach 100mph in  seconds with rpm <  5 148411 Minimum Dwell time in Gear 2 2 4311015 Mining Requirements from Closed-Loop Models 28/45

29. Experimental Results  Found max overshoot with 7000+ simulations in 13 hours  Attempt to mine maximum observed settling time:  stops after 4 iterations  gives answer t settle = simulation time horizon (shown in trace below) Experimental Engine Control Model Mining Requirements from Closed-Loop Models 4000+ Simulink blocks, Look-up tables + Very nonlinear dynamics 29/45

30. Mining can expose deep bugs  Each iteration produced intermediate requirements  Pushed falsification to explore trajectories more likely to altogether violate requirement  Discussion with control designer revealed it to be a real bug  Root cause identified as wrong value in a look-up table, bug was fixed  Why mining could be useful for bug-finding:  Mining provides better “direction” information to optimizer  Looking for bugs  Mine for negation of bug Experimental Engine Control Model Mining Requirements from Closed-Loop Models 30/45

31. References  B REACH & STL: http://www.eecs.berkeley.edu/~donze/breach_page.html 1. Alexander Donzé, Oded Maler. Robust satisfaction of temporal logic over real- valued signals. Formal Modeling and Analysis of Timed Systems, 2010. 2. Alexander Donzé. Breach: A Toolbox for Verification and Parameter Synthesis of Hybrid Systems. CAV, 2010. 3. Eugene Asarin, Alexander Donzé, Oded Maler and D. Nickovic. Parametric identification of temporal properties. Runtime Verification, 2011.  S-T ALIRO : https://sites.google.com/a/asu.edu/s-taliro/s-taliro 1. Sriram Sankaranarayanan and Georgios Fainekos. Falsification of temporal properties of hybrid systems using the cross-entropy method. HSCC 2012. 2. Y. Annpureddy. C. Liu, G. E. Fainekos, and S. Sankaranarayanan. S- TaLiRo: A tool for Temporal Logic Falsification for Hybrid Systems: TACAS 2011. Mining Requirements from Closed-Loop Models 31/45

32. Sneak Preview: Mining Performance Bounds Principal Investigator: Jim Kapinski Joint work with: Sriram Sankaranarayanan (Univ. of Colorado Boulder) Nikos Arechiga (Carnegie Mellon University) & me

33. Control-theory influenced Requirements  Better characterization of performance bounds  Can be easier to understand for a control designer  Use CEGIS approach for stability-like properties Mining Requirements from Closed-Loop Models 33/45

34. Lyapunov Stability Mining Requirements from Closed-Loop Models 34/45

35. Lyapunov’s Second Method Mining Requirements from Closed-Loop Models 35/45

36. Simulation-Guided Search Mining Requirements from Closed-Loop Models 36/45

37. CEGIS of Candidate Lyapunov functions Mining Requirements from Closed-Loop Models 37/45

38.  Combined constraints give:  Solve: Candidate function using a SDP solver Semi-Definite Program Solve using YaLMIP, SeDuMi Semi-Definite Program Solve using YaLMIP, SeDuMi Mining Requirements from Closed-Loop Models 38/45

39. Global Optimizer for Counter-Examples  Optimizer finds traces violating Lyapunov conditions  Add constraints from new traces to refine candidate  Terminate when no more counterexamples found  For Extra Credit:  Compute largest invariant set within region of interest using convex optimization Mining Requirements from Closed-Loop Models 39/45

40. Results: Time-Reversed Vanderpol Able to prove true Lyapunov function using SoS techniques, KeyMaera and Mathematica Mining Requirements from Closed-Loop Models 40/45

41. Results: Moore-Greitzer Jet Engine Verified Lyapunov function using KeyMeara/Mathematica Mining Requirements from Closed-Loop Models 41/45

42. Results: Switched Mode Linear System Johansson shows: LMI-based formulation utilizing system dynamics for finding a common quadratic Lyapunov function fails Example from “Piecewise Linear Control Systems,” by Mikael Johansson Found common Lyapunov function from just simulations Mining Requirements from Closed-Loop Models 42/45

43. Results: Damped Mathieu System Transcendental, Time-varying dynamics Verified Lyapunov function using KeyMeara/Mathematica Mining Requirements from Closed-Loop Models 43/45

44. References  Topcu, U., Seiler, P., Packard, A.: Local stability analysis using simulations and sum- of-squares programming. Automatica 44, 2669-2675 (2008)  Topcu, U., Packard, A.: Stability region analysis for uncertain nonlinear systems. IEEE Transactions on Automatic Control 54, 1042–1047 (2009)  Prajna, S.: Optimization-based methods for nonlinear and hybrid systems verification. Ph.D. thesis, Pasadena, CA, USA (2005)  SeDuMi: http://sedumi.ie.lehigh.edu/  YalMIP: http://users.isy.liu.se/johanl/yalmip/ Mining Requirements from Closed-Loop Models 44/45

45. Thank You! Mining Requirements from Closed-Loop Models 45/45

46. Mining Requirements from Closed-Loop Models 46/45 Backup Slides

47. Syntax & Semantics SyntaxSemantics Mining Requirements from Closed-Loop Models 47/45

48. Quantitative Semantics of STL  Following (satisfaction value) does the trick Mining Requirements from Closed-Loop Models 48/45

49. Quantitative Semantics Demystified 010.5-0.5 0.5 0.10.20.30.40.5 0.6 0.7 1 2 00.5 1 1 sup over each interval Mining Requirements from Closed-Loop Models 49/45

50. Quantitative Semantics Demystified 010.5-0.5 0.5 0.10.20.30.40.5 0.6 0.7 1 2 00.5 1 1 = 0.5 inf over result from previous step Mining Requirements from Closed-Loop Models 50/45