1. Model Based Systems Engineering Jonathan Sprinkle Center for Hybrid and Embedded Software Systems http://www.eecs.berkeley.edu/~sprinkle/

2. 2 22 May 2006 Jonathan Sprinkle, UC Berkeley Overview Nature/Nurture Motivation Methods Domain-specific modeling Generative techniques Applications (Previous/Possible) Avionics Autonomy Toolchain Constraints Wrap up, & looking forward

3. 3 22 May 2006 Jonathan Sprinkle, UC Berkeley Nature/Nurture Originally from Northeast TN, Southwest VA Graduated from Tennessee Tech, 1999 Double major: EE, CompE (1 st ever) Earned stripes as an engineer Graduate School at Vanderbilt University Masters Degree, 2000 PhD, 2003 Learned science of formal modeling Postdoc at UC Berkeley Cut teeth on application problems Executive Director of Center for Hybrid and Embedded Software Systems (Chess) Earned perspective on “big picture”

4. 4 22 May 2006 Jonathan Sprinkle, UC Berkeley These computers these days… Embedded systems are used for many kinds of purposes and products Fault diagnostics Onboard/autonomous strategies Medical devices Sensor networks Mobile phones Tricky part: software is I.Nontrivial II.Unpredictable III.Uncomposable IV.I and II V.II and III VI.I, II, and III Thanks to Gabor Karsai and Janos Sztipanovits for the inspiration for this slide.

5. 5 22 May 2006 Jonathan Sprinkle, UC Berkeley Back when I was a kid... Managing system level code from the code itself is an intractable problem Too many crosscutting req’ts (power, cache, ‘correctness’) Tight coupling between –System and (physical) environment –Language and (logical) encoding Small changes to requirements translate into large changes to implementation (i.e., large cost) “Why, back when I was a budding programmer, we didn’t even have keyboards!! We had to discover our own tools! And we worked our hairy little fingers to the bone. AND WE LIKED IT!” 2001: A Space Odyssey

6. 6 22 May 2006 Jonathan Sprinkle, UC Berkeley Proposed solution: Domain-Specific Models! Create model of the system Perform Analysis Architecture exploration Simulation Generate Configuration Code Executables From the same models! Example Domains & Environments: - VLSI Layout (e.g., Altera) - Engg Drawing (e.g., AutoCAD) - Physical Modeling (e.g., SolidWorks) - Signal Processing (e.g., LabVIEW) - Controls (e.g., Simulink) Model Interpretation Model Builder Model Interpreters Models DS Modeling Environment Application Domain App. 1 2 3 Application Evolution

7. 7 22 May 2006 Jonathan Sprinkle, UC Berkeley Domain-Specific Design: An abstract view Domain Concepts

8. 8 22 May 2006 Jonathan Sprinkle, UC Berkeley Domain-Specific Design: An abstract view Domain ConceptsDefns of Domain Assumptions and Givens

9. 9 22 May 2006 Jonathan Sprinkle, UC Berkeley Domain-Specific Code Generators Domain “Instance” DS Code Generator

10. 10 22 May 2006 Jonathan Sprinkle, UC Berkeley Domain-Specific Design: Analysis Domain “Instance” DS Code Generator Advantages: Infer execution structure from domain assumptions Reduce implementation-layer design/input errors Keep implementation details flexible Check design constraints during design Restrict User’s Implementation Space Disadvantages: Learning curve for design environment Time to build design environment Re-use cost

11. 11 22 May 2006 Jonathan Sprinkle, UC Berkeley Closing the loop: Metamodels Model Interpretation Model Builder Model Interpreters Models DS Modeling Environment Application Domain App. 1 2 3 Application Evolution Environment Evolution Meta-Level Translation Metamodeling Environment Formal Specifications

12. 12 22 May 2006 Jonathan Sprinkle, UC Berkeley Metamodels Allows: Rapid creation of Modeling Environment Formal structure of Model Builder Strong typing and constraint checking Automatic Modeling Environment Generation Advantages: Definition of metamodel strongly reflects system domain Language can be visually defined and implemented Documentation is the metamodel End results: System design can be managed by domain experts, not software experts Complex interdependencies checked through structural analysis, not enforced through style guides or memoranda

13. 13 22 May 2006 Jonathan Sprinkle, UC Berkeley Structure—Instance Sequence—meta Sequence—Instance Structure—meta Metamodel: Hierarchical Finite State Machines

14. 14 22 May 2006 Jonathan Sprinkle, UC Berkeley Execution Semantics While Event e i, and in State, s c After, e i.delay, and in State, s c, –Stop clock –If exists Transition t e : (src=s c, dst=s n ), set s c = s n –Else if s c.parent=null, set e i = e i.amSrc.sequence.dst –Else transition through s c.parent –Advance clock

15. 15 22 May 2006 Jonathan Sprinkle, UC Berkeley Modeling example: HFSM

16. 16 22 May 2006 Jonathan Sprinkle, UC Berkeley Power of Modeling: Example

17. 17 22 May 2006 Jonathan Sprinkle, UC Berkeley Power of Modeling: Example

18. 18 22 May 2006 Jonathan Sprinkle, UC Berkeley Generative Techniques Using the structure of the model, transform domain objects into executable objects, or into other transformable objects. Decide which static domain concepts, structural simplifications, requirements/constraints can be globally or locally determined during transformation, rather than during design.

19. 19 22 May 2006 Jonathan Sprinkle, UC Berkeley Domain-specific modeling w/ Generative Techniques Abstract Model Executable Model Solve problem in domain terms Assembler Map to code, implement Software Model Map to UML Generate, Add bodies Components Domain Model Generate calls to components No map! Code Map to code, implement Slide created by Juha Pekka Tolvanen, Matti Rossi, Jonathan Sprinkle

20. 20 22 May 2006 Jonathan Sprinkle, UC Berkeley Modeling language: formal definition L = < C ; A ; S ; M s ; M c > Thanks to Janos Sztipanovits for the inspiration of this slide. Abstract Syntax, A Semantic Domain, S Concrete Syntax, C Semantics M c M s

21. 21 22 May 2006 Jonathan Sprinkle, UC Berkeley Performing abstraction is… Removing Detail Implementation Specificity Adding Generality Paradigm-independence “Universal” meaning Extracting common elements into a parameterizable representation that can be used to recover the original representation, and possibly other archetypally equivalent representations.

22. 22 22 May 2006 Jonathan Sprinkle, UC Berkeley Where do they all fit? Generative Techniques Domain-specific modeling Embedded Controls Applications

23. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 23 22 May 2006 Jonathan Sprinkle, UC Berkeley Applications SEC Capstone Demonstration Pursuit/Evasion of fixed-wing aircraft Joint work with Dr. Mike Eklund, Dr. Jin Kim, Prof. Shankar Sastry [1] J. Mikael Eklund, Jonathan Sprinkle, S. Shankar Sastry, "Implementing and Testing a Nonlinear Model Predictive Tracking Controller for Aerial Pursuit Evasion Games on a Fixed Wing Aircraft", Proceedings of American Control Conference (ACC) 2005, 1509 – 1514, Portland, OR, Jun., 8–10, 2005. [2] Jonathan Sprinkle, J. Mikael Eklund, H. Jin Kim, S. Shankar Sastry, "Encoding Aerial Pursuit/Evasion Games with Fixed Wing Aircraft into a Nonlinear Model Predictive Tracking Controller", Proceedings of the 43rd IEEE Conference on Decision and Control, vol. 3, 2609 – 2614, Nassau, Bahamas, Dec., 14 – 17, 2004.

24. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 24 22 May 2006 Jonathan Sprinkle, UC Berkeley End Aircraft Control L ( ¢ ), x T k X 0 x k + u T k U 0 u k + b T m 1 B 0 1 b m 1 ( 1 ) Begin

25. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 25 22 May 2006 Jonathan Sprinkle, UC Berkeley End Aircraft Control L ( ¢ ), x T k X 0 x k + u T k U 0 u k + b T m 1 B 0 1 b m 1 ( 1 ) + b T m 2 B 0 2 b m 2 ( 2 ) Begin Obstacle

26. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 26 22 May 2006 Jonathan Sprinkle, UC Berkeley End Aircraft Control L ( ¢ ), x T k X 0 x k + u T k U 0 u k + b T m 1 B 0 1 b m 1 ( 1 ) + b T m 2 B 0 2 b m 2 ( 2 ) + b T m 3 B 0 3 b m 3 ( 3 ) Obstacle Boundary Begin

27. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 27 22 May 2006 Jonathan Sprinkle, UC Berkeley End Aircraft Control? Enemy… L ( ¢ ), x T k X 0 x k + u T k U 0 u k + b T m 1 B 0 1 b m 1 ( 1 ) + b T m 2 B 0 2 b m 2 ( 2 ) + b T m 3 B 0 3 b m 3 ( 3 ) + b T m ? B 0 ? b m ? ( 4 ) Begin Boundary Obstacle Whatsa Matter? Are ya Chicken? J = N ¡ 1 P k = 0 L ( x k ; u k ; b 1 k :: M k ) = 0

28. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 28 22 May 2006 Jonathan Sprinkle, UC Berkeley Constraints/Rules Ingress Endpoint “Win” Point for Friendly Adversary Ingress Waypoint Friendly Ingress Waypoint 10 Minutes Time Limitation for Adversary Win Condition Pursuer has targeted Evader Boundary for all vehicles

29. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 29 22 May 2006 Jonathan Sprinkle, UC Berkeley Open Source ITAR Restricted System Architecture x86 Laptop, Linux RH9 (perfctr-kernel 2.4) Boeing T-33 Trainer Jet ca. 1953 API Exposed u = h u _ v ; u _ Ã ; u _ z i Dynamics Abstracted _ x = f ( x ; u ) = D emo S i m ( u ) DemoSim u _ x Pursuit/Evasion Logic + map ( u _ v ) = 8 < : ¡ 50 [ f / s ] [ ¡ 50 ; 50 ][ f / s ] 50 [ f / s ] ¡ 1 < u _ v < ¡ 1 ¡ 1 6 u _ v 6 1 1 < u _ v < 1 map ( u _ Ã ) = 8 < : ¡ ¼ = 50 [ s ¡ 1 ] [ ¡ ¼ = 50 ; ¼ = 50 ][ s ¡ 1 ] ¼ = 50 [ s ¡ 1 ] ¡ 1 < u _ Ã < ¡ 1 ¡ 1 6 u _ Ã 6 1 1 < u _ Ã < 1 map ( u _ z ) = 8 < : ¡ 10 [ f t / s ] [ ¡ 10 ; 10 ][ f t / s ] 10 [ f t / s ] ¡ 1 < u _ z < ¡ 1 ¡ 1 6 u _ z 6 1 1 < u _ z < 1 ; Control Input Constraints Game Constraints Enabled by Component technology

30. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 30 22 May 2006 Jonathan Sprinkle, UC Berkeley Methodology

31. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 31 22 May 2006 Jonathan Sprinkle, UC Berkeley Application Results: Pre-VIP Testing

32. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 32 22 May 2006 Jonathan Sprinkle, UC Berkeley Application Results: VIP Day Pilot comment: “Plane reacted just like pilots are trained to do” Sprinkle Translation: “I couldn’t tell whether it was a computer or person controlling plane.”

33. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 33 22 May 2006 Jonathan Sprinkle, UC Berkeley Applications SEC Capstone Demonstration Landing/Wave-off scenario (safety calculation) Joint work with Dr. Mike Eklund, Dr. Ian Mitchell, Prof. Shankar Sastry [3] Jonathan Sprinkle, Aaron D. Ames, J. Mikael Eklund, Ian Mitchell, S. Shankar Sastry, "Online Safety Calculations for Glideslope Recapture", Innovations in Systems and Software Engineering, vol. 1, no. 2, pp. 157 – 175, Sep. 2005. [4] Jonathan Sprinkle, J. Mikael Eklund, S. Shankar Sastry, "Deciding to Land a UAV Safely in Real Time", Proceedings of American Control Conference (ACC) 2005, 3506–3511, Portland, OR, Jun. 8 – 10, 2005.

34. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 34 22 May 2006 Jonathan Sprinkle, UC Berkeley Landing Scenario Consider a fixed-wing UAV following its glideslope

35. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 35 22 May 2006 Jonathan Sprinkle, UC Berkeley Motivating Example It is directed off its landing path

36. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 36 22 May 2006 Jonathan Sprinkle, UC Berkeley Motivating Example And after some time redirected to land Can the decision to safely land: - be made in real time? - be guaranteed as true? - be guaranteed as true?

37. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 37 22 May 2006 Jonathan Sprinkle, UC Berkeley Engineering Problems Answering “in time” Question asked Now x x x timesteps

38. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 38 22 May 2006 Jonathan Sprinkle, UC Berkeley Engineering Problems Question asked Now x x x timesteps

39. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 39 22 May 2006 Jonathan Sprinkle, UC Berkeley Engineering Problems Question asked Now x x x timesteps

40. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 40 22 May 2006 Jonathan Sprinkle, UC Berkeley Engineering Problems Question asked Now timesteps x x x Answer givenAction will be taken The computation interval should influence the state data used for the calculation (derived from validation interval) i.e., you should use the validation interval to ask about the time at which you would actually be able to do something Computation Interval Validation Interval

41. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 41 22 May 2006 Jonathan Sprinkle, UC Berkeley Technology and Analysis Solutions for Reachability Figures by Ian Mitchell Forward: Must be recomputed for each start point  Both dimensionally exponential  5 dimen: ~hours to compute  6 dimen: ~weeks Backward: Must be recomputed for each end point

42. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 42 22 May 2006 Jonathan Sprinkle, UC Berkeley Implementation and Results InitialRunway

43. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 43 22 May 2006 Jonathan Sprinkle, UC Berkeley Implementation and Results All pieces fit together, step size changes by power of 10 to match required resolution [0,3) [3,10)

44. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 44 22 May 2006 Jonathan Sprinkle, UC Berkeley Implementation and Results All pieces fit together, step size changes by power of 10 to match required resolution [0,1) [1,3) [3,10)

45. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 45 22 May 2006 Jonathan Sprinkle, UC Berkeley Reach-Set Generator Compilation/Linking Generative Strategy Decision Controller Generator G 0 = 8 > > > > > > < > > > > > > : µ G 2 [ 2 : 85 ± ; 3 : 15 ± ] ; Ã G 2 [ ¡ 0 : 2 ± ; + 0 : 2 ± ] ; x 2 2 [ ¡ 100 ; + 100 ] f t ; x 3 2 [ ¡ 15 ; + 15 ] f t ; x 1 = 0 Testbed Deployment #define #define #include... #define #define #include... #define #define #include... 0100101 1011110 0101101 0001010 1101010 0011111 0100101 1011110 0101101 0001010 1101010 0011111 0100101 1011110 0101101 0001010 1101010 0011111 online query real- time result H ( ~ x ; p ) = m i n u 2 U p T ~ f ( ~ x ; u ) f x 1 ( ~ x ) 0 @ _ x 1 _ x 3 _ µ 1 A = f ( x 1 ; x 3 ; µ ; K ( x 1 ; x 3 ; µ ))

46. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 46 22 May 2006 Jonathan Sprinkle, UC Berkeley Analysis Safety of ground and vehicle increased reduced stress and decision load for pilot aircraft training less of a factor than before hyper-accurate safe set calculations Design lends itself to multiple aircraft simply create new sets based on constraints no increase in computation (simple lookup) uniform integration strategy Level of autonomy increased multiple sets for different scenarios (hazardous weather, wartime, etc.) guaranteed within operational parameters

47. 47 22 May 2006 Jonathan Sprinkle, UC Berkeley Other Applications: Automotive Comms Drivetrain communication Software-enabled Emissions Reduction Entertainment Options Passenger Comfort settings Traction Control Force Feedback Supplemental Restraint System Computational and control units (ECUs) must be composable. Increasing numbers make communication difficult, and provable confidence elusive. Wireless Communication Weight Reduction

48. Led by J. Mikael Eklund, Shankar Sastry 48 22 May 2006 Jonathan Sprinkle, UC Berkeley Other Applications: Healthcare@Home

49. 49 22 May 2006 Jonathan Sprinkle, UC Berkeley Modeling Agenda: Toolchain Support Simulation Center for Hybrid and Embedded Software Systems Exploration Generation Construction Verification Platform

50. 50 22 May 2006 Jonathan Sprinkle, UC Berkeley Conclusions Applications of Embedded Systems are cool! Complexity and interconnectedness require theoretical understanding in order to succeed Only with high-confidence systems can autonomous, assistive, and x-by-wire systems be deployed in society Through model-based design, high-confidence systems can be built by domain experts

51. 51 22 May 2006 Jonathan Sprinkle, UC Berkeley More Reading [1] J. Mikael Eklund, Jonathan Sprinkle, S. Shankar Sastry, "Implementing and Testing a Nonlinear Model Predictive Tracking Controller for Aerial Pursuit Evasion Games on a Fixed Wing Aircraft", Proceedings of American Control Conference (ACC) 2005, 1509–1514, Portland, OR, Jun. 8–10, 2005. [2] Jonathan Sprinkle, J. Mikael Eklund, H. Jin Kim, S. Shankar Sastry, "Encoding Aerial Pursuit/Evasion Games with Fixed Wing Aircraft into a Nonlinear Model Predictive Tracking Controller", Proceedings of the 43rd IEEE Conference on Decision and Control, vol. 3, 2609–2614, Nassau, Bahamas, Dec. 14–17, 2004. [3] Jonathan Sprinkle, Aaron D. Ames, J. Mikael Eklund, Ian Mitchell, S. Shankar Sastry, "Online Safety Calculations for Glideslope Recapture", Innovations in Systems and Software Engineering, vol. 1, no. 2, pp. 157–175, Sep. 2005. [4] Jonathan Sprinkle, J. Mikael Eklund, S. Shankar Sastry, "Deciding to Land a UAV Safely in Real Time", Proceedings of American Control Conference (ACC) 2005, 3506–3511, Portland, OR, Jun. 8–10, 2005. [5] Jonathan Sprinkle, "Model-Integrated Computing", IEEE Potentials, vol. 23, no. 1, pp. 28–30, Feb. 2004. [6] Jonathan Sprinkle, "Generative Components for Hybrid Systems Tools", J. of Obj. Tech., vol. 4, no. 3, pp. 35–40, Apr. 2005.

52. 52 22 May 2006 Jonathan Sprinkle, UC Berkeley Thanks! http://www.eecs.berkeley.edu/~sprinkle/ http://sprinkletoday.com/ sprinkle@EECS.Berkeley.Edu sprinkle@IEEE.org sprinkle@acm.org

53. 53 22 May 2006 Jonathan Sprinkle, UC Berkeley

54. 54 22 May 2006 Jonathan Sprinkle, UC Berkeley Backup slides...

55. 55 22 May 2006 Jonathan Sprinkle, UC Berkeley Fixed wing application MPC as a supervisory controller already operational on rotary UAVs Dynamics and constraints are quite different than for rotary wing aircraft Entirely new aircraft model required Tactics Function of constraints on fixed wing aircraft, in particular –Minimum airspeed –Maximum turn rate

56. 56 22 May 2006 Jonathan Sprinkle, UC Berkeley Pursuit/Evasion Game Rules Goals Evader: get to final waypoint or avoid evader for 10 minutes Pursuer goal: ‘target’ evader Targeting Evader and Pursuer cones are aligned Wrinkles Evader can become Pursuer, if “target of opportunity” is recognized Pursuer may not know Evader’s state Technicals 3000’+ of vertical separation at all times (safety) Pursuer (F-15) “pretends” performance constraints

57. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 57 22 May 2006 Jonathan Sprinkle, UC Berkeley Logical Implementation (a) safe-set of operation relative to the desired point of landing on the virtual runway (f). (b) vector-off maneuver requested (c) command to land (if possible) is given (d) aircraft will continue to vector-off (if landing is unsafe) or will issue commands to recapture the glideslope at some point (e).

58. With Aaron D. Ames, J. Mikael Eklund, Ian M. Mitchell, Shankar Sastry 58 22 May 2006 Jonathan Sprinkle, UC Berkeley Final Words Implementation Use backward reach set to make one lookup table for each 3D vector –~7MB total size –Lookup time: ~10ms (~5ms each) –Time to generate: ~15 mins/set, 2 hours compilation time Total development time ~2 man months of coding, including design and research required for safe sets Results Flown T-33, landing on “virtual” runway at a high altitude Ground controller gives vector-off and recapture commands –1 successful landing –1 go-around after “unsafe” answer (later verified offline as a correct result)

59. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 59 22 May 2006 Jonathan Sprinkle, UC Berkeley Early simulation results

60. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 60 22 May 2006 Jonathan Sprinkle, UC Berkeley Early simulation results

61. With J. Mikael Eklund, H. Jin Kim, Shankar Sastry 61 22 May 2006 Jonathan Sprinkle, UC Berkeley Early simulation results

62. 62 22 May 2006 Jonathan Sprinkle, UC Berkeley IDE 1 Transform 1 Transform 2 DSME 1 Desig n.cpp Compile/Link exe IDE 1 Transform 1 Transform 2 DSME 2 Desig n Compile/Link exe.cpp Design IDE 1 Transformer 1 Transformer 2 #define #define #include... #define #define #include... #define #define #include... Com/Link 1 0100101 1011110 0101101 0001010 1101010 0011111 0100101 1011110 0101101 0001010 1101010 0011111 0100101 1011110 0101101 0001010 1101010 0011111

63. 63 22 May 2006 Jonathan Sprinkle, UC Berkeley Design IDE 1 Transformer 1 Transformer 2 Com/Link 1 Design IDE 1 Transformer 1 Transformer 2 Com/Link 1

64. 64 22 May 2006 Jonathan Sprinkle, UC Berkeley Features Stability S A F E T Y Confidence Complexity Composability

65. 65 22 May 2006 Jonathan Sprinkle, UC Berkeley Generative Techniques Arguably, a model is nothing more than documentation unless it is directly useful in the design and/or implementation of the final system Generative techniques move you from “art history” to “systems science” Ahh, Magritte’s pipe. The complex expression of realism combined with the birth of abstract art and the (arguable) founding of post-modernist expression through paradox. This is not Magritte’s Pipe.