1. Model-based Testing of Hybrid Systems Michiel van Osch IPA Spring Days on Testing 19 April – 21 April 2006

2. 2 Motivation Hybrid Systems Testing might be expensive, dangerous, or resources might be limited Discrete and real-time model-based testing does not test the continuous aspects of the system

3. 3 Content Refresh –Model-based Testing –Discrete Conformance & ioco Tests –Timed Conformance & tioco Tests Hybrid Systems Hybrid Input-output Conformance Hybrid Input-output Conformance Tests Conclusions –Results –Limitations and Future Work

4. 4 Automated Model Based Testing model SUT IUT conforms to model pass fail test tool test generation tool test execution tool IUT passes tests IUT conf model    soundexhaustive

5. 5 Input-output Conformance Impl. ioco Spec. iff for all traces α: out(Impl. after α)  out(Spec. after α)

6. 6 Test-case Generation and Execution 1.Terminate with verdict pass 2.Select an input from the specification and apply it to the implementation 3.Observe an output or a timeout from the implementation and check if it is allowed according to the specification

7. 7 Example s0s1s4 s2 s3 ?Button2 !Tea ?Button1 ?Button2 ?Button1 !XLCoffee !Coffee t0 t1 ?Button1 t3 ?Button2 Pass !XLCoffee t2 !! Fail !Coffee!Tea !! Fail !Coffee!Tea !XLCoffee

8. 8 Timed Input-output conformances Different views: time as observable event versus time as parameter of the (output) actions s0s3 !tick !tick !XLCoffee Vs. s0s3 !XLCoffee(4)

9. 9 Timed Input-output Conformance (Krichen) Impl. tioco Spec. iff for all traces α: out(Impl. after α)  elapse(Impl. after α)  out(Spec. after α)  elapse(Spec. after α)

10. 10 Timed Test-case Generation and execution 1.Terminate with verdict pass 2.Select an input from the specification and apply it to the implementation 3.Observe an output or time passing from the implementation and check if it is allowed according to the specification

11. 11 Example Test-case Generation t0 t1 ?Button1 t2 !tick Fail t3 ?Button2 !tick Pass !XLCoffee Fail !tick Fail !tick s0s1s4 s2 s3 ?Button2 !Tea ?Button1 ?Button2 ?Button1 !XLCoffee !Coffee !tick !tick !tick !tick !tick !tick

12. 12 Moving forward …

13. 13 Content Refresh –Model-based Testing –Discrete Conformance & ioco Tests –Timed Conformance & tioco Tests Hybrid Systems Hybrid Input-output Conformance Hybrid Input-output Conformance Tests Conclusions –Results –Limitations and Future Work

14. 14 Hybrid Systems Practically: –Discrete behavior plus continuous behavior –Continuous behavior can be input observed through sensors or output generated by actuators Mathematically –Discrete actions plus flow of continuous variables (trajectories) –Variables can be input variables and output variables

15. 15 Hybrid Systems Time Coffee 0 ?Button1 ?Button2 s0s1 s2 s3 ?Button1 ?Button2 ?Button1 Coffee’ = 4 cl/sec. Coffee’ = 3 cl/sec. Coffee’ = 0

16. 16 Hybrid Systems s0s1 s2 s3 ?Button1 ?Button2 ?Button1 Water’ = 4 cl/sec. Coffee’ = Water’ Water’ = 3 cl/sec. Coffee’ = Water’ Water’ = 0 Coffee’ = 0 Water’ = 0 Coffee’ = 0 Water

17. 17 Content Refresh –Model-based Testing –Discrete Conformance & ioco Tests –Timed Conformance & tioco Tests Hybrid Systems Hybrid Input-output Conformance Hybrid Input-output Conformance Tests Conclusions –Results –Limitations and Future Work

18. 18 The Idea Besides for the output actions, also for every reachable state the set of trajectories allowed by the implementation is a subset of the set of trajectories allowed by the specification No quiescence because continuous output always occurs

19. 19 Hybrid Conformance (continuous output only) Impl. is input-output conform a Spec. iff for all traces α: out(Impl. after α)  out(Spec. after α) and traj(Impl. after α)  traj(Spec. after α) s0s1 s2 s3 ?Button1 ?Button2 ?Button1 Coffee’ = 4 Coffee’ = 3 Coffee’ = 0

20. 20 Hybrid Conformance Issues The continuous output directly depends on the continuous input, therefore we have to define our relation on both input and output trajectories. The implementation is input enabled (for both discrete behavior and continuous behavior), but we do not want the specification to be input complete.

21. 21 Hybrid Conformance (continuous input plus output) s0s1 s2 s3 ?Button1 ?Button2 ?Button1 Water’ = 4 Coffee’ = Water’ Water’ = 3 Coffee’ = Water’ Water’ = 0 Coffee’ = 0 Water’ = 0 Coffee’ = 0 infilter(traj(Impl. after α), traj(Spec. after α)) is the set of trajectories allowed by the implementation, with respect to the input trajectories allowed by the specification Water’ = 3 Coffee’ = 3 Water’ = 2 Coffee’ = 2

22. 22 Hybrid Input-output Conformance Impl. hioco Spec. iff for all traces α: out(Impl. after α)  out(Spec. after α) and infilter(traj(Impl. after α), traj(Spec. after α))  traj(Spec. after α)

23. 23 Hybrid Conformance (continuous input plus output) s0s1 s2 s3 ?Button1 ?Button2 ?Button1 Water’ = 4 Coffee’ = Water’ Water’ = 3 Coffee’ = Water’ Water’ = 0 Coffee’ = 0 Water’ = 0 Coffee’ = 0 s4s5 !”NoWater” Water’ =0 Coffee’ = Water’ Water’ = 0 Coffee’ = Water’ Coffee’ = 0 Water = 0 out(Impl. after α)  out(Spec. after α) and infilter(traj(Impl. after α), traj(Spec. after α))  traj(Spec. after α)  {if the implementation cannot do output action “!NoWater”}   {“NoWater”} and    In state s4:

24. 24 Hybrid Conformance (continuous input plus output) If there is a trajectory from state s then out(s) = {o in O| s →}  {ξ} else out(s) = {o in O| s →} o o out(Impl. after α)  out(Spec. after α) and infilter(traj(Impl. after α), traj(Spec. after α))  traj(Spec. after α)  {if the implementation cannot do output action !”noWater”} {ξ}  {“NoWater”} and    In state s4:

25. 25 Hybrid Input-output Conformance Impl. hioco Spec. iff for all traces α: out(Impl. after α)  out(Spec. after α) and infilter(traj(Impl. after α), traj(Spec. after α))  traj(Spec. after α)

26. 26 Content Refresh –Model-based Testing –Discrete Conformance & ioco Tests –Timed Conformance & tioco Tests Hybrid Systems Hybrid Input-output Conformance Hybrid Input-output Conformance Tests Conclusions –Results –Limitations and Future Work

27. 27 Hybrid Test-case Tree like structure Two terminal states: pass and fail Deterministic for actions Strongly time deterministic for trajectories

28. 28 Hybrid Test-case Generation 1.Terminate with verdict pass 2.Select an input from the specification and apply it to the implementation s0s1 ?Button1 Water’ = 0 Coffee’ = 0 Water’ = 0 Coffee’ = 0 Specification t0 t1 ?Button1 Test-Case

29. 29 Hybrid Test-case Generation 3.Select an input trajectory from the specification, apply it to the implementation and observe the output trajectory simultaneous, possibly interrupted by an output action. Specification s1 Water’ = 0 Coffee’ = 0 Test-Case t1 t2 Water’ = 0 Coffee’ = 0 Fail !”Boilerlow”

30. 30 Hybrid Test-case Generation 4.If an output action has to happen immediately according to the specification then observe an output action and check if it is allowed according to the specification or let time pass by selecting and applying and observing a trajectory fail s4s5 !”NoWater” Coffee’ = 0 Water = 0 Specification t4 Pass !”NoWater” Test-Case

31. 31 Example Test-case Generation t0 t1 ?Button1 t3 ?Button2 Fail Water’ = 0 Coffee’ = Water’ t4 t2 Fail Water’ = 0 Coffee’ = 0 !”Boilerlow” Fail Pass !”NoWater” fail

32. 32 Content Refresh –Model-based Testing –Discrete Conformance & ioco Tests –Timed Conformance & tioco Tests Hybrid Systems Hybrid Input-output Conformance Hybrid Input-output Conformance Tests Conclusions –Results –Limitations and Future Work

33. 33 Results A hybrid conformance theory Sound and exhaustive Natural extensions to other theories Still, open issues …

34. 34 Limitations and Future Work Continuous input depending on continuous output Uncountable many states and trajectories Taking into Practice –Sampling –Test-selection –Inaccuracy / robustness –…

35. 35 Questions?