1. CSCI1600: Embedded and Real Time Software Lecture 28: Verification I Steven Reiss, Fall 2015

2. Requirements  Requirements are central to embedded systems  Failure might now be an option  Types of Requirements  Safety: the car won’t crash  Timing: the solenoid will go on within 10ms of bumper switch  Fairness: as long as the heating system is on, the room will eventually be comfortable

3. Requirements: Goal  Show the system meets its requirements  That the system is safe  That the system works correctly  Have confidence in the system’s safety  Show the system meets its requirements  Under all possible circumstances

4. Helpful Techniques: SE  Good requirements analysis  Understanding all the possible problems and solutions  Good specifications  Accurate modeling  Showing the models are correct  Petri net and FSA validation  Design for security  Defensive coding  Sanity checks

5. Helpful Techniques: Monitors  Add monitor tasks to the code  These check that the system is operating correctly  Detect if something is going wrong  Too many solenoids turned on  Train moving onto a block containing another train  Lights have been on for > 10 ms  If something is detected (unusual or wrong)  Move the system to a “safe” state

6. Helpful Techniques: Validation  Testing  Use lots of test cases  Simulate the hardware and ensure system runs  Simulate potential failures and see what the system does  Multiple simultaneous failures as well as single ones  Run time assertion checking  Preconditions and postconditions  Defensive coding  Sanity checking  But what do these tell you  How sure can you be?

7. Verification: Basic Idea  Mathematically prove the program is correct  Show the program works for all possible runs  Not just the ones covered by the test cases  All possible (and impossible) inputs  Show the timing works for all possible schedules

8. Verification: Issues  What does it mean to be correct  If we are going to prove something, it has to be precise  How can you state precisely what the program should do?  Especially for a complex system  How can you model all possible runs?  There are an infinite number of these (nonterminating)  Or a very large finite number

9. Verification: Issues  Programs are very complex  Proving anything about a program is undoable  Almost anything you want to prove is undecidable  Halting problem  Actual not true since machines are finite, but effectively true  Even restricting values to finite ranges, etc.  Can involve huge numbers of program states and value sets

10. Verification: Issues  Proofs are very complex  The mathematics involved is complex  Would you believe a 1000 page proof of a program  Suppose it is done by a computer  How long is the program doing the proof  Has it been shown to be correct  What about the compiler and the hardware?  Yet the work has to be done  And techniques have been developed to do it

11. Verification: Basic Idea  Steps  Model the program in a checkable representation  Prove properties of the program in that representation  Prove that the program satisfies the model  We’ve seen approximations before  Petri nets: only consider elements affecting synchronization and task coordination  Queueing theory: only consider tasks and queues Program Model Proof in Model Proof in Program

12. Modeling the Program  Goal: Map the program into a finite state automata  Why finite state automata?  It already is, but the number of states is very large  Reduce the number of states  Restrict variable values to reduce the number of states  Restrict the number of threads, tasks, etc.  Show that the restriction doesn’t affect proof

13. Proving the Model is Correct  Provide a mapping from the code to the model  Automatically generate the model from the code  Prove the code implements the model  Show that other aspects of the program don’t matter  Consider all possible interpretations of those properties

14. Model Checking Program Model Proof in Model Proof in Program

15. Model Checking: Properties  First we need to be able to state the property  In a form that is understandable  In a form that can be checked  Represent the property to be proved in finite form  As a finite automata  As properties over a finite automata  States in a finite automata are labeled with propositions  Then you want to state properties over those propositions  Taking time into account

16. Model Checking: Program  Model the program as a generalized finite automata  Need to generalize to handle infinite programs  Automatically map code into model  To ensure the mapping is correct  With some help from the programmer  For example, providing set of legal values for an int  The mapping has to be conservative  Cover all possible executions  Might allow “impossible” executions as well

17. Model Checking: Proofs  Show that all executions of the finite program  Satisfy the property  Can be done in terms of executions  Can be done in terms of languages (symbolic math)

18. Defining Properties  Modularize correctness  Rather than trying to show the whole program is correct  Break correctness down into smaller pieces  Define correctness in terms of specific program properties  Prove each of these separately  This is often easier than trying to show everything at once

19. Example: HVAC System  My heating system  3 floors  Basement and 2 nd floor share common water unit  Basement is radiant floor heating, 2 nd floor is water->air  These have separate thermostats  Want to ensure that not both heating and cooling the water  How might you express this?

20. Example: HVAC System  Think about the HVAC controller as a FSA  Transitions between states determined by events  Including timer events and random events (outside temp)  We can define different properties of interest  Heating On (H), Air Conditioning On (AC)  Build an FSA with states based on these properties ~H, ~AC H, ~AC ~H, AC H, AC

21. Example  How would you model the program to check this  Look only at elements that affect turning on heat and air conditioning and their interaction  Assume the rest of the system can do whatever it wants  Arbitrary setting of thermostats, temperatures, etc.  Based on what program actually does

22. Example: HVAC System  Other Examples  Show the heat is on if the room is too cool  Show that air conditioning is on if the room is too hot  Show neither is on if the room is comfortable  Show that air conditioning doesn’t come on within 5 minutes of its last being on  Show temperature will be near comfortable levels at a designated time  Are these going to be true?  What would you really show

23. Example: Your Systems  How would you express your requirements

24. Example: Pinball  Show that you don’t blow fuses  No more than 2 solenoids on at once  No more than 8 lights on at once  Show that each element scores correctly  Show ball logic is correct (number, free balls, …)

25. Next Time  We’ll look at formalizing this process  And how it can be made practical