1. SEEFM 07, Thessaloniki, Nov 20071/ Teaching the construction of correct programs using invariant based programming Ralph-Johan Back Johannes Eriksson Linda Mannila Åbo Akademi / Dept. of Information Technologies Turku, Finland

2. SEEFM 07, Thessaloniki, Nov 20072/ Formal methods in CS education  Formal methods are perceived as difficult and requiring mathematical sophistication  The CS curriculum is divided into “theory” and “practice”  Formal methods taught independently of programming courses  Students get impression that formal methods are not applicable in practice  Testing and debugging is therefore the main (only) programming method that they learn from CS studies

3. SEEFM 07, Thessaloniki, Nov 20073/ Overview of talk  A short introduction to invariant based programming  The Socos tool  Teaching formal methods at Åbo Akademi using invariant based programming  Experience report on a first year course on invariant based programming

4. SEEFM 07, Thessaloniki, Nov 20074/ Constructing correct programs Program code Contracts Invariants Verification conditions “a posteriori verification”“constructive approach”“invariant based programming”

5. SEEFM 07, Thessaloniki, Nov 20075/ Example: Sort an array! A=A0 A: Int[N] Sorted(A,0,N) A: Int[N] Permutation(A,A0) Start with a pre-/postcondition specification

6. SEEFM 07, Thessaloniki, Nov 20076/ Example: Sort an array! A=A0 Sorted(A,0,N) A: Int[N] Permutation(A,A0) Structure according to invariants

7. SEEFM 07, Thessaloniki, Nov 20077/ Construct a loop Example: Sort an array! A=A0Sorted(A,0,N) A: Int[N] k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] Permutation(A,A0) 0kN sorted un- sorted less than or equal to all A[k..N-1] ! LOOP

8. SEEFM 07, Thessaloniki, Nov 20078/ Add initial transition Example: Sort an array! A=A0Sorted(A,0,N) A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0 A: Int[N] ⇒ 0: Int 0≤0≤N Sorted(A,0,0) ∀i,j:Int 0≤i<0 ∧ 0≤j<N ⇒ A[i]≤A[j] A: Int[N] Permutation(A,A0) ✔ A=A0 ✔ ✔ ✔ ✔ ✔ What needs to be checked?

9. SEEFM 07, Thessaloniki, Nov 20079/ Example: Sort an array! A=A0Sorted(A,0,N) A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0 [k=N] Add exit transition Trivial: Sorted(A,0,k) ∧ k=N ⇒ Sorted(A,0,N)

10. SEEFM 07, Thessaloniki, Nov 200710/ Example: Sort an array! A=A0Sorted(A,0,N) A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0 [k=N] [k<N] m:=min(A,k,N); A:=A[ k←A[m], m←A[k] ]; k:=k+1 Add loop transition A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] A’: Int[N] Permutation(A’,A0) k+1: Int 0≤k+1≤N Sorted(A’,0,k+1) ∀i,j:Int 0≤i<k+1 ∧ k+1≤j<N ⇒ A’[i]≤A’[j] k<N m=min(A,k,N) ∧ A’= A[ k←A[m], m←A[k] ] ⇒

11. SEEFM 07, Thessaloniki, Nov 200711/ Example: Sort an Array! A=A0Sorted(A,0,N) A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0 [k=N] [k<N] A:=Swap(A,k,min(A,k,N)); k:=k+1 0≤N-k Add a termination function Variant decreases: N-(k+1) < N-k Bounded from below: 0≤k≤N ⇒ 0≤N-k

12. SEEFM 07, Thessaloniki, Nov 200712/ Socos: a prototype environment for: teaching formal methods state-of-the-art automatic and interactive verification invariant based programming

13. SEEFM 07, Thessaloniki, Nov 200713/ Invariant Diagrams in SOCOS

14. SEEFM 07, Thessaloniki, Nov 200714/ Verification in SOCOS  Three types of verification conditions:  Consistency (for transitions)  Completeness (for situations)  Termination (for loops)  Verification conditions are sent to external proof tools  Simplify (automatic proofs),  PVS (interactive proof checking)  Prover daemon works in the background  Monitors changed files  (Re)generates verification conditions and reruns proofs

15. SEEFM 07, Thessaloniki, Nov 200715/ Backends Testing Diagram is converted to a Python program, with run-time evaluation of invariants Testing Diagram is converted to a Python program, with run-time evaluation of invariants Static Checking Verification conditions are sent to Simplify, a fully automatic prover Static Checking Verification conditions are sent to Simplify, a fully automatic prover Full Verification PVS is used for full verification of the final components Full Verification PVS is used for full verification of the final components Higher assurance→

16. SEEFM 07, Thessaloniki, Nov 200716/ Teaching invariant based programming at Abo Akademi  Spring 2005: Ph.D. course on invariant based programming -- testing out the idea  Spring 2007: a course on IBP for first year students  2008 -- : IBP now part of standard CS curriculum  Planned next step: teaching IBP as a special math course in high school

17. SEEFM 07, Thessaloniki, Nov 200717/ New first year CS curriculum  Mathematics courses  algebra  probability theory  Computer Science courses  Introduction to CS  Python programming (to show that programming is fun)  Structured derivations (to teach mathematical and logical reasoning)  Invariant based programming (to teach how to construct programs that are correct)  Java programming  Systems design course Formal methods bundle

18. SEEFM 07, Thessaloniki, Nov 200718/ Invariant based programming course (spring 2007)  aimed at first or second year students  interactive, emphasizing student participation  17 sessions a 90 min  11 lectures  6 practical excercises  Socos tool support only used in 4 last sessions  only automatic proofs (Simplify), no PVS proofs  16 active participants  half with no background in formal methods

19. SEEFM 07, Thessaloniki, Nov 200719/ Collecting data about the course  pre- and postcourse questionairs  observations  hand-in assignments  final exam  8 students selected for semi-structured interview

20. SEEFM 07, Thessaloniki, Nov 200720/ Main results - 1  The students found the course  useful,  interesting,  somewhat fun and  of medium difficulty level.  On average, students found invariant based programming  rather easy to learn,  useful in practice and  made the general structure of the program more comprehensible

21. SEEFM 07, Thessaloniki, Nov 200721/ Main results - 2  Difficulties were mainly in  constructing proofs and  finding the invariant for more complex programs  The programs written by the students show that  they had understood the idea behind IBP, and  were able to construct and prove simple invariant based programs.

22. SEEFM 07, Thessaloniki, Nov 200722/ Main results - 3  The students appreciated the diagrammatic notation of IBP  most students are visual learners,  textual programming languages or pseudocode may not be the best way for expressing algorithms to these students  We had expected that identifying the invariants would be the most difficult task, but this was not the case  writing proofs by hand seemed to be most problematic, as they required much time and effort  formulating postconditions was also sometimes problematic  Students found that IBP provides good support for finding bugs during the program construction  instead of after the program is ready

23. SEEFM 07, Thessaloniki, Nov 200723/ Main results - 4  Starting with informal reasoning in the course before introducing the formal framework was not appreciated  the students would have wanted the formal proof obligations to be introduced earlier  it seems that students who are not mathematically mature do not know how to reason ”informally” but first need to learn a formal approach with a fixed set of rules  Socos  supporting a formal method with a computer based tool in the course was very well received  the students preferred SOCOS over pen and paper, as the automation increased productivity.  unfamiliarity with the SOCOS syntax was the main cause of difficulty

24. SEEFM 07, Thessaloniki, Nov 200724/ Thank You http://mde.abo.fi/SOCOS