1. Safety as a Software Metric Matthias Felleisen and Robert Corky Cartwright Rice University

2. Why Safety as a Metric? Measuring Software: Syntax versus Semantics What is Programming Language Safety ? What Makes an Individual Program Safe ? How about Teaching Program Safety?

3. Why Measure Software? correct and efficient software maintainable software extensible software

4. What do Metrics Measure? lines of code number of procedures, gotos, loops, modules, statements versus expressions, … in short: Syntactic Attributes of software

5. What should Metrics Measure? correctness extensibility maintainability in short: semantic and organizational attributes

6. Measuring Correctness is Difficult … goal: measure certain aspects of correctness specifically: assume the programming language is safe, what kind of problems can we predict?

7. Safe Programming Languages

8. Safety -- A High-Level View (1) “Close the valve by 10 degrees!” “Turned the valve by 10 degrees!”

9. Safety -- A High-Level View (2) “Close the valve by 10 degrees!” “Turned the valve by 15 degrees!”

10. Safety -- A High-Level View (3) “Close the valve by 10 degrees!” “OUCH!”

11. Safety -- A High-Level View (4) ------------- ------- ------------- --------- ----------

12. Safety -- A High-Level View (5) ------------- ------- ------------- --------- ---------- ERROR!

13. C and C++ are NOT Safe! int f(int n, int m) { int r = n % m; if (0 == r) return m; else return f(m,r); } main() { char a = 'a'; char b = 'b'; int mn[2] = {24,6}; char c = 'c'; char d = 'd'; printf("%d\n",f(mn[0],mn[1])); printf("%d\n",f(mn[0],c)); printf("%d\n",f(mn[0],mn[2])); }

14. Safety in Programming Languages a safe language protects every computational primitive, e.g., +, *, if, vector-lookup, record dereference, … protection is implemented with a mixture of compile- time and run-time checks safety guarantees errors are caught safety greatly increases effectiveness of debugging

15. Safety … is NOT just TYPE checking!

16. Examples Fortran C C++ Perl ML Eiffel Java Scheme (untyped, but safe) UNSAFE Languages SAFE Languages

17. Safe Programs and Measuring Safety

18. Measuring the Safety of Programs programs in safe languages signal errors programs should not signal errors determine whether any computational primitive might signal an error make programmers explain potential faults

19. MrSpidey: Measuring the Safety of Scheme Programs Scheme is a dialect of Algol and LISP lexical scope, first-class functions (“mini-objects”) LISP’s syntax (parentheses) and primitives (cons, car, and cdr)

20. some function call, somewhere in the program

21. SYMBOLS are bad for +

22. general input shapes

23. Measuring Safety is More than Checking Types check general “data shapes” lists with at least N items vector references …

24. list with at least one NUMBER

25. NIL is not okay

26. An Elaborate Example from the Scheme Front-end S-expression (let ( ) ) (( lambda ( ) ) )

27. weak invariant … yields many checks

28. stronger invariant yields stronger results

29. Teaching with Safety Metrics

30. Program Construction: Rice University, Fall 1998 course on program safety understanding measuring based on Scheme and Java

31. On Safety of Languages and Programs programming language safety program safety theory and tools for “measuring” program safety –logics that conservatively approximate semantics –logics that extend the logic of type checking

32. The Pragmatics of MrSpidey using MrSpidey: –checking –understanding potential fault sites: data set data flow –is it a problem with the program? –is it a problem with the theory/tool? –if the latter, can a re-organization help?

33. Hands-on Work homework assignments –sets of problems for each bullet –increasing complexity –theory and practice project: implement sequential subset of Java –modules and data invariants that cross boundaries –exploring large pieces of code

34. Evaluation (1) course evaluation: excellent targeted questions: –understanding of language safety –understanding of program safety –understanding of measuring safety with theorem provers –effectiveness of homeworks versus project

35. Evaluation (2) Positives: –appreciate safety –appreciate tools –appreciate theory –understand the above based on homework Negatives –project too large

36. Summary new, semantics-based thinking about “metrics” extensions: measuring stronger invariants (numeric constraints, polyvariant); measuring organization (patterns?) teaching: a good approach to have students understand partial correctness

37. Thank You Matthew Flatt Shriram Krishnamurthi Robby Findler Mike Fagan (92) Andrew Wright (94) Cormac Flanagan (96)