1. ESP: A Language for Programmable Devices Sanjeev Kumar Princeton University Advisor : Kai Li

2. Sanjeev KumarESP: A Language for Programmable Devices2 Programmable Devices Network Card Bus CPU D Mem R Disk CPU D D D Mem R Main CPUMain Memory Move functionality from main CPUs to devices

3. Sanjeev KumarESP: A Language for Programmable Devices3 Programming Devices Challenging –Concurrent, low-overhead, and reliable Event-driven state machines (vs. Threads) –1 stack is shared by all state-machines –Fast context switches Only need to save and restore the Program Counter –Low overhead: memory & CPU –Less functionality provided

4. Sanjeev KumarESP: A Language for Programmable Devices4 Programming Devices in C WaitReq WaitNet WaitDMA State machines communicating using events A state machine event DmaFree execute event UserReq execute processUser() event KernelReq execute processKernel() event NetFree execute

5. Sanjeev KumarESP: A Language for Programmable Devices5 Programming Devices in C Cont’d Programming is hard –Sequential code split into handlers Difficult for C compiler to optimize –Explicitly save values in global variables Hand optimizations make it worse –Fast paths (violate modularity) –Embed ad-hoc scheduling decisions int i = 5; block(); j = i;

6. Sanjeev KumarESP: A Language for Programmable Devices6 Case Study: VMMC High-performance communication –Bypass OS for data transfers Used Myrinet network cards –Gigabit network –33 MHz CPU, 1 MB memory VMMC firmware –Event-driven state machines in C Data OS Network Card Application Network

7. Sanjeev KumarESP: A Language for Programmable Devices7 Our Experience with Programming VMMC Firmware Event-driven state machines in C –Hard to program VMMC: 15,600 lines of C code Add new functionality –Hard to debug Complicated race conditions (still encounter bugs) Trusted by the OS –Achieves good performance Requires hand optimization

8. Sanjeev KumarESP: A Language for Programmable Devices8 Outline Motivation ESP overview ESP language Verifying ESP programs Performance of ESP programs Conclusions

9. Sanjeev KumarESP: A Language for Programmable Devices9 ESP: Event-driven State-machines Programming Domain-specific language Goals –Easy to program Concise, modular programs –Allow extensive testing Use existing model-checking verifiers like Spin –Performance Low overhead Case study: VMMC firmware

10. Sanjeev KumarESP: A Language for Programmable Devices10 ESP Approach pgm1.spin pgmN.spin pgm.C ESP Compiler pgm.ESP help.C Generate Firmware Develop and Test using Verifier test1.spin testN.spin Spin Models Detailed Model Memory Safety Model Abstract Models

11. Sanjeev KumarESP: A Language for Programmable Devices11 Related Work Verification + Code generation –Esterel: Control of reactive systems –Teapot: Coherence protocols for shared memory –Promela++: Layered network protocols –@-format: C extension for state machines Concurrent languages –CSP, Squeak, OCCAM, Java, CML, Devil Debugging Tools – Meta-level compilation, SLAM, Verisoft

12. Sanjeev KumarESP: A Language for Programmable Devices12 Outline Motivation ESP overview ESP language Verifying ESP programs Performance of ESP programs Conclusions

13. Sanjeev KumarESP: A Language for Programmable Devices13 The ESP Language Concurrent language Processes & Channels –in : Receive a message on a channel –out : Send a message on a channel –alt : Wait on multiple in/out operations Channels are synchronous or unbuffered Processes and channels are static

14. Sanjeev KumarESP: A Language for Programmable Devices14 A Process Encodes a State Machine channel chan1: int channel chan2: int process add5 { $v: int = 0; while( true) { in( chan1, v); out( chan2, v+5); } States are implicit Events are messages on channels Easy to read & optimize –Control flow is obvious Initial waitChan2 waitChan1

15. Sanjeev KumarESP: A Language for Programmable Devices15 Data Types and Control Constructs Data types –Simple types: int, bool –Complex data types: records, unions, arrays –No recursive data types Control constructs –if-then-else, while, etc. –No functions; Use processes instead

16. Sanjeev KumarESP: A Language for Programmable Devices16 Channels for Communication Processes communicate only using channels –Pure message passing communications To enforce this –No shared mutable data structure No global variables Only immutable objects on channels Immutable mutable objects

17. Sanjeev KumarESP: A Language for Programmable Devices17 Memory Management Explicit –malloc/free –Fast –Unsafe Automatic –Garbage collection –More CPU & memory –Safe Concurrent programSafe Limited CPU & memoryFast

18. Sanjeev KumarESP: A Language for Programmable Devices18 Memory Management Cont’d ESP supports explicit management Explicit management is difficult –Global property of program ESP makes this a local property –Objects are not shared by processes –Objects sent over channels are “copied” Use verifiers to check correctness –Check each process separately ESP supports safety through verification

19. Sanjeev KumarESP: A Language for Programmable Devices19 Support for Dispatch channel pktc: union of { ping: int, status: bool } process A {...; out( pktc, v);...} process B {...; in( pktc, { ping |> $node});...} process C {...; in( pktc, { status |> $flag});...} Pattern-matching supports dispatch

20. Sanjeev KumarESP: A Language for Programmable Devices20 External Interface C interface: volatile memory, device registers Spin interface: specify properties to be verified Traditional approach: function interface ESP uses channels –Some channels have external reader or writer –Unified mechanism (C and Spin) –Use in/out to block on external events

21. Sanjeev KumarESP: A Language for Programmable Devices21 Case Study: VMMC Firmware Implemented VMMC using ESP –8 processes, 19 channels –500 lines ESP + 3000 lines C code Modular programs that are easy to maintain Order of magnitude less code

22. Sanjeev KumarESP: A Language for Programmable Devices22 Outline Motivation ESP overview ESP language Verifying ESP programs Performance of ESP programs Conclusions

23. Sanjeev KumarESP: A Language for Programmable Devices23 Using Model-Checking Verifiers State-space exploration –Try all possible scheduling options Advantages –Automatic –Produces counter example Disadvantages –Computationally expensive (exponential) ESP currently uses Spin model checker

24. Sanjeev KumarESP: A Language for Programmable Devices24 Spin Model-Checking Verifier Designed for software systems –Supports processes and synchronous channels Specify properties to be verified –Assertions, deadlocks, Linear Temporal Logic 3 levels of checking with varying coverage –Exhaustive –Partial –Simulation

25. Sanjeev KumarESP: A Language for Programmable Devices25 ESP Approach Models extracted automatically –Reduces programmer effort –Avoids mismatch Debugged using verifier Test files can be reused pgm1.spin pgmN.spin pgm.C ESP Compiler pgm.ESP help.C Generate Firmware Develop and Test using Verifier test1.spin testN.spin

26. Sanjeev KumarESP: A Language for Programmable Devices26 Extracting Spin Models Detailed models Memory-safety models –Detailed model + additional checks Abstract models –Necessary for larger subsystems –Drop unnecessary details Depending on the property being verified Abstraction specified by the programmer Compiler makes safe conservative approximation

27. Sanjeev KumarESP: A Language for Programmable Devices27 Conservative Approximations in Abstract Models $b2: boolean = true;... $b1: boolean = b2; type recT = #record of { int count; } $r1: recT = {0}; if (b) { r2 = r1; }... r1.count = 5; if :: b1 = true :: b1 = false fi if :: r2.count = 5 :: skip fi X X X X X

28. Sanjeev KumarESP: A Language for Programmable Devices28 Case Study: VMMC Firmware Develop and debug retransmission protocol –Easier to debug than on the network card –Took 2 days (10 days in earlier implementation) Check for memory safety and leaks –Found deliberately introduced bugs, an earlier bug Check for deadlocks –Hard-to-find bugs –Found 7 bugs using abstract models

29. Sanjeev KumarESP: A Language for Programmable Devices29 Case Study: VMMC Firmware Cont’d ProcessProperty Time (in seconds) Memory (in Mbytes) Complete search? remoteReplySafety2.330.55Yes localReqSafety0.125.30Yes reliableSendSafety67.634.45Yes AllDeadlock84.0268.35No AllDeadlock14250.0167.92No* * Using partial mode

30. Sanjeev KumarESP: A Language for Programmable Devices30 Outline Motivation ESP overview ESP language Verifying ESP programs Performance of ESP programs Conclusions

31. Sanjeev KumarESP: A Language for Programmable Devices31 ESP Code Generation Compiling for sequential execution –Combine them [Berry et al., Proebsting et al.] No context switching overhead Worst-case exponential increase in executable size –Low-overhead process management Small context switching overhead –Only the program counter needs to be saved Small executable

32. Sanjeev KumarESP: A Language for Programmable Devices32 ESP Code Generation Cont’d C as back-end language –Generates one large C function –Links with other functions provided by the programmer Compiler optimizations –Whole program analysis Avoid indirect jumps, unnecessary allocation –Per process Constant folding, copy propagation

33. Sanjeev KumarESP: A Language for Programmable Devices33 Case Study: VMMC Firmware Measure overhead of using ESP –Microbenchmarks –Applications Compare performance –Earlier implementation (vmmcOrig) –Earlier implementation without fast paths (vmmcOrigNoFastPaths) –New implementation using ESP (vmmcESP)

34. Sanjeev KumarESP: A Language for Programmable Devices34 Microbenchmarks Impact of fast paths –4 Bytes: 100% –4 Kbyte: 38% Impact of using ESP –64 Bytes: 35% –4 Kbytes: 0% Message size (in Bytes) ss Latency

35. Sanjeev KumarESP: A Language for Programmable Devices35 Applications SPLASH2 applications on a 16 (4 X 4) node cluster ESP: 3.5 % on average Speedup Fast paths: < 1% on average

36. Sanjeev KumarESP: A Language for Programmable Devices36 Performance Summary Significant difference in microbenchmarks –Most due to the brittle fast paths –Required the programmer to manually optimize Small impact on applications –Applications are less sensitive Microbenchmarks represent worst case New functionality can help a lot more –40% for SVM applications [ Bilas et al. ]

37. Sanjeev KumarESP: A Language for Programmable Devices37 Outline Motivation ESP overview ESP language Verifying ESP programs Performance of ESP programs Conclusions

38. Sanjeev KumarESP: A Language for Programmable Devices38 Conclusions ESP: Domain-specific language –Supports concise, modular programs Order of magnitude less code –Extensive testing with Spin Develop code Check for memory safety & absence of deadlocks Effective but has limitations –Low performance overhead 3.5% for applications Could use further optimizations

39. Sanjeev KumarESP: A Language for Programmable Devices39 Future Directions More experience with ESP –Other devices e.g. Intel IXP –Other protocols e.g. TCP/IP Compiler optimizations –Selective process inlining –Support for fast paths Using model checking more effectively –Partial searches

40. Sanjeev KumarESP: A Language for Programmable Devices40 Acknowledgements Kai Li Andrew Appel Edward Felten Randy Wang Larry Peterson Doug Clark Jaswinder Pal Singh Steve Dirk Rudro Patrick Yuanyuan Tammo George Angelos Stef Liviu Others Melissa and the rest of administrative staff Technical staff

41. Sanjeev KumarESP: A Language for Programmable Devices41 - The End -

42. Sanjeev KumarESP: A Language for Programmable Devices42 Esterel Synchronous programming language –Encode control of reactive systems Generate software or hardware Deterministic reaction But –Only encodes control flow –Constraints on the language –Harder to compile efficiently

43. Sanjeev KumarESP: A Language for Programmable Devices43 Teapot Domain-specific language –Implement coherence protocols Specify a state machine Use continuations to simplify programming But –Only encodes control flow –Uses handlers: Code fragmentation –Similar approach would be less modular

44. Sanjeev KumarESP: A Language for Programmable Devices44 Promela++ Language for Layered Network Protocol –Non-strict extension of promela Asynchronous communication But –Layered structure –Asynchronous: overhead, ordering –No support for memory management

45. Sanjeev KumarESP: A Language for Programmable Devices45 Bandwidth Microbenchmarks M B / s Message Size (in Bytes) Bidirectional Bandwidth M B / s Message Size (in Bytes) One-way Bandwidth

46. Sanjeev KumarESP: A Language for Programmable Devices46 Fast Paths Challenges –Identify fast paths in a concurrent program –Fast path extraction –Robust fast paths