1. 6/12/20011 KaffeOS: Isolation, Resource Management and Sharing in Java Godmar Back School of Computing University of Utah Dissertation Defense

2. Dissertation Defense 6/12/2001 2 Motivation l Java is widely used –Examples: applets, servlets, Enterprise Java Beans , mobile agents, active packets, database procedures –Untrusted code: possibly malicious or buggy –Multiple applications on behalf of multiple users l Primary motivation  Want to provide robust environment for these applications l Secondary motivation –Make efficient use of resources; increase scalability –Run on “small” systems (handhelds, embedded systems)

3. Dissertation Defense 6/12/2001 3 Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

4. Dissertation Defense 6/12/2001 4 Java Applications Applet / Agent / Servlet / Database Procedure … JVM Base OS This work is about system structure.

5. Dissertation Defense 6/12/2001 5 Current Options - Single JVM with ad-hoc layer -Efficient sharing App 1 App 2 App 3 Ad hoc layer Base OS JVM App 1 App 2 App 3 JVM Base OS - Multiple JVMs -Strong isolation -Good resource management

6. Dissertation Defense 6/12/2001 6 Java Operating System App 1 App 2 App 3 App 4 Java OS Base OS + Good isolation + Good resource management + Efficient sharing  requires process model

7. Dissertation Defense 6/12/2001 7 Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

8. Dissertation Defense 6/12/2001 8 KaffeOS Core Ideas To provideKaffeOS uses Isolation and Safe Termination Red Line Memory ManagementSeparate Heaps Interprocess Communication Shared Heaps Flexible Resource Management Hierarchical Resource Framework

9. Dissertation Defense 6/12/2001 9 The Red Line in Hardware-based OS l MMU prevents access to kernel data structures in user mode l System calls enter kernel mode Cheriton 1994

10. Dissertation Defense 6/12/2001 10 The Red Line for Safe Termination l Isolation of applications and system: user/kernel boundary –Kernel classes must not be corrupted –Termination is deferred while inside kernel code –All exceptional situations are handled in kernel code NB: different from built-in synchronize

11. Dissertation Defense 6/12/2001 11 class Queue { synchronized Object get(); synchronized void put(Object); } Queue producer, consumer; void run () { while (!done) { Object obj = producer.get(); consumer.put(obj); } l Producer/Consumer Example –Invariant: all produced items are consumed Guaranteeing Consistency Kernel.enter(); Kernel.leave();

12. Dissertation Defense 6/12/2001 12 Red Line in KaffeOS User/kernel dimension is different from trust dimension: User GC System GC User code (untrusted) Kernel code (trusted) Runtime Libs (trusted) Finalization User Kernel User modeKernel mode TrustedRuntime codeKernel code UntrustedUser code

13. Dissertation Defense 6/12/2001 13 Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

14. Dissertation Defense 6/12/2001 14 1.Limit memory use of processes 2.Fully reclaim processes’ memory after termination 3.Garbage collect processes individually Memory Management: Goals

15. Dissertation Defense 6/12/2001 15 Single Heap (Conventional JVM) Reference Object

16. Dissertation Defense 6/12/2001 16 Separate Heaps Kernel Heap User Heaps Goal 1: Limit memory use

17. Dissertation Defense 6/12/2001 17 Reclaiming Memory l References can make objects unreclaimable Hardware-based OS MMU maps virtual memory OS manages physical pages Java runtime system No MMU Garbage Collector frees unreachable objects

18. Dissertation Defense 6/12/2001 18 Legal and Illegal References Legal reference Illegal reference Goal 2: Full reclamation

19. Dissertation Defense 6/12/2001 19 Preventing Illegal References l Use GC write barriers –Technique used in incremental and generational collection l Intercept each write, check heaps –Throw SegmentationViolationError if illegal –Only PUTFIELD, PUTSTATIC, and AASTORE bytecode instructions have to be checked l Small cost on legal write (common case)

20. Dissertation Defense 6/12/2001 20 Use of Entry and Exit Items Entry Item n nn x x x x Exit ItemRefCount 1 12 Goal 3: Separate Collection

21. Dissertation Defense 6/12/2001 21 Collecting Cycles l Merge heaps upon termination into kernel heap

22. Dissertation Defense 6/12/2001 22 Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

23. Dissertation Defense 6/12/2001 23 IPC: Direct Sharing l Alternatives: –No sharing (copy between processes) –Indirect sharing (use proxy objects or surrogates) l Objects contain direct pointers to shared objects –Preserve Java model l Should not compromise accurate accounting or full reclamation!

24. Dissertation Defense 6/12/2001 24 Shared Heaps Legal reference Illegal reference User Heap Kernel Heap Shared Heap User Heap

25. Dissertation Defense 6/12/2001 25 Shared Heaps: Model l Accounting of shared objects –Shared heap’s size is fixed after creation –Sharers are all charged for shared heaps: double charging l Reclamation –As soon as garbage collector detects that nothing on shared heap is referenced l Restrictions on programming model –Cannot allocate new objects after heap is frozen –Cannot have references to user heaps

26. Dissertation Defense 6/12/2001 26 Shared Heaps: Trie Example package shared; class Trie { static class TrieNode { final static int WIDTH = 96; TrieNode [] children = new TrieNode[WIDTH]; boolean eow = false; } private TrieNode root; private int maxlength; Trie(Reader r) throws IOException { … } public void dump(PrintStream s) { … } public boolean lookup(String s) { … } } import shared.Trie; Shared sh = Shared.registerClass("shared.Trie", 1); Trie trie = (Trie) sh.getObject(0); … trie.lookup(“aback”); A A B C A B C A R O N K F T A

27. Dissertation Defense 6/12/2001 27 Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

28. Dissertation Defense 6/12/2001 28 Hierarchical Resource Framework l Resource API to allow for different policies l Hierarchical management l Work-conserving

29. Dissertation Defense 6/12/2001 29 CPU and Memory Allocation l Memory –Must not require partitioning –Soft limits: Limit overall use, allow temporary use of surplus memory –Hard limits: Provide memory guarantees if desired l CPU –Stride-scheduling for processes as a whole –Priority-based scheduler for threads in process

30. Dissertation Defense 6/12/2001 30 Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation –Performance for well-behaved code –Performance for adverse loads l Related Work l Future Work and Conclusion

31. Dissertation Defense 6/12/2001 31 Performance for Well-behaved Code l Baseline performance l Write barrier overhead –Microbenchmarks –SpecJVM98 benchmarks l Compared against –Base JVM: Kaffe (Kaffe00 from June 2000, www.kaffe.org) –IBM JDK 1.1.8 – fastest industrial 1.1 JVM l Microbenchmarks –Instruction cost: »11 cycles minimum with one word overhead per object, 43 cycles minimum with no memory overhead

32. Dissertation Defense 6/12/2001 32 SpecJVM Performance of KaffeOS

33. Dissertation Defense 6/12/2001 33 Write Barrier Counts Benchmark Dynamic Count (millions) Write Barrier Overhead (Best-case) for No Heap Pointer Conservative upper bound for IBM JDK compress0.47-0.5% (0.0%)0.7% jess7.94-1.1% (1.0%)10.5% db30.092.5% (3.8%)8.1% javac20.785.6% (2.5%)7.3% mpegaudio5.51-1.4% (0.8%)-1.7% mtrt3.060.3% (0.5%)-0.2% jack19.903.4% (1.9%)14.1%

34. Dissertation Defense 6/12/2001 34 Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation –Performance for well-behaved code –Performance for adverse loads l Related Work l Future Work and Conclusion

35. Dissertation Defense 6/12/2001 35 Performance for adverse loads l DOS Scenarios against –Memory –CPU –Garbage collector l KaffeOS –is more robust –delivers more effective service –improves performance under adverse loads

36. Dissertation Defense 6/12/2001 36 DoS Scenario l Servlet Engine: Apache 1.3.12, JServ 1.1, JSDK 2.0 l Some number of “good” servlets l Plus one MemoryHog servlet –Allocate lots of memory –Try to crash/tie up JVM l How would we deal with this? –Run 1 servlet per IBM JVM (IBM/1) –Run all servlets on one IBM JVM (IBM/n) –Run each servlet in a KaffeOS process

37. Dissertation Defense 6/12/2001 37 Service Under DoS Attack

38. Dissertation Defense 6/12/2001 38 MemHog Stresstest

39. Dissertation Defense 6/12/2001 39 CPU Denial of Service Scenario l Problem: cannot tell a CPU denial-of-service attack from a process doing useful work l Instead: ensure that processes obtain their share when runnable l Allow for manipulation of shares to reflect user priorities l MD5 Servlet computes MD5 hash sum over /usr/dict/word for each request –Requests/sec is measure of service

40. Dissertation Defense 6/12/2001 40 CpuHog Scenario

41. Dissertation Defense 6/12/2001 41 GarbageHog Scenario

42. Dissertation Defense 6/12/2001 42 Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

43. Dissertation Defense 6/12/2001 43 Related Java Work l Java VMs –Multi-processing JVM [Balfanz 98] –J-Kernel [Hawblitzel et al. 98] –Alta [Tullmann 99] –IBM OS/390 JVM [Dillenberger 00] –MVM [Czajkowski 01] –JSR 121 l Java Language Extensions –Luna [Hawblitzel 99] l Java Realtime Extensions –Realtime Working Group [Sun et al. 00]

44. Dissertation Defense 6/12/2001 44 Related OS Work l Single-address-space OS –Opal [Chase et al. 94] l Single-language OS –Pilot [Redell et al. 80] –Cedar [Swineheart et al. 86] –Inferno [Dorward et al. 97] l Extensible OS –SPIN [Bershad et al. 95]

45. Dissertation Defense 6/12/2001 45 Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

46. Dissertation Defense 6/12/2001 46 Future Work l JanosVM (ongoing) –Java operating system for active network node l Integration in higher-performing JVM –More accurate evaluation of costs and benefits l Static analysis using meta-level compilation –Ensure properties about user/kernel mode, shared and reloaded classes etc. statically l Automatic user-level sharing

47. Dissertation Defense 6/12/2001 47 Contributions l Operating system principles apply to language runtime systems –Architectural concept: user/kernel boundary l Software mechanisms can be used to implement full isolation of processes in a Java virtual machine –GC mechanisms: write barriers, distributed GC techniques l KaffeOS demonstrates how to reconcile isolation, resource management and sharing –Can stop resource-based denial-of-service attacks with small penalty for well-behaved code

48. Dissertation Defense 6/12/2001 48 Acknowledgments l Wilson Hsieh l Patrick Tullmann l Jason Baker l Tim Stack l Jay Lepreau and rest of Flux group for support and feedback on earlier papers

49. Dissertation Defense 6/12/2001 49 The End l Time for more Questions

50. Dissertation Defense 6/12/2001 50 Soft and Hard Limits l Hard limits: –are debited in full from parent –provide memory guarantees l Soft limits: –only actual use is debited –limit overall memory use –allow siblings to use surplus memory 64 MB 60 MB Limit Use 48 MB 44 MB 48 MB 30 MB 48 MB 14 MB roothardsoft 16 MB 9 MB

51. Dissertation Defense 6/12/2001 51 CPU Scheduling l Stride scheduler for processes l Priority-based scheduler for threads in process Root 100%60%10%20%40% Thread Priority

52. Dissertation Defense 6/12/2001 52 MemHog (Without User/kernel)

53. Dissertation Defense 6/12/2001 53 Summary l Java operating systems are necessary to support untrusted Java applications l KaffeOS –Architectural concepts taken from operating systems –Mechanisms taken from garbage collection »Distributed GC »Write barriers l Big issue: dealing with memory/GC l Resource-based denial-of-service attacks can be stopped

54. Dissertation Defense 6/12/2001 54 The KaffeOS Java Operating System l Separation –Processes do not interfere with each other –Safe termination l Resource management –Resource consumption can be limited –Recover all resources when an application dies –Account for shared resources reasonably l Direct sharing –Interprocess communication can be efficient –Preserve the Java model

55. Dissertation Defense 6/12/2001 55 What Is a Java Operating System? l Execution environment for Java bytecode –Support for multiple applications: operating system –Implemented fully in software: no hardware support »Can be embedded in another application »Does not depend on underlying OS for features l Not a “single-language OS” –OS does not have to be written entirely in Java –Does not need to be standalone

56. Dissertation Defense 6/12/2001 56 J-Kernel l Processes are called tasks –Separate namespaces l Pure Java –portability across JVMs l Indirect sharing –capabilities –bytecode rewriting to support automatic marshaling/unmarshaling for remote calls –thread migration –no direct cross-process references –gc is shared

57. Dissertation Defense 6/12/2001 57 Alta l Modeled after Fluke microkernel [Ford 96] –process hierarchy, nesting l Hierarchical management –parent controls resource consumption of children –Siblings can share objects directly l Common heap –parent must judiciously control childrens communication to avoid cross-heap refs –GC is shared service

58. Dissertation Defense 6/12/2001 58 K0 l Modeled after monolithic kernel l Separation –separate heaps with separate garbage collection –write barrier to avoid cross-heap references l Clear user/kernel separation l Direct sharing of objects –restricted to enable full reclamation

59. Dissertation Defense 6/12/2001 59 Copying  Simple accounting  Full reclamation  Requires serialization of objects  Can be slow P1P1 P2P2

60. Dissertation Defense 6/12/2001 60 Indirect Sharing  Revocable proxies  Full reclamation  Method invocation requires copying P1P1 P2P2

61. Dissertation Defense 6/12/2001 61 Direct Sharing  Separate heaps  Dedicated shared heap  Full reclamation  Direct sharing is restricted  Requires monitoring of store instructions P1P1 P2P2

62. Dissertation Defense 6/12/2001 62 Hybrid  Supports both direct and indirect sharing  More flexible  Direct references can prevent full reclamation  Method invocation may require copying P1P1 P2P2 Parent

63. Dissertation Defense 6/12/2001 63 Comparison RTJ and KaffeOS l Different goals and assumptions l Assumptions: –KaffeOS: user code is untrusted, kernel code is trusted –RTJ: all code is trusted (embedded environment) l Primary Goals: –KaffeOS: guard against malicious or buggy applications –RTJ: provide predictable (bounded, “real-time”) performance

64. Dissertation Defense 6/12/2001 64 ScopedMemory vs. Separate Heaps l Both –control pointer assignments –can be limited: resource limits vs. budgeted allocation –can allow for bounded allocation (KaffeOS hard resources vs. LTMemory) l RTJ ScopedMemory –is not garbage collected, does not allow any cross-heap refs –threads can enter/leave scopes l KaffeOS Heaps –are garbage collected (exception: shared heaps!) –GC time is subject to process scheduling –allows cross-heap refs where needed –threads cannot switch heaps (Janos threads can)

65. Dissertation Defense 6/12/2001 65 Memory and CPU Accounting l Accurate and complete per-process accounting –Minimize number of objects on kernel heap –Minimize amount of time spent in kernel code l Separate garbage collection –Minimize unaccounted resource use –Avoid priority inversion –Maintain isolation

66. Dissertation Defense 6/12/2001 66 Code Structure User GC System GC User code (untrusted) Kernel code (trusted) Runtime Libs (trusted) Finalization User Kernel Java C

67. Dissertation Defense 6/12/2001 67 User/Kernel vs. AIEs l Both –Asynchronously affect a thread’s execution –Are deferred during certain sections of code l RTJ: –Thread decides when it’s safe to receive AIEs –AIEs disabled during synchronized { } and constructors –Limited compiler support l KaffeOS –Only kernel can defer termination –User can be terminated any time – system consistency is unaffected –Compiler support future work

68. Dissertation Defense 6/12/2001 68 l Or: can you have your cake and eat it too? Are AIEs useful? class Queue { synchronized Object get(); synchronized void put(Object); } Queue producer, consumer; void run () throws AsynchronouslyInterruptedException { while (true) { Object o = producer.get(); /* is it safe to terminate here? */ consumer.put(o); /* how about here? */ }

69. Dissertation Defense 6/12/2001 69 Using synchronized (I) l Decreased parallelism l More deadlocks void run () throws AsynchronouslyInterruptedException { while (true) { synchronized { Object o = producer.get(); consumer.put(o); }

70. Dissertation Defense 6/12/2001 70 Using synchronized (II) l Blurs semantics of “synchronized” l Error-prone and inefficient l Synchronization != Deferred termination void run () throws AsynchronouslyInterruptedException { while (true) { synchronized (new Object()) { Object o = producer.get(); consumer.put(o); }

71. Dissertation Defense 6/12/2001 71 void poll() throws AsynchronouslyInterruptedException {}; void run () { while (true) { Object o = producer.get(); consumer.put(o); try { poll(); } catch (AsynchronouslyInterruptedException) { return; } Polling l But: polling is better done explicitly instead of using “magic” – optimize for readability!

72. Dissertation Defense 6/12/2001 72 Better polling l No AIEs needed l Question: is it possible to write large pieces of code that accesses shared state and can cope with asynchronous event notification? l If no: deferred delivery provides a false sense of security void run () { while (!stopped) { Object o = producer.get(); consumer.put(o); }

73. Dissertation Defense 6/12/2001 73 KaffeOS vs. RTJ l Very similar mechanisms, very different goals l KaffeOS implementation base could easily provide ScopedMemory –Mechanisms in place (write barriers, separate allocators etc.) l Question: will RTJ be enough for embedded systems? Or will they too need OS structure to isolate applications?