1. DARPA Dr. Douglas C. Schmidt dschmidt@darpa.mil DARPA/ITO Approved for Public Release, Distribution Unlimited Adaptive and Reflective Middleware Systems Friday, May 22, 2015

2. D. Schmidt DARPA 2 High-performance, real-time, fault- tolerant, and secure systems Adaptive & reflective autonomous distributed embedded systems Power-aware ad hoc, mobile, distributed, & embedded systems Middleware, Frameworks, & Components Patterns & Pattern Languages Standards & Open- source Addressing the COTS “Crisis” However, this trend presents many vexing R&D challenges for mission- critical DoD systems, e.g., Inflexibility and lack of QoS Confidence woes & global competition Distributed systems increasingly must reuse commercial-off-the-shelf (COTS) hardware & software i.e., COTS is essential to R&D success Why DARPA should care : Recent advances in COTS software technology can help to fundamentally reshape distributed embedded system R&D Despite IT commodization, progress in COTS hardware & software is often not applicable for mission-critical DoD distributed embedded systems

3. D. Schmidt DARPA 3 There are multiple COTS layers & multiple research opportunities Historically, mission-critical apps were built directly atop hardware The common middleware & domain- specific services layers are where many of the open R&D challenges reside The Evolution of COTS Standards-based COTS middleware helps: Manage distributed resources Leverage HW/SW technology advances Evolve to new environments & requirements & OS This was extremely tedious, error-prone, & costly over system life-cycles QoS specification & enforcement Real-time features & optimizations Layered resource management Transparent power management Early COTS middleware lacked: Advanced R&D has address some, but by no means all, of these issues

4. D. Schmidt DARPA 4 More emphasis on integration rather than programming Increased technology convergence & standardization Mass market economies of scale for technology & personnel More disruptive technologies & global competition Lower priced--but often lower quality-- hardware & software components The decline of internally funded R&D Potential for complexity cap in next- generation complex systems Consequences of COTS & IT Commoditization Not all trends bode well for long-term competitiveness of traditional R&D leaders Ultimately, competitiveness will depend upon longer-term R&D efforts on complex distributed & embedded systems

5. D. Schmidt DARPA 5 The DARPA/ITO Embedded Systems Family of Programs SEC Hybrid, adaptive, control & computation Hybrid, adaptive, control & computation Quorum Quality-of-service & translucent layers MoBIES Design technology & software CAD ARMS Adaptive & reflective middleware Adaptive & reflective middleware PCES Composable embedded systems NEST Deeply networked embedded systems PCA Polymorphous computing architecture

6. D. Schmidt DARPA 6 Example of DARPA ITO Impact: Real-time CORBA Specification www.cs.wustl.edu/~schmidt/PDF/orc.pdf Protocol Properties Explicit Binding Thread Pools Scheduling Service Standard Synchronizers Portable Priorities

7. D. Schmidt DARPA 7 Example of ITO Impact: COTS in Real-time Avionics Key System Characteristics Deterministic & statistical deadlines Periodic & aperiodic processing Complex dependencies Low latency & jitter Continuous platform upgrades Static scheduling & validation Small-scale network topology Static scheduling & validation Small-scale network topology Limited fault tolerance & security support Used some non- standard COTS features Limited fault tolerance & security support Used some non- standard COTS features Limitations Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF Drove Real-time CORBA standardization Key Results Goals Demo applicability of COTS & open systems for mission-critical RT avionics

8. D. Schmidt DARPA 8 Example of ITO Impact: Real-time Image Processing Goals Examine glass bottles for defects in real- time System Characteristics Process 20 bottles per sec i.e., ~50 msec per bottle Networked configuration ~10 cameras Key Software Solution Characteristics Affordable, flexible, & COTS Embedded Linux (Lem) Compact PCI bus + Celeron processors Affordable, flexible, & COTS Embedded Linux (Lem) Compact PCI bus + Celeron processors Remote booted by DHCP/TFTP Real-time CORBA www.krones.com

9. D. Schmidt DARPA 9 Example of R&D Impact: Hot Rolling Mill Control Framework Goals Control the processing of molten steel moving through a hot rolling mill in real-time System Characteristics Hard real-time process automation requirements i.e., 250 ms real-time cycles System acquires values representing plant’s current state, tracks material flow, calculates new settings for the rolls & devices, & submits new settings back to plant Key Software Solution Characteristics Affordable, flexible, & COTS Product-line architecture Design guided by patterns & frameworks Affordable, flexible, & COTS Product-line architecture Design guided by patterns & frameworks Windows NT/2000 (+ VMS!) Real-time CORBA www.siemens.de

10. D. Schmidt DARPA 10 Time-critical targets require immediate response because: They pose a clear and present danger to friendly forces & Are highly lucrative, fleeting targets of opportunity Example of ITO Impact: WSOA/QuoTE Time-Critical Target Prosecution Challenges are also relevant to TBMD & NMD WSOA Goals Detect, identify, track, & destroy time-critical targets Key Solution Characteristics Affordable & flexible COTS-based Affordable & flexible COTS-based High confidence Safety critical High confidence Safety critical Real-time mission-critical sensor-to-shooter needs Highly dynamic QoS requirements & environmental conditions Multi-service & asset coordination via Link 16 Key System Characteristics

11. D. Schmidt DARPA Example of ITO Impact: Large-scale Switching Systems IOM BSE Goal Switch ATM cells + IP packets at terabit rates Key Software Solution Characteristics High confidence & scalable computing architecture Networked embedded processors Distribution middleware FT & load sharing Distributed & layered resource management Affordable, flexible, & COTS High confidence & scalable computing architecture Networked embedded processors Distribution middleware FT & load sharing Distributed & layered resource management Affordable, flexible, & COTS Key System Characteristics Very high-speed WDM links 10 2 /10 3 line cards Stringent requirements for availability Multi-layer load balancing, e.g.: Layer 3+4 Layer 5 www.arl.wustl.edu

12. D. Schmidt DARPA 12 Large-scale (~5-250 miles) Integrated QoS properties Highly dynamic environments Semi-autonomous & re- configurable distributed systems Network Broader scope Hybrid scheduling & RT ARM Heterogeneous middleware & languages Hard real-time deadlines (~40hz) Low jitter & latency (~300us) Very hard real- time deadlines (>= 100hz) Very low latency (~10us) Highly autonomous & re-configurable hybrid & embedded systems Address fundamental QoS properties of distributed embedded & hybrid systems Enhance confidence of open-source R&D processes and V&V techniques Devise middleware-centric methods & tools to develop, optimize, & manage systems with multiple QoS properties Research Goals RTP DNS HTTP UDPTCP IP TELNET ETHERNETATMFDDI FIBRE CHANNEL FTP INTERNETWORKING ARCH TFTP Advance the state-of-the-art in adaptive & reflective middleware to coordinate multiple QoS properties of mission-critical distributed embedded & hybrid systems Why the “waist” works: 1.Decouples hardware from software so they can evolve separately 2.Decouples low-level capabilities from higher- level capabilities to enhance innovation 3.Decouples fast changing layers from slower changing layers However, the waist can also restrict choices… What Are We Trying to Do? VIRTUAL MACHINE ARCH Ix86TI DSP68K PA/RISC PowerPC Java VM Interpreter JavaAdaC/C++ Java Bytecode WINNTLINUXLYNXOS SOLARIS VXWORKS CORBA SERVICES CORBA APPLICATIONS MIDDLEWARE ARCH

13. D. Schmidt DARPA 13 New Challenges: Theater Ballistic Missile Defense Goal Detect, identify, track, & destroy multiple theater ballistic missiles Meeting hard real-time sensor-to- shooter needs in highly dynamic environment Providing load-invariant performance Supporting QoS for distributed weapons coordination Key Research Challenges Required Compute Power Required Processing TBMD Phases Radar Control Program Sizing Estimates Estimated data, based on 6 Ph3 & 7 Ph1 design & projections of Block I & Block II NTW requirements AREA 6 Ph 3 AREA 7 Ph 1 NTW BLK 1A NTW BLK 1B NTW BLK 1C NTW BLK 2 TBMD Phases: SMP Load Scale Limits Key Solution Characteristics Highly dependable & scalable computing architecture High-speed networked clusters of multi-processor computers Distribution middleware Fault tolerance & load sharing Distributed & layered global resource management Affordable, flexible, & COTS Highly dependable & scalable computing architecture High-speed networked clusters of multi-processor computers Distribution middleware Fault tolerance & load sharing Distributed & layered global resource management Affordable, flexible, & COTS

14. D. Schmidt DARPA 14 Example: Real-time Retargeting of UAVs Key System Characteristics Autonomous behavior e.g., battlespace loitering of UAVs Coordinated strikes e.g., C 4 ISR integration & UAV swarming for SEAD missions Real-time sensor-to-shooter QoS scheduling of sea/land/air assets Developing efficient, predictable, & safe adaptive HW/SW systems for autonomous UAV processing Real-time sensor-to-shooter QoS scheduling of sea/land/air assets Developing efficient, predictable, & safe adaptive HW/SW systems for autonomous UAV processing Secure integration to C 4 ISR infosphere e.g., proof-carrying code & automated policy-based control Transition to COTS HW/SW to control costs & leverage technology advances Secure integration to C 4 ISR infosphere e.g., proof-carrying code & automated policy-based control Transition to COTS HW/SW to control costs & leverage technology advances Key Research Challenges Goal Reconfigure & retarget unmanned air vehicles (UAVs) in real-time

15. D. Schmidt DARPA 15 New Challenge: Situational Awareness Systems Key System Characteristics Mission-critical, sensor-rich Multi-asset coordination e.g., UAVs, MEMs, SUOs Ad hoc wireless/mobile infrastructure Highly non-linear and dynamic Non-uniform resource constraints Integrate multiple QoS properties simultaneously e.g., dependability, security, & bandwidth management Tele-immersion situation monitoring e.g., QoS-based event fusion Integrate multiple QoS properties simultaneously e.g., dependability, security, & bandwidth management Tele-immersion situation monitoring e.g., QoS-based event fusion Adaptive & reflective peer-to-peer information system coordination e.g., power-aware systems COTS-based to control costs & to leverage rapid technology advances e.g., wireless tracking & local info Adaptive & reflective peer-to-peer information system coordination e.g., power-aware systems COTS-based to control costs & to leverage rapid technology advances e.g., wireless tracking & local info Key Solution Characteristics Goal Support dispersed & rapidly deployable teams

16. D. Schmidt DARPA 16 Example: Coordinated Disaster Response Key System Characteristics Mission-critical & time-critical systems Multi-agency coordination e.g., police, fire, FBI, medical Ad hoc wireless/mobile infrastructure Highly non-linear and dynamic Non-uniform resource constraints Goal Support rapidly deployable & dis- persed emergency response teams Integrate multiple QoS properties simultaneously e.g., dependability, security, & bandwidth management Tele-immersion situation monitoring e.g., QoS-based event fusion Integrate multiple QoS properties simultaneously e.g., dependability, security, & bandwidth management Tele-immersion situation monitoring e.g., QoS-based event fusion Key Solution Characteristics Adaptive & reflective peer-to-peer information system coordination e.g., power-aware systems COTS-based to control costs & to leverage rapid technology advances e.g., wireless tracking & local info Adaptive & reflective peer-to-peer information system coordination e.g., power-aware systems COTS-based to control costs & to leverage rapid technology advances e.g., wireless tracking & local info

17. D. Schmidt DARPA 17 Technology Enablers and Business Drivers

18. D. Schmidt DARPA 18 Problem Abstraction Distributed Environment Heterogeneous hardware/software Mixed COTS & non-COTS components Mixed RT and non-RT requirements Wireline & wireless interconnects Distributed Environment Heterogeneous hardware/software Mixed COTS & non-COTS components Mixed RT and non-RT requirements Wireline & wireless interconnects Adaptivity & Reflection Targets Dynamic component distribution & reconfiguration Changing interconnection topology Changing power-levels, CPU/network bandwidth, latency, security, & dependability requirements Adaptivity & Reflection Targets Dynamic component distribution & reconfiguration Changing interconnection topology Changing power-levels, CPU/network bandwidth, latency, security, & dependability requirements

19. D. Schmidt DARPA 19 The Hard Problems  Decoupling functional aspects from QoS aspects  Automatically generating & optimizing multiple QoS properties adaptively & reflectively  Articulating pattern languages & reifying them into QoS-enabled frameworks & components  Leveraging, customizing, enhancing, & validating open-source COTS components

20. D. Schmidt DARPA 20 Conventional COTS Limitations Many hardware & software APIs and protocols are now standardized, e.g.: Inflexible COTS negatively affects researchers & developers While COTS standards promote reuse, they limit design choices, e.g.: Networking protocols Concurrency & scheduling Caching Fault tolerance Security Historically, COTS tightly couples functional & QoS aspects e.g., due to lack of “hooks” TCP/IP, ATM POSIX & JVMs CORBA ORBs & components Intel x86 & Power PC chipsets Ada, C, C++, RT Java

21. D. Schmidt DARPA 21 Promising New Solution: Adaptive & Reflective Middleware Research Challenges MECHANISM/PROPERTY MANAGER SYS COND DELEGATE CONTRACT LOCAL RESOURCE MANAGERS LOCAL RESOURCE MANAGERS Preserve critical set of application QoS properties end-to-end e.g., efficiency, predictability, scalability, dependability, & security Achieve load invariant performance & system stability Preserve critical set of application QoS properties end-to-end e.g., efficiency, predictability, scalability, dependability, & security Achieve load invariant performance & system stability Maximize longevity in wireless & mobile environments e.g., control power-aware hardware via power-aware middleware Automatically generate & integrate multiple QoS properties Maximize longevity in wireless & mobile environments e.g., control power-aware hardware via power-aware middleware Automatically generate & integrate multiple QoS properties Adaptive & reflective middleware is middleware whose functional or QoS- related properties can be modified either Statically, e.g., to better allocate resources that can optimized a priori or Dynamically, e.g., in response to changes in environment conditions or requirements

22. D. Schmidt DARPA 22 The Hard Problems  Decoupling functional aspects from QoS aspects  Automatically generating & optimizing multiple QoS properties adaptively & reflectively  Articulating pattern languages & reifying them into QoS-enabled frameworks & components  Leveraging, customizing, enhancing, & validating open-source COTS components

23. D. Schmidt DARPA 23 Network Key Themes of WSOA Real-time mission replanning & collaboration e.g., C2 node & F-15 share data imagery & annotations Shows adaptive QoS behavior is feasible within demanding real-world constraints Showcase academic & industry synergy Limitations “Stove-pipe” architectures Only “opportunistic” integration Lack of multi-property QoS integration Not fully autonomous Limitations “Stove-pipe” architectures Only “opportunistic” integration Lack of multi-property QoS integration Not fully autonomous State-of-the-Art in QoS Demos DARPA, AFRL, & Boeing test flight in ‘01

24. D. Schmidt DARPA 24 Key Themes Handle variation translucently QoS aspect languages Smart proxies & interceptors Pluggable protocols & adapters Middleware gateways/bridges Ideally, implementations should be generated from higher-level specifications Promising New Solution: Middleware Frameworks for Integrating Multiple QoS Properties Research Challenges Model, compose, analyze, & optimize QoS framework component properties Leverage configurable & adaptive hardware capabilities e.g., power management, high-speed QoS-enabled bus & network interconnects Research Challenges Model, compose, analyze, & optimize QoS framework component properties Leverage configurable & adaptive hardware capabilities e.g., power management, high-speed QoS-enabled bus & network interconnects

25. D. Schmidt DARPA 25 Early compilers required Separate internal representations hand-written for each programming language and Separate hand-written optimizers for each target backend Developing, verifying, validating, & evolving all these components separately is costly, time-consuming, tedious, & error-prone The problem only gets worse as more languages & target backends emerge Applying Reflection as an Optimization Technique C Compiler Internal Rep. Ix86 Opt. Ix86 PPC Opt. PPC 68K Opt. 68K C Program To illustrate the benefits of reflection as an optimization technique, consider the evolution of compiler technology: C++ Compiler Internal Rep. Ix86 Opt. PPC Opt. 68K Opt. Ix86PPC68K C++ Program Ada Compiler Internal Rep. Ix86 Opt. PPC Opt. 68K Opt. Ix86PPC68K Ada Program

26. D. Schmidt DARPA 26 Applying Reflection as an Optimization Technique C/C++/Ada Compiler Common Internal Rep. Ix86 Opt. PPC Opt. 68K Opt. C/C++/Ada Programs Ix86.md PPC.md 68K.md Modern compilers, such as GNU GCC, support A common internal representation (still hand-written) for each programming language Based on generalizing the language semantics 1. Read the target machine description Optimizer Generator 2. Use discrimination network to analyze the optimization rules & opportunities 3. Generate an optimizer that is customized for the particular platform/language A generated optimizer that is customized automatically for each target backend Based on reflective assessment of algebraic target machine description Key Benefit of “Static” Reflection New targets can be supported by writing a new machine description, rather than writing a new code generator/optimizer

27. D. Schmidt DARPA 27 Developing, verifying, validating, & evolving all these components separately is costly, time- consuming, tedious, & error-prone Moreover, it is even harder to hand-configure support for dynamic platform variations & complex application use-cases The problem only gets worse as more middleware, target platforms, & complex applications emerge Separate hand-written & hand-optimized implementations for each embedded target platform e.g., various OS/network/HW configurations Conventional middleware require Separate tools and interfaces hand-written for each ORB middleware specification e.g., CORBA, Java RMI, COM+ Applying Reflection to Optimize Middleware Statically CORBA ORB & Assorted Tools CORBA Application Conventional middleware for embedded systems is developed & optimized in a manner similar to early compiler technologies: WinNT Impl Solaris Impl VxWorks Impl Java RMI & Assorted Tools WinNTSolaris Linux Java Application WinNT Impl Solaris Impl Linux Impl COM+ ORB & Assorted Tools WinNTWin98 WinCE COM+ Application WinNT Impl Win98 Impl WinCE Impl

28. D. Schmidt DARPA 28 Applying Reflection to Optimize Middleware Statically Common ORB + Assorted Tools Common Semantic Representation Plat 1 Impl The functional and QoS-related aspects of middleware can be improved greatly by advanced R&D on the following topics: A common internal representation (ideally auto- generated) for each middleware specification Based on generalizing the middleware semantics Middleware Generator 2. Use discrimination network to analyze the optimization rules & opportunities 3. Generate middleware that is customized for a particular platform & application use-case A generated implementation that is optimized automatically for each target platform & application use-case Based on reflective assessment of platform descriptions & application use-case Plat 2.pd Plat 2 Impl Plat 3 Impl 1. Read the target platform description & application requirements Application Requirements CORBA/Java/COM+ Applications Plat 3.pd Plat 1.pd

29. D. Schmidt DARPA 29 ClientObject ORB endsystem ORB endsystem Resource Applying Reflection to Optimize Middleware Dynamically Key System Characteristics Integrate observing & predicting of current status & delivered QoS to inform the meta-layer Meta-layer applies reflection to adapt system policies & mechanisms to enhance delivered QoS Delegate QuO Contracts Applying reflection as an optimization is even more relevant to middleware than compilers due to dynamism & global resources: Probes Piggybacked Measurements Status Expected QoS Measured QoS Correlate Probes Resource Status Service Collect Translate Integrate Infer/Adapt Feedback Loop

30. D. Schmidt DARPA 30 Key Research Challenge: Providing QoS Guarantees for Multiple Adaptive Feedback Loops Goals Ensuring stable QoS support at varying granularity & scope levels for integrated, multi-property feedback paths across different locations & time scales Determining patterns, protocols, and architectures necessary to integrate COTS components ClientObject Combined System-level & Application-level Management Feedback End-to-End Application-centric Feedback End-to-End Application-centric Feedback Local Resource- centric Feedback Local Resource- centric Feedback

31. D. Schmidt DARPA 31 The Hard Problems  Decoupling functional aspects from QoS aspects  Automatically generating & optimizing multiple QoS properties adaptively & reflectively  Articulating pattern languages & reifying them into QoS-enabled frameworks & components  Leveraging, customizing, enhancing, & validating open-source COTS components

32. D. Schmidt DARPA 32 New Idea: Pattern Languages for QoS Research Challenges Identifying QoS pattern languages Broaden the focus of conventional pattern-related tools and pattern languages, which focus on simple structural & functional behavior Model QoS-enabled middleware via pattern languages Must understand how to build high-confidence systems before we can automate V&V Identifying QoS pattern languages Broaden the focus of conventional pattern-related tools and pattern languages, which focus on simple structural & functional behavior Model QoS-enabled middleware via pattern languages Must understand how to build high-confidence systems before we can automate V&V Formal semantics Articulate QoS properties of core architectures Automation i.e., auto-generate portions of frameworks & components from pattern languages Formal semantics Articulate QoS properties of core architectures Automation i.e., auto-generate portions of frameworks & components from pattern languages Key Theme Patterns & pattern languages codify expert knowledge to help generate software architectures by capturing recurring structures & dynamics and resolving common design forces

33. D. Schmidt DARPA 33 The Hard Problems  Decoupling functional aspects from QoS aspects  Automatically generating & optimizing multiple QoS properties adaptively & reflectively  Articulating pattern languages & reifying them into QoS-enabled frameworks & components  Leveraging, customizing, enhancing, & validating open-source COTS components

34. D. Schmidt DARPA 34 “ Everything gets cheaper forever ” John Chambers, CEO Cisco Systems Quality team Teamwork Drive Change No technology religion Empowerment Frugality Market Transitions Stretch Goals Trust/Fair/ Integity Open Communication Cisco Culture Why COTS? Commercial & military suppliers are increasingly driven by competitive forces e.g., time-to-market/mission pressures & heavy competition for engineering talent COTS can contribute systematic reuse, continuous innovation, & cost reduction via 3 rd party life-cycle management COTS can potentially reduce V&V costs Lack of realistic alternatives… Key COTS R&D Challenges Key Technology Inhibitors to Success Integration woes COTS components are often not designed for composition V&V and security woes COTS components rarely designed for certification or high assurance Integration woes COTS components are often not designed for composition V&V and security woes COTS components rarely designed for certification or high assurance Inefficient feedback loops e.g., “binary-only,” closed-source deployment hampers usability COTS is not always standard Non-standard COTS can greatly increase “refresh” costs Inefficient feedback loops e.g., “binary-only,” closed-source deployment hampers usability COTS is not always standard Non-standard COTS can greatly increase “refresh” costs

35. D. Schmidt DARPA 35 Emerging trend in commodity IT market: Standard Open-source COTS Open-source is a highly scalable and cost effective software process based on the following observations: Validation scales, development does not “End-to-end argument” applies to software i.e., more resources at the “edges” Benefits for Developers Standards help to ensure longer-term viability of technology investments Standard COTS helps control life-cycle costs Standard open-source COTS helps to focus expertise, e.g.: Leverage “everyone’s a beta-tester” syndrome Resolves “COTS vs. ownership” conundrum in system acquisition Benefits of Open-source for Researchers: Leverage existing technology base for rapid prototyping of new technologies Promote broad visibility of novel R&D activities Accelerate the pace & impact of technology transfer Lead, rather than follow, COTS software trends

36. D. Schmidt DARPA 36 New opportunity: High Confidence Open-source Software Systems EMACS, ACE, & USENET servers COTS desktop productivity tools Linux, Apache, GNU tools, & TAO Solaris & Windows NT Next-generation middleware Flight critical software Open-source Closed-source No Spec Informal Spec Formal Spec Open-source standard COTS is now mainstream at certain layers Bold Stroke Key Themes We know how to build open-source software quickly and cheaply Quality and security remain key challenges, however… Open-source enables whitebox V&V techniques e.g., analysis methods can extend across layers & thru components Reuse of middleware components can help amortize V&V efforts No need to (re)start from scratch Middleware is often written in relatively “civilized” languages cf. operating system kernels Middleware defines natural module boundaries for specification & testing e.g., define QoS properties via QDLs

37. D. Schmidt DARPA 37 Concluding Remarks Researchers & developers of distributed systems face common challenges, e.g.: The application of formal methods along with adaptive & reflective patterns, frameworks, & components can help to resolve these challenges Carefully applying these techniques can yield efficient, scalable, predictable, & flexible middleware & applications Connection management, service initialization, error handling, flow control, event demuxing, distribution, concurrency control, fault tolerance synchronization, scheduling, & persistence Summary of Research Themes in the ARMS Program Decouple functional aspects & QoS aspects Specify & apply component QoS as meta-data Devise adaptive & reflective methods,optimizations, & tools that can provide scalable, multi-property QoS guarantees end-to-end Enable high-confidence autonomous system capabilites Leverage standard COTS APIs But not necessarily COTS implementations or protocols