Context-Specific Middleware Specialization Techniques for Optimizing Software Product-line Architectures Arvind S. Krishna, Aniruddha S. Gokhale, Douglas C. Schmidt Institute for Software Integrated Systems, Dept of EECS Vanderbilt University Nashville, TN, USA Venkatesh P. Ranganath, John Hatcliff Dept of Computer and Information Sciences Kansas State Univ Manhattan, KS, USA Eurosys’06, Leuven, Belgium April 18-21, 2006

2 F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Middleware for Product Line Architectures Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) e.g., Boeing Boldstroke Avionics mission computing PLA for Boeing fighter aircrafts (F-15, F/A-18, AV-8B, UCAV/JUCAS) DRE system with 100+ developers, 3,000+ software components, 3-5 million lines of C++ Used as open experimentation platform Air Frame AP Nav HUDGPS IFF FLIR Domain-specific Services

3 F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) However, standards-based, general-purpose, layered middleware is not yet adequate for the most demanding & mission-critical PLA-based DRE systems Air Frame AP Nav HUDGPS IFF FLIR Domain-specific Services Middleware for Product Line Architectures

4 F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Specialized Middleware Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) However, standards-based, general-purpose, layers middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems Air Frame AP Nav HUDGPS IFF FLIR Soln: Middleware Specialization for PLA-based DRE systems Middleware for Product Line Architectures

5 Middleware Specialization Evaluation Criteria Premise Application of specialization techniques should result in considerable improvements in QoS over & above horizontal general-purpose middleware optimizations Handcrafting specializations infeasible for large-scale DRE systems => need for tools and processes Specializations should have minimal impact on standards compliance (APIs) Evaluation Criteria Use TAO ( as gold standard with several general-purpose optimizations Set performance improvements ~30 to 40% improvement from application of specializations cumulatively Turning on just one/two optimizations might improve performance by ~10 to 15%

6 Opportunities for Middleware Specialization Certain functionality can be excessive for PLAs e.g., layered demultiplexing, leading to unnecessary performance overhead Challenge: automatically remove specification- imposed generality when it’s not needed Goal is to devise techniques that apply to any standards compliant middleware, not just an implementation Dimension #1: Specification-imposed generality Standards-based general purpose middleware functionality defined by specifications such as CORBA, J2EE etc 4

7 Dimension #2: Middleware framework generality General-purpose middleware implementations need to work across applications that have varying functional & QoS requirements Accommodate variability by providing hooks e.g., for different protocol, concurrency & demultiplexing strategies Hooks add overhead  indirections & dynamic dispatching PLAs however require one alternative; one protocol TCP/IP, VME, SCTP, SHMIOP Thread-pool, Single- threaded, Thread-per connection Challenge: Automatically specialize middleware frameworks to eliminate unnecessary hooks Goal is devise techniques applicable to distributed systems that apply common patterns Opportunities for Middleware Specialization

8 Dimension #3: Platform generality Middleware implementations run on different hardware/OS/compiler platforms Platforms provide certain optimizations that can be leveraged to enhance QoS gcc 3.2 (no exceptions), timesys kernel Green-hills compiler, vxWorks platform Challenge: Automatically discover PLA deployment platform characteristics to improve QoS Goal is to devise techniques that apply to any host infrastructure middleware (e.g., ACE or JVMs) targeting heterogeneous OS, compiler, & hardware platforms

9 Bold Stroke PLA Scenario Example PLA configuration: Basic Single Processor (BasicSP) – DRE system scenario based on Boeing Bold Stroke challenge problems from DARPA PCES & MoBIES ACE_wrappers/TAO/CIAO/DAnCE/examples/BasicSPCoSMIC/examples/BasicSP Goal: Select representative DRE system, e.g., “rate based” events for control information & operations that transmit common data Timer Component – Triggers periodic refresh rates GPS Component – Generates periodic position updates Airframe Component – Processes input from the GPS component & feeds to Navigation display Navigation Display – Displays GPS position updates

10 Identifying “Ahead of Time” System Invariants A specific Reactor used Protocol: A specific protocol used Specification Invariance Framework Invariance Deployment Invariance Does not support native exceptions Single method interfaces: Sends same operation on wire

11 Specializing Dispatch Resolution Normal layered path Optimized Fast path processing Request Processing Paths Normal layered path  Uses general- purpose optimization for request lookup Optimized fast path  Bypasses middleware layers to directly perform operation invocation ~15 processing steps across three layers in ACE+TAO middleware Invariants The same operation is invoked on a given connection This specializations is an example of Memoization + Layer-folding specialization, where a pre-computed result is cached & used to bypass middleware layers Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

12 Specializing Reactor Component Reactor Framework Reactor separates event detection from demultiplexing; ~5,000 LOC across ACE+TAO modules Applies bridge pattern to support different reactors Specialize Bridge Pattern Remove indirection, e.g., Reactor_Impl base class completely (all virtual methods concrete) No changes to component interface & doesn’t break compatibility Invariants Middleware components, such as Reactor and protocol components remain same for a PLA scenario Select Reactor select () Reactor_Impl select () WFMO_Reactor select () Thread_Pool Reactor select () This specialization is an example of Aspect Weaving specialization, where crosscutting middleware features are customized Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

13 Deployment Platform Specializations Invariants Hardware, OS, & compiler characteristics do not change Deployment Platform Generality Platform characteristics affect QoS TANGO  Timesys linux + gcc ACE  vxWorks + Greenhills compiler Certain versions of gcc do not support exceptions Specialize Middleware for Deployment platform Empirically (via benchmarks) determine the right set hardware, OS & compiler characteristics suitable for the deployment platform (similar to research done by Yotov et.al) This specialization is a variation of Constant Propagation where the constant (platform characteristic) is used to tailor the middleware Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

14 Feature Oriented CUStomizer (FOCUS) Middleware Instrumentation Phase Middleware Specialization Phase FOCUS addresses specialization challenges by building specialization language, tool, & process to capture & automate middleware specializations ~1,000 Perl SLOC Parser + weaver ~2,500 XML SLOC specialization files ~50 (files) annotations Capture specialization transformations via FOCUS specialization language Annotate middleware source code with specialization directives Create a domain-specific language (DSL) to capture middleware variability Analyses & determines the type of specializations applicable FOCUS transformation engine selects the appropriate transformations & uses the annotations to automate specializations

15 FOCUS Specialization Language (FSL) FOCUS uses an XML DTD to create a DSL for capturing specializations FSL Capability to perform code substitutions Devirtualize interfaces Replace base classes with derived class FSL Approach : Capability to do … on code ACE_Reactor_Impl ACE_Select_Reactor_Impl Select_Reactor.h HOOK-START HOOK-END HOOK-COPY Capability to weave code at specified points The layer folding specializations require code to be woven in along the request processing path FSL Approach : that specifies the annotated point; where data is specified Capability to specialize base implementations Framework specialization requires code to be copied from derived to base classes FSL Approach : Copy code from a specific point to a specific point between different files  start copying  hook to stop copying  destination FORWARD_DECL #include “Select_Reactor.h”

16 Specialization Experimental Setup Goals Application of specialization techniques should result in considerable improvements in QoS over & above horizontal general-purpose middleware optimizations TAO baseline Active demultiplexing & perfect hashing for O(1) request demultiplexing Buffer caching & direct collocation optimization Optimized configuration for different ORB components Experiment Setup Pentium III 850 Mhz processor, running Linux Timesys kernel, 512 MB of main memory, TAO version compiled with gcc Timers at the client & within ORB used to collect data Used Emulab testbed Specialized TAO Middleware

17 Results for Layer-folding Specialization Average end-to-end measures improved by ~16% Average path measures improved by ~40% Worst case path measure improved by ~20% Worst case end-to-end latency improved by ~14% Path specialized latency measures Path defined as code- path when a request is received until the upcall is dispatched on the skeleton Experiment End-to-end latency measures for: General-purpose optimized TAO with active demultiplexing & perfect hashing Specialized TAO with layer folding specialization enabled Specialization applied at the server side (can also be applied at the client side) Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality Dispersion improves by a factor of ~1.5 for both cases

18 Aspect Weaving Specialization Results Experiment Client/server communicate by sending a simple long End-to-end latency measures for: General-purpose optimized TAO and Reactor, Protocol frameworks specialized with FOCUS compared Result Synopsis Average latency improves by ~5% for both protocol & reactor specializations Average Measures: Reactor 4.5% Protocol: 5% Jitter improves, though not considerably Jitter better; specialization does not compromise predictability 99% values closer to average & worst-case values are better indicating better predictability Worst case measures improved compared to general-purpose TAO Specialization also applied to other frameworks in TAO: Messaging, Wait Strategies Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

19 Results for Autoconf Specialization Deployment platform specialization Exception support: Loop unrolling optimization During ORB configuration, autoconf used to use emulated exceptions & loop unrolling optimization Average latency improves ~17% Predictability improved over general-purpose middleware Worst-case measures improve by ~50 µs Future research will extend research on these empirical optimizations to more model-driven approaches for performance prediction Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

20 Cumulative Specialization Results Worst-case measures improved by ~45% End-to-end client side throughput improved by ~65%. Results exceeded the hypothesis & evaluation criteria Specification related Layer folding Memoization Constant propagation (ignoring endianess) Framework Aspect weaving (Reactor + protocol) Deployment Loop unrolling + emulated exceptions Average end-to- end measures improved by ~43% Jitter results twice as good as general-purpose optimized TAO Layer folding, deployment platform, memoization, constant propagation

21 Evaluating FOCUS Pros & Cons Strengths Provides a lightweight, zero (run- time) overhead middleware specialization Designed to work across different languages (e.g., Java & C++) KSU applying FOCUS & specializations to Java ORBs XML-based rule capture Easy language extension, ability to add new features easily If/when C++ aspect technologies mature, can transform them into aspect rules via XSLT transforms Execute transformations via scripting Integration with QA tools; code generation from models Drawbacks Doesn’t provide full-fledged language parser, i.e., join points identified via annotations or via regular expressions Need to synchronize annotations with specialization files, so modifying source code requires change to specialization files Ameliorated via distributed continuous QA; Limitation exists even with aspects Correctness of transformations have to be validated externally; unlike AspectJ Need higher level tools to validate combinations of specializations FOCUS available in latest ACE+TAO distribution in ACE_wrappers/bin/FOCUS

22 Lessons Learned (1/2) Need to document specialization interplay Specializations applied such that no dependencies/conflicts exists As more specializations are developed it is necessary to document dependencies between specializations QoS benefits implementation specific Optimizations improve QoS over & above general-purpose optimizations Other ORBs, e.g., JacORB, do not use such optimizations; performance gains in this case would be more Preliminary results [Daugherty] on ZEN show that speed up was better compared to TAO TAO uses active demultiplexing & perfect hashing for O(1) demux time [Daugherty] Gary Daugherty, A Proposal for the Specialization of HA/DRE Systems, ACM PEPM 04

23 Lessons Learned (2/2) Specializations have potential to improve task schedulability Specializations help high priority tasks finish ahead of their time to complete Tasks with priority 50 finish early, increasing time available for scheduling priorities with 35 & 10 Specializations can benefit both hard real-time & soft & softer real-time tasks Adaptation with specializations can adversely affect QoS Specializations do not consider any form of recovery if invariance assumptions fail Adaptation requires loading general-purpose code, add checks along request processing path; increases jitter for DRE systems POA S1S2 POA S3 20 S4 35 POA S5 40 S5 50 Specializations increase slack in the system

24 Future Work  System Optimizations FOCUS approach applied to middleware optimizations Model-Driven Technologies Domain-Specific Modeling Languages Future work will focus on identifying system level (middleware, platform, application) specializations Goal is to drive the specialization process to optimize systems layer-to-layer Capturing invariants in models and using generative technologies to drive specializations Other QoS parameters

25 F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Concluding Remarks Resolving the tension between Generality  Middleware is designed to be independent of particular application requirements Specificity  PLAs are driven by the functional & QoS requirements for each product variant (using SCV analysis) Specialized Middleware Stack Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Domain-specific Services Domain-specific language (DSL) tools & process for automating the specializations Development of reusable specialization patterns Identifying specialization points in middleware where patterns are applicable Latency improvements of 45%

QUESTIONS ?

27 FOCUS Research Goals Contributions: The goal of this work isn’t on specialization languages per se, but instead to quantify the improvements that come from applying various types of specializations to middleware Transformational approaches: Stratego-XT, DMS support a broader range of specializations Domain specific language for capturing specializations; minimize accidental complexity Process to support specialization with evolution Language agnostic; leverage annotations Ensuring transformations (1) do not incur additional overhead; (2) do not compromise portability Develop a process that enables role separation Applying specializations improve latency ~30 – 40%

28 FOCUS Specialization Phase //File: Reactor.h REACTOR_SPL_INCLUDE_HOOK // Code woven by FOCUS: #include "ace/Select_Reactor.h" // END Code woven by FOCUS class Reactor { public: int run_reactor_event_loop (); // Other public methods.... private: // Code woven by FOCUS: ACE_Select_Reactor *reactor_impl_; // End Code woven by FOCUS // }; //File: Reactor.h REACTOR_SPL_INCLUDE_HOOK class Reactor { public: virtual int run_reactor_event_loop (); // Other public methods.... private: ACE_Reactor_Impl *reactor_impl_; // }; Annotated middleware sourceTransformed middleware source These specializations don’t affect framework “business logic,” just its structure Existing framework is still available when developers need OO capabilities Removed virtual keyword Replaced ACE_Reactor_Impl with ACE_Select_Reactor #include concrete header file Annotations help identify join points

29 Automating Layer Folding Specialization Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality TAO_DISPATCH_OPT_ADD_HOOK 1. if (__dispatch_resolved__) 2.{ 3: //First invocation normal path 4: __dispatch_resolved_ = 1; 5: } 6: else 7:{ // All invocations go through --- Optimized path 8: this->request_disaptcher__.dispatch (…) 9:} Normal path; the dispatch is cached in the process Directly use cached dispatcher ComponentSource Files {h,cpp} (loc)FOCUS Specialization file Layer folding Demultiplexing (tao, PortableServer) tao/GIOP_Base  1,200 PortableServer/Servant_Base  2,000,, Specialization file ~250 loc Tag related changes (5 locations) in TAO & Portable Server Automating Specialization via FOCUS On a per connection basis determine the target dispatcher; For subsequent requests, directly use the dispatcher to process requests FOCUS enables specification of specializations in corresponding language syntax

30 Automating Reactor Specialization ComponentSource Files {h,cpp} (loc)FOCUS Specialization file Aspect Weaving Reactor Framework (ACE, TAO modules) ace/Reactor  1,500 ace/Reactor_Base  1,800 ace/Select_Reactor  2,500 tao/advanced_resouces  2,500,, Specialization file ~700 loc Tag related changes (10 locations) < 0.1% of the code within the Reactor framework Automating Specialization via FOCUS, : weave code that creates specialized Reactor to move implementation from Select_Reactor to base Reactor component : comment unspecialized code TAO_REACTOR_SPL_COMMENT_HOOK_END ACE_NEW_RETURN (impl, ACE_Select_REACTOR (….), 0); ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } TAO_REACTOR_SPL_COMMENT_HOOK_END } Transformations preserve hooks; Minimizes clutter in middleware source Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

31 Automating Platform Specializations Deployment platform specialization Exception support: Autoconf’s AC_COMPILE_IF_ELSE macro to check if compiler supports native exception Loop unrolling optimization Autoconf’s AC_RUN_IF_ELSE macros to compile & run benchmark to quantify memcpy() performance improvements ACE_CACHE_CHECK([if ACE memcpy needs loop unrolling], [ace_cv_memcpy_loop_unroll], [AC_RUN_IFELSE([AC_LANG_SOURCE([[ /// …. Program that will run the benchmark to //// determine if an optimization is better Variable that will be checked to set a compilation flag Directive to run the benchmark Provide actual source language that will be compiled & run This approach can be applied automatically to discover OS- specific system calls & compiler settings that maximize QoS ComponentSource Files {h,cpp} (loc)FOCUS Specialization file Constant Propagation (autoconf) ace/CDR_Stream  4,000 ace/Message_Block  2,000 Code changes ~650 Benchmark ~250 Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality

32 Concluding Remarks Domain-specific language (DSL) tools & process for automating the specializations Specification-imposed specializations Layer-folding Constant propagation Memoization Framework specializations Aspect Weaving techniques Bridge  Reactor Template method  Protocol Strategy  Messaging, Wait, Demultiplexing Deployment platform specializations Unroll copy loops Use emulated exceptions Leverage zero-copy data transfer buffers Development of reusable specialization patterns Identifying specialization points in middleware where patterns are applicable

33 F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Research Challenges Resolving the tension between Generality  Middleware is designed to be independent of particular application requirements Standards-based, General-purpose, Layered Middleware Architecture Specificity  PLAs are driven by the functional & QoS requirements for each product variant (using SCV analysis) Specialized Middleware Stack Research Challenges Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Domain-specific Services

34 Related Research CategoryRelated Research Automatic language mechanisms T. Mogensen & A. Bondorf, “Logimix: A Self-Applicable Partial Evaluator for Prolog”, Program Evaluation & Program Specialization conference (PEPM), 1993 S.M. Abramov, “A Compiler Based on Partial Evaluation”, Problems of Applied Mathematics & Software Systems, Moscow State University Press, Domain: Operating Systems Massalin et al. “The Synthesis Kernel”, Computing Systems Journal, 1998 Pu et al. “Optimistic Incremental Specialization: Streamlining a Commercial Operating System”, SOSP 95 Compilers & code optimizers Yotov et al. “A Comparison of Empirical & Model Driven Optimizations”, PLDI 2003 Domain: Feature oriented specialization Zhang et al. “Towards Just in Time Middleware”, Aspect Oriented Software Engineering, (AOSD 2005), Chicago, 2004 Zhang et al. “Resolving Feature Convolution in Middleware”, ACM SIGPLAN OOPSLA Conference, Vancouver, Canada 2004 Pluggable Middleware GOPI Middleware ( PoliORB ( Current research on specializations does not address how techniques & tools can be applied to middleware to improve QoS

35 ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } TAO_REACTOR_SPL_COMMENT_HOOK_END } Middleware Annotations Middleware developer annotates middleware with hooks Hooks represented as “meta-comments,” i.e., opaque to compilers FOCUS Instrumentation Phase 11: 12: 13: 14: TAO_REACTOR_SPL_COMMENT_HOOK_START 15: TAO_REACTOR_SPL_COMMENT_HOOK_END 16: 17: 18: 23:.... Annotations help identify join points & relieve FOCUS from Implementing a full fledged language parser

36 Lessons Learned (2/2) Specializations have potential to improve task schedulability Specializations help high priority tasks finish ahead of their time to complete Tasks with priority 50 finish early, increasing time available for scheduling priorities with 35 & 10 Specializations can benefit both hard real- time & soft & softer real-time tasks Adaptation with specializations can adversely affect QoS Specializations do not consider any form of recovery if invariance assumptions fail Adaptation requires loading general-purpose code, add checks along request processing path; increases jitter for DRE systems QoS improvements are scenario specific All our specializations improve path latencies considerably than end-to-end latency; More the specialized code path is traversed, greater is QoS improvement POA S1S2 POA S3 20 S4 35 POA S5 40 S5 50 Specializations increase slack in the system

37 Applicability of Specialization Approaches Resolving Specification Imposed Generality Applicable to standards compliant CORBA middleware Layer folding specialization  CORBA demultiplexing; Other layered demultiplexing approaches Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality Avoiding (de)marshaling checks  middleware standards such as J2EE,.NET that target heterogeneous deployment Big endian Little endian

38 Applicability of Specialization Approaches Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality TCP/IP, VME, SCTP, SHMIOP Thread-pool, Single- threaded, Thread-per connection Resolving Framework generality  Specialize design pattern Reactor Framework specialization  Bridge pattern Protocol framework specialization  Template method pattern Techniques to distributed systems that apply these patterns (

39 Applicability of Specialization Approaches Dim #1: Specification Imposed generality Dim #2: Framework generality Dim #3: Deployment generality Green-hills compiler, vxWorks platform gcc 3.2 (no exceptions), timesys kernel Dimension #3: Resolving Deployment generality Host Infrastructure middleware (ACE, JVMs) targeting heterogeneous OS, compiler & hardware characteristics Systems concerned with QoS

40 Comparing FOCUS with Related Work CategoryRelated Research Aspect Oriented Mechanisms Kiczales, “An Overview of AspectJ”, Lecture Notes in Computer Science, 1999 Lohmann et al. “Generative Programming with Aspect C++”, GPCE 2004, Vancouver, Canada Todd Veldhuizen “Using C++ meta programs”, C++ Report 1999 Transformational approaches Ira Baxter “Design Maintenance System”, Semantic Designs Inc, Stratego-XT ( Template code generation approaches Jostraca ( Java Emitter Templates ( FOCUS Characteristics FOCUS design motivated by template code-generation approaches & generative programming techniques Allows annotations to be directly placed in code (placed as special comments) Annotations explicitly identify variability; support evolution A light-weight specialization approach (does not implement a full-blown language parser)