1. Context Specific Middleware Specialization Techniques for Optimizing Product-line Architectures Arvind S. Krishna arvindk@dre.vanderbilt.edu Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee http://www.dre.vanderbilt.edu/~arvindk/MW-Spl.pdf

2. Presentation Road Map Technology Context Research Challenges Middleware Specialization Techniques Feature Oriented Customizer (FOCUS) for Middleware Applying FOCUS to PLAs & Middleware – A Case Study Concluding Remarks & Future Work

3. Motivation Air Frame GPS FLIR Legacy distributed real-time & embedded (DRE) systems have historically been: Stovepiped Proprietary Brittle & non-adaptive Expensive Vulnerable Consequence: Small HW/SW changes have big (negative) impact on DRE system QoS & maintenance GPS FLIR AP Nav HUD IFF Cyclic Exec F-15 Air Frame AP Nav HUD GPS IFF FLIR Cyclic Exec A/V-8B Air Frame Cyclic Exec AP Nav HUD IFF F/A-18 Air Frame AP Nav HUD GPS IFF FLIR Cyclic Exec UCAV

4. F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Motivation Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) Air Frame AP Nav HUDGPS IFF FLIR Domain-specific Services

5. F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Host Infrastructure Middleware Distribution Middleware Common Middleware Services Motivation Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) However, standards-based, general-purpose, layered middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems Air Frame AP Nav HUDGPS IFF FLIR Domain-specific Services

6. F-15 product variant A/V 8-B product variant F/A 18 product variant UCAV product variant Product-line architecture Hardware (CPU, Memory, I/O) OS & Network Protocols Customized Middleware Motivation Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility Essential for product-line architectures (PLAs) However, standards-based, general-purpose, layers middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems Air Frame AP Nav HUDGPS IFF FLIR My research optimizes middleware for PLA-based DRE systems

7. Presentation Road Map Technology Context Research Challenges Middleware Specialization Techniques Feature Oriented Customizer (FOCUS) for Middleware Applying FOCUS to PLAs & Middleware – A Case Study Concluding Remarks & Future Work

8. Overview of Research Challenges Resolving the tension between Generality  Middleware is designed to be independent of particular application requirements Customized Middleware StackStandards-based, General-purpose, Layered Middleware Architecture Product-line Variant Specificity  PLAs are driven by the functional & QoS requirements for each product variant Research Challenges

9. Overview of Research Challenges Specialization Automation Mature middleware implementations for PLAs, e.g., ACE+TAO (www.dre.vanderbilt.edu) have over million lines of code; hand crafting specializations for production middleware not feasible Framework Specialization –Middleware architecture define “hooks” Plug protocol implementations; PLAs however require one alternative –How can middleware frameworks be specialized to remove hooks? Deployment Specialization –Middleware implementations run on different hardware/OS/compiler platforms; PLAs are deployed on specific platforms –How can we leverage platform characteristics to improve QoS? Middleware Functionality Specialization –Specifications impose certain middleware functionality Need to work on machines with different endianess; Not required for homogenous PLAs –How can specification imposed generality be specialized to improve QoS?

10. Presentation Road Map Technology Context Research Challenges Middleware Specialization Techniques Feature Oriented Customizer (FOCUS) for Middleware Applying FOCUS to PLAs & Middleware – A Case Study Concluding Remarks & Future Work

11. Overview of FOCUS Middleware Instrumentation Phase Middleware developers Capture specialization transformations via FOCUS specialization language (FSL) Annotate middleware source code with specialization directives Create a domain-specific language (DSL) to capture middleware component variability Middleware Specialization Phase PLA application developer Analyses & determines the type of specializations applicable FOCUS transformation engine selects the appropriate transformations & uses the annotations to automate specializations FOCUS addresses specialization challenges by building specialization language, tool, & process to capture and automate middleware specializations

12. FOCUS Motivation FOCUS design motivated by template code-generation approaches Jostraca (www.jostraca.org) and Java Emitter Templates (www.eclipse.org) Both have code generation directives in the form of meta-data, e.g. Hello, World FOCUS approach augments template-based approaches by: –Enhancing directives: Enables placing meta-data within code directly –Supporting Code Evolution: FOCUS allows specializations and code to evolve in parallel minimizing skew –Minimizing accidental complexity: FOCUS allows ease of specialization specification via XML, does not require understanding of complicated specialization language syntax Transformational approaches: Stratego-XT ( www.program- transformations.org/stratego ), DMS ( www.semdesigns.com ) support a broader range of specializations to be specified in abstract and concrete language syntax The goal of this work isn’t on specialization languages per se, but instead to quantify the improvements that come from applying various types of specializations to middleware

13. FOCUS Goals Generated code should work across different compilers, platforms, etc. Should not rely on external libraries Specialized code should not incur performance/foot-print overhead due to the specialization Shield middleware developers from tool maturity issues Toolkit based on COTS components and technologies Work across different middleware implementations in Java, C++, etc Support middleware specialization with evolution Extensible, easy to use; minimize accidental complexities in developing specializations Should not be bothered about actual specialization details

14. FOCUS Dynamics Feature Selection FOCUS Customization Engine: Transforms annotations Source-to-Source Transformations PLA application developer selects the features to be specialized Perl-based FOCUS transformation engine selects the appropriate specialization file from the repository & executes the specialization C++ or Java compiler

15. Presentation Road Map Technology Context Research Challenges Middleware Specialization Techniques Feature Oriented Customizer (FOCUS) for Middleware Applying FOCUS to PLAs & Middleware – A Case Study Concluding Remarks & Future Work

16. Bold Stroke PLA Scenario Example System Basic Single Processor (BasicSP) –distributed real-time & embedded (DRE) application scenario based on Boeing Bold Stroke Timer Component – triggers periodic refresh rates GPS Component – generates periodic position updates Airframe Component – processes input from the GPS component & feeds to Navigation display Navigation Display – displays GPS position updates ACE_wrappers/TAO/CIAO/DaNCE/examples/BasicSPCoSMIC/examples/BasicSP Representative DRE application: “rate based” Events  Control information Operation  simple data

17. Identifying Ahead of Time Known System Invariance Single method interfaces: Sends same operation on wire A specific Reactor used Protocol: A specific protocol used Specification Invariance Framework Invariance Deployment Platform Invariance Do not support native exceptions 1 2 3

18. Framework Specialization Reactor Pattern Reactor pattern separates the event detection from the demultiplexing Allows a single-thread to do both activities Multiple Reactor implementations; need only one! Reactor Specialization Remove indirection, e.g., Reactor_Impl base class completely (all virtual methods concrete) No changes to component interface. Does not break compatibility Invariance Middleware components such as Reactor, protocol, components remain same for a PLA scenario Select Reactor select () Reactor_Impl select () WFMO_Reactor select () Thread_Pool Reactor select () This specialization is an example of Aspect Weaving specialization, where crosscutting middleware features are customized The resolution of this challenge illustrates how middleware frameworks can be specialized

19. Capturing Specialization in FSL (1/2) FOCUS uses an XML DTD to create a DSL for capturing specializations –Use Reactor specialization as an example to bring out FSL language features Tags used to represent different transformations –  Substitution capability.. … Capability to search & replace (supports regular expressions within tags) –  Remove code within the tag (supports regular expressions) ACE_Reactor_Impl ACE_Select_Reactor_Impl virtual These tags do not require any annotations to the middleware source code Removes indirection in the Reactor framework Once indirections is removed, virtualness can be eliminated

20. Capturing Specialization in FSL (2/2) –  Add code based on a “hook” Annotated point within middleware where code needs to be woven Code to be woven at that point REACTOR_SPL_INCLUDE_HOOK #include "ace/Select_Reactor.h" –  Comment a region out between hooks Annotations that define where to start & stop copy TAO_REACTOR_SPL_COMMENT_HOOK_START TAO_REACTOR_SPL_COMMENT_HOOK_END Bring in concrete Reactor declarations into the base Reactor component Eliminate code that creates a “vanilla Reactor component” Annotations help identify join points & relieve FOCUS from implementing a full fledged language parser FOCUS is language agnostic, i.e., can be written in any language

21. ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; //@@ TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } //@@ TAO_REACTOR_SPL_COMMENT_HOOK_END } Middleware Annotations Middleware developer annotates middleware with hooks Hooks represented as “meta-comments,” i.e., opaque to compilers Instrumenting Middleware Implementation 11: 12: 13: 14: TAO_REACTOR_SPL_COMMENT_HOOK_START 15: TAO_REACTOR_SPL_COMMENT_HOOK_END 16: 17: 18: 23:....

22. FOCUS Specialization Phase //File: Reactor.h //@@ REACTOR_SPL_INCLUDE_HOOK // Code woven by FOCUS: #include "ace/Select_Reactor.h" // END Code woven by FOCUS class Reactor { public: int run_reactor_event_loop (); // Other public methods.... private: // Code woven by FOCUS: ACE_Select_Reactor *reactor_impl_; // End Code woven by FOCUS //...... }; //File: Reactor.h //@@ REACTOR_SPL_INCLUDE_HOOK class Reactor { public: virtual int run_reactor_event_loop (); // Other public methods.... private: ACE_Reactor_Impl *reactor_impl_; //...... }; Annotated middleware sourceTransformed middleware source These specializations don’t affect framework “business logic”, just its structure Existing framework is still available when developers need OO capabilities Removed virtual keyword Replaced ACE_Reactor_Impl with ACE_Select_Reactor #include concrete header file

23. Aspect Weaving Specialization Results Experiment –Client/server communicate by sending a simple long –End-to-end latency measures for: General-purpose optimized TAO and Reactor, Protocol frameworks specialized with FOCUS compared Result Synopsis Average latency measures improve by ~5% for both protocol and reactor specializations Average Measures: Reactor 4.5% Protocol: 5% Jitter improves though not considerably Jitter better; specialization does not compromise predictability 99% values closer to average and worst-case values are better indicating better predictability Worst case measures improve by ~50 µsecs Specialization also applied to other frameworks in TAO: Messaging, Wait Strategies

24. Scenario Invariance Specialization Specialize Middleware Paths Create middleware fast paths based on different invariants Normal layered path Optimized Fast path processing Request Processing Fast Paths Normal layered path  Uses general-purpose optimization for request lookup Optimized fast path  Bypasses middleware layers to directly perform operation invocation Invariance The same operation is invoked on a given connection This specializations is an example of Memoization + Layer-folding specialization, where a pre-computed result is cached and used to bypass middleware layers Resolution of this challenge illustrates how specification imposed functionality can be specialized

25. Consequences Specialization compliant with CORBA POA Current not maintained Default_Servant and Servant_Manager POA policies not honored with this optimization –Rarely used with DRE applications Automating Dispatch Resolution Specialization Automating Specialization via FOCUS On a per connection basis determine the target dispatcher; For subsequent requests, directly use the dispatcher to process requests TAO_DISPATCH_OPT_ADD_HOOK 1. if (__dispatch_resolved__) 2.{ 3: //First invocation normal path 4: __dispatch_resolved_ = 0; 5: } 6: else 7:{ // All invocations go through --- Optimized path 8: this->request_disaptcher__.dispatch (…) 9:} FSL’s add capability leveraged Normal path; the dispatch is cached in the process Directly use cached dispatcher FOCUS enables specializations to be specified in the corresponding language syntax

26. Results for Layer-folding Specialization Average end-to-end measures improved by ~16% Average path measures improved by ~40% Worst case path measure improved by ~20% Worst case end-to- end latency improved by ~14% Dispersion improves by a factor of ~1.5 for both cases Path specialized latency measures Path defined as code- path when a request is received until the upcall is dispatched on the skeleton Experiment –End-to-end latency measures for: General-purpose optimized TAO with active demultiplexing and perfect hashing Specialized TAO with layer folding specialization enabled Specialization applied at the server side; this optimization can be applied at the client side as well

27. Deployment Platform Specializations Invariance Endianess of the platforms remains the same for the PLA scenario Deployment Platform Generality Platform characteristics affect QoS –TANGO  Timesys linux + gcc –ACE  vxWorks + Greenhills compiler Certain versions of vxWorks do not support exceptions Specialize Middleware for Deployment platform Run benchmark tests and programs automatically using autoconf before ORB initialization to determine right hardware, OS and compiler characteristics This specialization is a variation of Constant Propagation where the constant (platform characteristic) is used to tailor the middleware The resolution of this challenge illustrates how deployment generality can be resolved to improve QoS

28. Automating Platform Specializations Deployment platform specialization –Exception support: Autoconf’s AC_COMPILE_IF_ELSE macro to check if compiler supports native exception –Loop unrolling optimization Autoconf’s AC_RUN_IF_ELSE macros to compile and run benchmark to quantify memcpy performance improvements ACE_CACHE_CHECK([if ACE memcpy needs loop unrolling], [ace_cv_memcpy_loop_unroll], [AC_RUN_IFELSE([AC_LANG_SOURCE([[ /// …. Program that will run the benchmark to //// determine if an optimization is better Variable that will be checked to set a compilation flag Directive to run the benchmark Provide the actual source language that will be compiled and run This approach can be applied to automatically discover OS specific system calls and compiler settings that maximize QoS

29. Results for Autoconf Specialization Deployment platform specialization –Exception support: –Loop unrolling optimization –During ORB configuration, autoconf used to use emulated exceptions and un-roll loops for memcpy Average latency improves ~17% Predictability improved over general purpose middleware Worst-case measures improve by ~50 µs Other types of optimization include sendfile optimization on Unix and Linux platforms to avoid copy between TCP and application buffers

30. Specialization Catalog Client Side Specialization Request Header Caching Pre-creating Requests Target Location Avoid demarshaling checks Server Side Specialization Specialize Request Processing Avoid aligning data types while marshaling ORB Component Specialization Multiple types of Reactor Pluggable protocol framework www.dre.vanderbilt.edu/~arvindk/MW-Spl.pdf

31. Cumulative Specialization Results Average end-to-end measures improved by ~43% Jitter results twice as better as general- purpose optimized TAO Worst-case measures improved by ~45% End-to-end client side throughput improved by ~65%. Results are notable as base line used is already optimized general-purpose TAO implementation

32. FOCUS Strengths & Drawbacks Strengths FOCUS provides a lightweight, zero (run- time) overhead approach for customizing middleware FOCUS is designed to work across different languages (Java, C++) XML-based approach to capture rules –Easy language extension, ability to add new features easily –If/when C++ aspect technologies mature, can transform them into aspect rules via XSLT transforms Execute transformations via scripting –Perl regular expression capability really mature for our transformation –Ease of automation – integrate with other quality assurance techniques Drawbacks Does not provide full-fledged language parser, i.e., join points need to be identified via annotations or via regular expressions Static compile-time specialization; Requires recompilation of source code –Aspect C++, DMS use this approach The need to synchronize annotations with specialization files; Modification of source code requires change to specialization files –Ameliorated with QA techniques by having continuous builds; Limitation exists even with aspects Correctness of transformations have to be validated externally; unlike AspectJ FOCUS soon available in the latest ACE+TAO distribution in ACE_wrappers/bin directory. Temporary location: www.dre.vanderbilt.edu/~arvindk/FOCUS

33. Presentation Road Map Technology Context Research Challenges Middleware Specialization Techniques Feature Oriented Customizer (FOCUS) for Middleware Applying FOCUS to PLAs & Middleware – A Case Study Concluding Remarks & Future Work

34. Concluding Remarks (1/2) Implications on QoS –Specializations applied such that no dependencies/conflicts exists –As more specializations are developed it is necessary to document dependencies between specializations Implications on ORB maturity –Optimizations improve QoS over and above general-purpose optimizations –Other ORBs, e.g., JacORB, do not use such optimizations; performance gains in this case would be more –Preliminary results [Daugherty] on ZEN show that speed up was better compared to TAO TAO uses active demultiplexing and perfect hashing for O(1) demux time [Daugherty] Gary Daugherty, A Proposal for the Specialization of HA/DRE Systems, ACM PEPM 04

35. Concluding Remarks (2/2) Implications on Schedulability Specializations help high priority tasks finish ahead of their time to complete –Tasks with priority 50 finish early, increasing time available for scheduling priorities with 35 and 10 Specializations can benefit both hard real- time and soft and softer real-time tasks Implications on Adaptability Specializations do not consider any form of recovery if invariance assumptions fail Adaptation requires loading general-purpose code, add checks along request processing path; increases jitter for DRE systems Implications on PLA Scenario All our specializations improve path latencies considerably than end-to-end latency; More the specialized code path is traversed, greater is QoS improvement POA S1S2 POA S3 20 S4 35 POA S5 40 S5 50 Specializations increase slack in the system

36. Future Work Middleware implementations accommodate variability by using patterns Greater benefits can be accrued by applying specialization approaches to other middleware frameworks; locking, buffering and concurrency strategies in middleware. http://www.cs.wustl.edu/~schmidt/POSA/ Future research will also examine applicability of specializations to Component based middleware implementations

37. Questions

38. FOCUS Instrumentation Phase (3/3) Transformation preserves the hooks, enables multiple specializations to use the same hook Minimizes clutter in middleware source code Hook names need not have any meaning Differs from Javadoc tag approach A drawback of this approach is the need to synchronize tags with specialization files However, same actor (i.e., middleware developer) maintains both entities ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; //@@ TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } //@@ TAO_REACTOR_SPL_COMMENT_HOOK_END } TAO_REACTOR_SPL_COMMENT_HOOK_END ACE_NEW_RETURN (impl, ACE_Select_REACTOR ((ACE_Sig_Handler*)0, (ACE_Timer_Queue*)0, 0, (ACE_Select_Reactor_Notify*)0, this->reactor_mask_signals_), 0); Weave code that creates Select_Reactor instead of a normal Reactor

39. FOCUS Specialization Language (FSL) –  Copy code from a specific point to a specific point between different files  hook from where to start copying  hook to stop copying  destination where you want the code to be placed Aids copying derived class specializations to base class Select_Reactor.h ACE_REACTOR_SPL_COPY_HOOK_START ACE_REACTOR_SPL_COPY_HOOK_END ACE_REACTOR_SPL_METHODS_ADD_HOOK Processing of the specialization tags is done in the following order:,,,, & Order can be configured to support different parsing mechanisms Enables middleware developers to define & enforce an ordering of how the individual transformations will be carried out Copies *additional* methods from the derived class onto base class

40. FOCUS: Termination Criteria Termination Criteria (Hypothesis) We will use TAO mature real-time ORB as our gold standard Greatest benefits will be accrued for scenarios that can turn on all/most of the optimizations –Performance improvements estimated ~ 30 – 40 % improvement in performance Turning on just one/two optimizations might improve performance by ~ 10 – 15 % improvement TAO baseline General-purpose optimized TAO ORB Active demultiplexing and perfect hashing for O(1) request demultiplexing Buffer caching and direct collocation optimization Most optimized configuration for different ORB components Experiment Setup A single threaded client and server; simple long exchanged Pentium III 850 Mhz processor, running Linux Timesys 2.4.7 kernel, 512 MB of main memory, TAO version 1.4.7 compiled with gcc 3.2.3 Timers at the client and within ORB used to collect data

41. FOCUS Instrumentation Phase (1/3) Specialization transformations Middleware developers use FSL to define transformations Portion of XML file for specializing TAO Reactor implementation Lines 1 – 2: module, package, & file to specialize Lines 3 – 8: add necessary forward declaration at a hook in middleware Lines 13 – 16: comment out unspecialized code within the markers 1: 2: 3: 4: REACTOR_SPL_INCLUDE_HOOK 5: class ACE_Sig_Handler; 6: class ACE_Sig_Action; 7: class ACE_Sig_Set; 8: 9: 10: 11: 12: 13: 14: TAO_REACTOR_SPL_COMMENT_HOOK_START 15: TAO_REACTOR_SPL_COMMENT_HOOK_END 16: 17: 18: 23:.... Annotations help identify join points & relieve FOCUS from implementing a full fledged language parser FOCUS is language agnostic, i.e., can be written in any language Similar to listing, package; AOP defines class directive Mimics AOP code weaving at join-points. AOP uses regex to identify points AOP around () advice can be employed; incurs run-time overhead though

42. Challenge Overview (1/2) Model Driven Development (MDD) approaches capture PLA characteristics –Functional: Type of protocol, concurrency strategy –QoS: end-to-end latencies, jitter, throughput Model-Driven Technologies Domain-Specific Modeling Languages Meta-data Deployment and Configuration information Meta-data Middleware still general-purpose, i.e., provides code for supporting all applications Ahead of time system information is not used to customize the middleware

43. Challenge Overview (2/2) Model-Driven Technologies Domain-Specific Modeling Languages Middleware Specialization Challenge How can the middleware be tailored for the specific PLA architecture modeled? How can middleware level optimizations identified and automated based on PLA system characteristics? Research addresses this specialization challenge via Domain Specific Language, tools and process for specializing middleware for PLA

44. Feature-Oriented CUStomizer (FOCUS) Goals Requirements for middleware specialization framework: Generated code should work across different compilers, platforms, etc. Should not rely on external libraries – Aspect J Specialized code should not incur performance/foot-print overhead due to the specialization AspectJ [Zhang] results revealed that performance suffered from overhead of aspect libraries Shield from tool maturity issues AspectC++ does not work with TAO; recently made to work with ACE Toolkit based on COTS components and technologies that are also open- source Work across different middleware implementations in Java, C++, etc. Support specialization with evolution, i.e., open-source models [Zhang] “Resolving Feature Convolution in Middleware”, OOPSLA Conference, Vancouver, 2004 FOCUS Goals: Build specialization language, tool, & process to automate middleware specializations across different platforms