1. Response Time Analysis of a Middleware Demultiplexing Pattern for Network Services Aniruddha Gokhale a.gokhale@vanderbilt.edu a.gokhale@vanderbilt.edu Asst. Professor of EECS, Vanderbilt University, Nashville, TN Swapna Gokhale ssg@engr.uconn.edu Asst. Professor of CSE, University of Connecticut, Storrs, CT ssg@engr.uconn.edu Presented at IEEE Globecom 2005 Symposium on Advances in Networks and Internet St. Louis MO Nov 28-Dec 1, 2005 Work supported by collaborative grant from NSF CSR-SMA Program Jeff Gray gray@cis.uab.edu Asst. Professor of CIS Univ. of Alabama at Birmingham Birmingham, AL

2. 2 Problem Statement: Estimating Performance Characteristics of Network Services at Design-time  Standards middleware is increasingly being used to develop network services e.g., J2EE,.NET, CORBA, Web services  Middleware frameworks incorporate elegant patterns-based building blocks  Problem boils down to estimating performance of middleware.e.g., VPN Service provided by a virtual router Provides differentiated services to customers, e.g., prioritized service VPN setup messages must be efficiently (de) multiplexed, serviced and forwarded Need to estimate capacity of the system at design-time  Network services need support for efficient (de)-multiplexing, dispatching and routing/forwarding

3. 3 Solution Approach: Middleware Performance Analysis using Stochastic Reward Nets Stochastic Reward Nets (SRNs) are an extension to Generalized Stochastic Petri Nets (GSPNs) which are an extension to Petri Nets. Extend the modeling power of GSPNs by allowing: Guard functions Marking-dependent arc multiplicities General transition probabilities Reward rates at the net level Allow model specification at a level closer to intuition. Solved using tools such as SPNP (Stochastic Petri Net Package). N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b) Transition Place Immediate transition Inhibitor arc Token

4. 4 Goal: Performance Analysis of Reactor Pattern in VR The Reactor architectural pattern allows event-driven applications to demultiplex & dispatch service requests that are delivered to an application from one or more clients. Customers send VPN setup messages to router VPN setup messages manifest as events at the VR VR must service these events (e.g., resource allocation) and honor the prioritized service, if any Accepted messages are forwarded Events could be dropped in overload conditions Reactor pattern decouples the detection, demultiplexing, & dispatching of events from the handling of events Participants include the Reactor, Event handle, Event demultiplexer, abstract and concrete event handlers

5. 5 Modeling VR Capabilities in a Reactor Consider VPN service for two customer classes  Reactor accepts and handles two types of input events Differentiated services for two classes  Events are handled in prioritized order Each event type has a separate queue to hold the incoming events. Buffer capacity for events of type one is   and of type two is  . Event arrivals are Poisson for type one and type two events with rates   and  resp. Event service time is exponential for type one and type two events with rates    and  2, resp. Model of a single-threaded, select- based reactor implementation

6. 6 Performance Metrics of Interest for VR (i.e., Reactor) Throughput: -Number of events that can be processed -Applications such as telecommunications call processing. Queue length: -Queuing for the event handler queues. -Appropriate scheduling policies for applications with real-time requirements. Total number of events: -Total number of events in the system. -Scheduling decisions. -Resource provisioning required to sustain system demands. Probability of event loss: -Events discarded due to lack of buffer space. -Safety-critical systems. -Levels of resource provisioning. Response time: -Time taken to service the incoming event. -Bounded response time for real-time systems.

7. 7 Modeling the Reactor using SRN (1/2) Models arrivals, queuing, and prioritized service of events. Transitions A1 and A2: Event arrivals. Places B1 and B2: Buffer/queues. Places S1 and S2: Service of the events. Transitions Sr1 and Sr2: Service completions. Inhibitor arcs: Place B1and transition A1 with multiplicity N1 (B2, A2, N2) - Prevents firing of transition A1 when there are N1 tokens in place B1. Inhibitor arc from place S1 to transition Sr2: - Offers prioritized service to an event of type one over event of type two. - Prevents firing of transition Sr2 when there is a token in place S1. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b) Event arr. Service queue Servicing the event Drop events on overflow Prioritized service Service completion

8. 8 Modeling the Reactor using SRN (2/2) N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b) Process of taking successive snapshots Reactor waits for new events when currently enabled events are handled Sn1 enabled: Token in StSnpSht & Tokens in B1 & No Token in S1. Sn2 enabled: Token in StSnpSht & Tokens in B2 & No Token in S2. T_SrvSnpSht enabled: Token in S1 and/or S2. T_EndSnpSht enabled: No token in S1 and S2. Sn1 and Sn2 have same priority T_SrvSnpSht lower priority than Sn1 and Sn2

9. 9 VR SRN: Performance Estimates SRN model solved using Stochastic Petri Net Package (SPNP) to obtain estimates of performance metrics. Parameter values:    sec      /sec,    sec      /sec. Two cases : N 1 = N 2 = 1, and N 1 = N 2 = 5. Observations: Probability of event loss is higher when the buffer space is 1 Total number of events of type two is higher than type one. Events of type two stay in the system longer than events of type one. May degrade the response time of event requests for class 2 customers compared to requests from class 1 customers N 1 = N 2 = 1 N 1 = N 2 = 5 Perf. metric #1 #2 Throughput 0.37/s 0.37/s 0.40/s 0.40/s Queue length 0.065 0.065 0.12 0.12 Total events 0.25 0.27 0.32 0.35 Loss probab. 0.065 0.065.00026.00026

10. 10 VR SRN: Sensitivity Analysis Analyze the sensitivity of performance metrics to variations in input parameter values. Vary  from 0.5/sec to 2.0/sec. Values of other parameters:      /sec,    sec      /sec, N 1 = N 2 = 5. Compute performance measures for each one of the input values. Observations: Throughput of event requests from customer class #1 increases, but rate of increase declines. Throughput of event requests from customer class #2 remains unchanged.

11. 11 Next Steps: Addressing Variability in Middleware  Per Building Block Variability – Incurred due to variations in implementations & configurations for a patterns-based building block – E.g., single threaded versus thread-pool based reactor implementation dimension that crosscuts the event demultiplexing strategy (e.g., select, poll, WaitForMultipleObjects Although middleware provides reusable building blocks that capture commonalities, these blocks and their compositions incur variabilities that impact performance in significant ways.  Compositional Variability – Incurred due to variations in the compositions of these building blocks – Need to address compatibility in the compositions and individual configurations – Dictated by needs of the domain – E.g., Leader-Follower makes no sense in a single threaded Reactor

12. 12 Composed System Next Steps: Model-driven Performance Analysis of Middleware-based Network Services  Build and validate performance models for invariant parts of middleware building blocks  Weaving of variability concerns manifested in a building block into the performance models  Compose and validate performance models of building blocks mirroring the anticipated software design of DPSS systems  Estimate end-to-end performance of composed system  Iterate until design meets performance requirements Applying design-time performance analysis techniques to estimate the impact of variability in middleware-based DPSS systems Invariant model of a pattern Refined model of a pattern variability weave Refined model of a pattern workload system

13. 13 Concluding Remarks  Network services are implemented using middleware building blocks  Need to estimate performance early in development lifecycle  Stochastic Reward Nets enables scalable & intuitive performance analysis  Goal is to use model-driven generative techniques to automatically synthesize performance models for network services  Analysis for other dimensions of quality of service e.g., trustworthiness, dependability www.cse.uconn.edu/~ssgwww.cse.uconn.edu/~ssg (Swapna Gokhale) www.dre.vanderbilt.edu/~gokhalewww.dre.vanderbilt.edu/~gokhale (Aniruddha Gokhale) www.gray-area.orgwww.gray-area.org (Jeff Gray)

14. Questions?

15. EXTRAS

16. 16 Reactor Dynamics  Registration Phase – Event handlers register themselves with the Reactor for an event type (e.g., input, output, timeout, exception event types) – Reactor returns a handle it maintains, which it uses to associate an event type with the registered handler  Snapshot Phase – Main program delegates thread of control to Reactor, which in turn takes a snapshot of the system to determine which events are enabled in that snapshot – For each enabled event, the corresponding event handler is invoked, which services the event – When all events in a snapshot are handled, the Reactor proceeds to the next snapshot

17. 17 Reactor SRN: Taking a Snapshot Part B: Process of taking successive snapshots Sn1 enabled: Token in StSnpSht & Tokens in B1 & No Token in S1. Sn2 enabled: Token in StSnpSht & Tokens in B2 & No Token in S2. T_SrvSnpSht enabled: Token in S1 and/or S2. T_EndSnpSht enabled: No token in S1 and S2. Sn1 and Sn2 have same priority T_SrvSnpSht lower priority than Sn1 and Sn2 N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

18. 18 Reactor SRN: Initial Marking Initial marking: StSnpSht = 1, B1 = 2, B2 = 2, S1 = 0, S2 = 0 Transitions enabled: Sn1 and Sn2 Sn1 fires. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

19. 19 Reactor SRN: Firing a Transition (1/6) Upon firing of Sn1: StSnpSht = 1, B1 = 1, B2 = 2, S1 = 1, S2 = 0 Transitions enabled: Sr1, Sn2, T_SrvSnpSht Sn2 and T_SrvSnhpSht are immediate transitions, have to fire before Sr1. T_SrvSnpSht has a lower priority over Sn2. Sn2 fires. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

20. 20 Reactor SRN: Firing a Transition (2/6) Upon firing of Sn2: StSnpSht = 1, B1 = 1, B2 = 1, S1 = 1, S2 = 1 Transitions enabled: Sr1, T_SrvSnpSht T_SrvSnhpSht is an immediate transition, has to fire before Sr1. T_SrvSnpSht fires. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

21. 21 Reactor SRN: Firing a Transition (3/6) Upon firing of T_SrvSnpSht: TSnpShtInProg = 1, B1 = 1, B2 = 1, S1 = 1, S2 = 1 Transitions enabled: Sr1 Snapshot in progress. Sr1 fires. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

22. 22 Reactor SRN: Firing a Transition (4/6) Upon firing of Sr1: TSnpShtInProg = 1, B1 = 1, B2 = 1, S1 = 0, S2 = 1 Transitions enabled: Sr2 Snapshot in progress Sr2 fires. N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

23. 23 Reactor SRN: Firing a Transition (5/6) Upon firing of Sr2: TSnpShtInProg = 1, B1 = 1, B2 = 1, S1 = 0, S2 = 0 Transitions enabled: T_EndSnpSht End of snapshot T_EndSnpSht fires N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

24. 24 Reactor SRN: Firing a Transition (6/6) Upon firing of T_EndSnpSht: StSnpSht = 1, B1 = 1, B2 = 1, S1 = 0, S2 = 0 Transitions enabled: Sn1 and Sn2 Back to initial state N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b)

25. 25 Reactor SRN: Performance Measures Reward rate assignments to compute performance measures N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b) Throughput: Rate of firing of transitions Sr1 (Sr2). Queue length: Number of tokens in place B1 (B2). Total number of events: Sum of the tokens in places B1 & S1 (B2 & S2) Probability of event loss: Number of tokens in place B1 == N1 (B2 == N2) Response time: Can be obtained using the tagged customer approach. SRN model solved using Stochastic Petri Net Package (SPNP) to obtain estimates of performance metrics.

26. 26 Designing & Evaluating SRNs for Network Services N1 N2 A1 A2 B1 B2 Sn1 Sn2 S2 S1 Sr1 Sr2 StSnpSht SnpShtInProg T_SrvSnpSht T_EndSnpSht (a)(b) Initial Step Obtain performance measures for individual patterns-based building blocks Iterative Algorithm Compose systems vertically and horizontally to form a DPSS system Determine performance measures for specified workloads and service times Alter the configurations until DPSS performance meets specifications.

27. 27 VR SRN: Disruption Detection Obtain an anomaly score for the Reactor based on each one of the performance metrics for each event type. Correlate the anomaly scores based on each event type to obtain an overall anomaly score for the Reactor. - Anomaly score for the Reactor used at each CE to demultiplex events from two groups within a single organization. Anomaly score for the Reactor in the VR used to demultiplex events from the two organizations. Correlate the anomaly score of the Reactor in the VR with the score of the Reactor in CE #1 to determine service disruptions for organization #1. Correlate the anomaly score of the Reactor in the VR with the score of the Reactor in CE #2 to determine service disruptions for organization #2. Source of disruption may be identified by correlating the scores at various layers.

28. 28 Collaborative Research  Performance analysis methodology (UConn – S. Gokhale) – Develop and validate performance models for invariant characteristics of building blocks. – Compose and validate performance models for common building block compositions. – Develop model decomposition and solution strategies to alleviate state- space explosion issue.  Model-driven generative methodology (Vanderbilt – A. Gokhale) – Manually developing performance models of each block with its variations is cumbersome – Compositions of building blocks cannot be made in ad hoc, arbitrary manner – Model-driven generative tools use visual modeling languages and model interpreters to automate tedious tasks and provide “correct-by- construction” development  Aspect-oriented methodology (Univ of Alabama, Birmingham – J. Gray) – Variability in building blocks and compositions is a primary candidate for separating the concern as an aspect – Aspect weaving technology can be used to refine and enhance the models by weaving in the concerns into the performance models

29. 29 VR SRN: Expected Behavior VPN service has two modes of operation: normal & inclement. Normal mode: - Daily basis, some employees have negotiated telecommute plans and use VPN for remote access. Inclement mode: - Hazardous driving conditions due to bad weather may keep people at home. - Large number of telecommuters - Increase in the connection set up and tear down requests. Modes of operation can be defined at a finer level of granularity, such as a few hours, rather than a day.

30. 30 VR SRN: Expected Behavior Normal mode: -   sec      /sec,    sec      /sec, N 1 = N 2 = 5 - Probability – 0.9 Inclement mode: -   sec      /sec,    sec      /sec, N 1 = N 2 = 5 - Probability – 0.1 Perf. Metric Normal Inclement Average Event #1 Throughput 0.40/s 0.90/sec 0.4510/s Queue length 0.12 1.86 0.2940 Loss probab. 0.09 0.21 0.0291