1. Copyright 2003 National ICT Australia Limited 1 Mining Patterns to Support Software Architecture Evaluation 4 th Working IEEE/IFIP Conference on Software Architecture Oslo, Norway 12-15 th June Liming Zhu, Muhammad Ali Babar, Ross Jeffery School of Computer Science, University of New South Wales Empirical Software Engineering Group, National ICT Australia

2. Copyright 2003 National ICT Australia Limited2 Presentation Outline Background: Architecture Evaluation and Architecture Patterns Current Problems and Research Goals Extracting Scenarios and Tactics from Patterns A Case Study – Architectural-level Performance Evaluation on “Stock-Online” system Conclusion

3. Copyright 2003 National ICT Australia Limited3 Background Scenario-based architecture evaluation (ALMA, ATAM, PASA etc.) Architecture patterns More and more large and complex software systems have been developed using architectural patterns

4. Copyright 2003 National ICT Australia Limited4 Research Goals Problems: Useful information in patterns is not used effectively and efficiently in architecture evaluation Scenario generation Reasoning framework selection and calibrating Patterns as “best practices” are not validated Research Goals: Useful information in pattern can be extracted and re-organized to improve architecture evaluation To improve scenario quality To help to select and calibrate quality reasoning framework By following the above method, an architecture evaluation result on a pattern-based architecture can qualify as one of the many pieces of evidence for validating the pattern. (Evidence-based Software Engineering)

5. Copyright 2003 National ICT Australia Limited5

6. 6 Extracting Scenarios A six elements framework [Bass, Clements and Kazman 03] Stimulus, Response, Source of stimulus, Environment, Stimulated Artifact, Response measure A general scenario extracted from J2EE Data Mapper or DTO (Data Transfer Object) pattern A periodic data structure change request (stimulus) from stakeholders (source of the stimulus) arrives when data use case changes after the system is deployed (environment). The system (stimulated artifact) has to be modifiable (response) according to the data structure change request within certain scope under certain time and maintenance cost (response measure).

7. Copyright 2003 National ICT Australia Limited7 Extracting Tactics Extracting Architectural Tactics: Providing information on independent parameters and reasoning framework Existing preliminary tactic framework: [Bass, Bachmann and Klein 03] [Bosch 04] Example: Tactics from DTOFactory J2EE pattern Tactic 1: Managing demands by managing event rate Independent parameters: number of requests Tactic 2: Localizing expected modification by limiting change options, isolating expected changes and maintaining semantic coherence Independent parameters: number of primary components affected

8. Copyright 2003 National ICT Australia Limited8 A Case Study System: Stock-Online System [Gorton 00] Typical online stock trading system Alternative design decisions Architecture-level Performance prediction (QNM-based) [Liu, Fekete and Gorton 04] Pattern: Read Mostly (RM) Pattern Platform: J2EE

9. Copyright 2003 National ICT Australia Limited9 A Case Study

10. Copyright 2003 National ICT Australia Limited10 Mining the RM Pattern Extracting scenarios from RM pattern General Scenario: A large number of client (stimulus source) requests, mostly read-only requests (stimulus) arrive at the server at normal condition (environment), the server (artifact) needs to process the requests (response) within certain response time (response measure) Concrete Scenario: A large number of stock transaction related requests (get/set/create accout/stockholders/stockitems/stockTx, 83.7% of them, read-only requests) arrive at the server at normal condition, the server needs to process the request within certain response time. Extracting tactics from RM Pattern Managing demands by reducing event rate Independent parameter: number of requests on resource intensive activities This will shorten the total response time for a large number of read-only requests

11. Copyright 2003 National ICT Australia Limited11 Selecting and Calibrating Reasoning Framework R1: number of requests related to T1: service time to synchronize with database R2: number of requests related to T2: service time to access entity bean p: read only requests ratio hr: cache hit ratio [Yan et. al 04] Introducing p as one of the independent parameters in reasoning framework helping to make design decisions validating “claimed” pattern benefits

12. Copyright 2003 National ICT Australia Limited12 Architecture Evaluation A benchmark application is built with the RM pattern Concrete scenarios used in the measurement are instances of general scenarios extracted from the RM pattern The independent parameter manipulated by the tactic is modelled in the reasoning framework Reasoning framework parameters are solved and populated through experimental measurements on the benchmark application Evaluation results (prediction) for the target application are in favour of RM pattern and can be explained by the read requests ratio - p Evaluation results are validated against target application

13. Copyright 2003 National ICT Australia Limited13 Conclusion Extracting information from pattern to improve architecture evaluation Accumulating evidence to validate pattern Deriving concrete scenario from extracted general scenario Modelling pattern tactics (independent parameters a pattern is trying to manipulate) in reasoning framework General scenarios Architecture tactics Concrete scenarios Quality reasoning framework Architectural Patterns