1. A Framework for Iterative, Interactive Analysis of Agent-Goal Models in Early Requirements Engineering (Research Proposal) Jennifer Horkoff 1 Eric Yu 2 Department of Computer Science 1 Faculty of Information 2 jenhork@cs.utoronto.cajenhork@cs.utoronto.ca yu@ischool.utoronto.cayu@ischool.utoronto.ca University of Toronto June 7, 2010 iStar’10

2. Early Requirements Engineering  Early stages of requirement analysis focusing on understanding: Stakeholders and systems Stakeholder’s needs Domain problems Different views of the problems  Challenges in Early RE Incomplete and imprecise information Difficult to quantify or formalize critical success criteria such as privacy, security, employee happiness, customer satisfaction Ideally Early RE should involve a high degree of stakeholder interaction  Gather and validate information 2 Framework for Analysis of Agent-Goal Models - Horkoff, Yu

3. Existing Approaches for Early RE  Example: Soft System Methodology (rich pictures) (Checkland, 2000)  Example: Text (or tables) Flexible, user-friendly, but difficult to systematically analyze or support via tools  Examples courtesy of RE’09 “Next Top Model” Competition Framework for Analysis of Agent-Goal Models - Horkoff, Yu 3

4. Existing Approaches for Early RE  Goal- and Agent-Oriented Models (GORE) (agent-goal models) Can allow modelers to model fuzzy concepts (softgoals) Provide useful views even over incomplete and imprecise information Allow for systematic analysis; however:  Existing analysis procedures often require specific information such as probabilities, costs, priorities, or quantitative estimates  (Giorgini et al., 2004), (Franch, 2006), (Letier & van Lamsweerde, 2004), (Amyot et al., 2010), (Bryl et al., 2007), (Gans et al., 2004), (Fuxman et al., 2004), etc.  Claim: Quantitative results are often based strongly on estimates, which are especially unsure during early stages  Most procedures are fully automated “push-button”-type  Claim: Difficult for stakeholders to understand or trust results produced automatically over incomplete and imprecise information Framework for Analysis of Agent-Goal Models - Horkoff, Yu 4

5. Research Objectives  Need: Methods and tools to support Early RE elicitation and analysis which: Are simple enough (on the surface) to use with stakeholders Are structured enough to:  provide user guidance  allow for systematic analysis  allow for tool support But are flexible enough to allow for:  representation of imprecise and incomplete information Allow for incomplete and imprecise information to be supplemented by domain knowledge Prompts iteration over domain knowledge  Increasing the likelihood of discovering objects, problems and alternative designs in the domain Framework for Analysis of Agent-Goal Models - Horkoff, Yu 5 Goal Models GM Analysis

6. Our Approach  A Framework for Iterative, Interactive Analysis of Agent- Goal Models in Early Requirements Engineering  Expand the capabilities of agent-goal models in the following ways: Survey and analysis of existing analysis procedures Interactive forward evaluation Interactive backward evaluation Multiple evaluations over a single model Human judgment management Assumptions and argumentation Supporting model iteration Suggested methodology Framework for Analysis of Agent-Goal Models - Horkoff, Yu 6

7. Survey and Analysis  Many different approaches for agent-goal model analysis Forward and backward satisfaction propagation: (Giorgini et al., 2004), (Amyot et al., 2010), (Letier & van Lamsweerde, 2004)… Metrics: (Franch, 2006)… Planning: (Bryl et al., 2007)… Simulation: (Gans et al., 2004)… Model Checking: (Fuxman et al., 2004)…  Which procedures support what GM syntax?  Which procedures to use in what circumstances? (How do you select among them?)  More specific comparison: what differences do different conventions in forward satisfaction propagation procedures have on the results? Framework for Analysis of Agent-Goal Models - Horkoff, Yu 7

8. Interactive Forward Satisfaction Analysis  Allow “What if?” questions  A question/scenario/alternative is placed on the model and its affects are propagated “forward” through model links  Interactive: user input (human judgment) is used to decide on partial or conflicting evidence “What is the resulting value?  Qualitative: uses a simple qualitative scale  Publications: CAiSE’09 (short paper), PoEM’09, IJISMD (to appear) Framework for Analysis of Agent-Goal Models - Horkoff, Yu 8 Human Judgment

9. Interactive Backward Satisfaction Analysis  Allow “Is this possible?” questions  A question/scenario/constraints are placed on the model and its affects are propagated “backward” through model links Expand SAT formalization in Giorgini et al. (2004) to take into account i* syntax and additional evaluation labels Single resulting label for each intention Iteratively asks for human judgment “What incoming values could produce the target value?”  Publications: istar’08, ER’10 (to appear) Framework for Analysis of Agent-Goal Models - Horkoff, Yu 9 Human Judgment Conflict Backtrack Highlight conflict sources and relevant intentions

10. Multiple Evaluations & Human Judgment Management & Model Iteration  Allow users to manage and compare alternatives over a model Need to allow users to conceptualize, and itemize alternatives, comparing results Works for both forward and backward procedures  Allow users to manage, reuse and change their judgments over the models (Optionally) reuse human judgments, build a DB of judgments per model Perform checks for consistency, make suggestions?  Support model iteration When users change the model or their judgments:  The effects of the change on evaluation results should be displayed  Re-evaluation should be allowed, but only for the results affected by the change Framework for Analysis of Agent-Goal Models - Horkoff, Yu 10

11. Assumptions, Arguments & Suggested Methodology  Allow users to record and use important domain information in the modeling analysis process Capture arguments behind model constructs and evaluation judgments Capture domain assumptions Explore ways to use assumptions and arguments beyond the model  Lists, views, tables, requirement specs  Provide a methodology to guide early modeling and analysis Guidelines for participatory modeling and evaluation:  Where to start, how to come up with useful evaluation questions?  Iterating over models First draft: PoEM’09, IJISMD (to appear) Framework for Analysis of Agent-Goal Models - Horkoff, Yu 11

12. Tool Support: OpenOME Framework for Analysis of Agent-Goal Models - Horkoff, Yu 12

13. Case Studies (Validation)  Application of forward procedure Trusted Computing, Knowledge Management, i* Patterns, Social Service Organization PST’06, HICSS’07, REFSQ’08, CAiSE’09, PoEM’09, IJISMD  Exploratory experiment tested benefits of forward procedure Model iteration, prompted further elicitation, improved understanding Careful examination of model vs. systematic procedure? CAiSE’09, PoEM’09, IJISMD  Expansion of experiment to individual case studies over more subjects using both forward and backward implementation (in progress) Comparison of results using and not using the procedure Initial results show issues in i* knowledge, usability issues in the analysis procedures and the affects of model and domain “buy-in”  Case Studies with groups/organizations: apply implementation of forward and backward procedure Inflo in-house case study (in progress) Later industrial case study (security patterns?) Framework for Analysis of Agent-Goal Models - Horkoff, Yu 13

14. Summary: Scientific Contributions  Early RE Analysis: Allowing analysis over informal, incomplete agent-goal models  Iterative, Interactive Algorithm: Detailed algorithm which iterates, adapting to user input  Model Iteration - Supporting iteration over the model by showing users effects of model and judgment changes  Minimal re-evaluation - after model changes  Multiple Case Studies - Assessing how agent-goal model evaluation can be used in practice with stakeholders through multiple case studies in a variety of settings Framework for Analysis of Agent-Goal Models - Horkoff, Yu 14

15. Thank you  jenhork@cs.utoronto.ca jenhork@cs.utoronto.ca  www.cs.utoronto.ca/~jenhork www.cs.utoronto.ca/~jenhork  yu@ischool.utoronto.ca yu@ischool.utoronto.ca  www.cs.utoronto.ca/~eric www.cs.utoronto.ca/~eric  OpenOME: https://se.cs.toronto.edu/trac/ome Framework for Analysis of Agent-Goal Models - Horkoff, Yu 15

16. Future Work  The suggested framework could be extended to: Support varying levels of qualitative scales Support varying levels of human interaction Tie into “Late” RE analysis using detailed information  Mixture of qualitative and quantitative values (use numbers where you have them) Framework for Analysis of Agent-Goal Models - Horkoff, Yu 16

17. References  Checkland, P.: Soft systems methodology: A Thirty Year Retrospective. Systems Research and Behavioral Science 17, (S1) S11--S58 (2000)  Giorgini, P., Mylopoulos, J., Sebastiani, R.: Simple and Minimum-Cost Satisfiability for Goal Models. In: Persson, A., Stirna, J. (eds) CAiSE 2004. LNCS, vol. 3084, pp. 20-3-5. Springer, Heidelberg (2004)  Franch, X.: On the Quantitative Analysis of Agent-Oriented Models: In Dubois, E., Pohl, K. (eds) CAiSE’06. LNCS, vol. 4001, pp. 495--509. Springer, Heidelberg (2006)  Amyot, D., Ghanavati, S., Horkoff, J., Mussbacher, G., Peyton, L., Yu, E.: Evaluating Goal Models within the Goal-oriented Requirement Language. Int. Journal of Intelligent Systems (IJIS) (2010)  E. Letier, and A. van Lamsweerde, “Reasoning about Partial Goal Satisfaction for Requirements and Design Engineering,” Proc. Int. Symp. on the Found. of SE (FSE’04), ACM Press, 2004, pp. 53-62.  V. Bryl, P. Giorgini, and J. Mylopoulos, “Supporting Requirements Analysis in Tropos: a Planning-Based Approach,” Proc. Pacific Rim Int. Work. on Multi-Agents (PRIMA’07), Springer, 2007, vol. 5044, pp. 243-254.  D. Gans, D. Schmitz, T. Arzdorf, M. Jarke, and G. Lakemeyer, “SNet Reloaded: Roles, Monitoring and Agent Evolution,” Proc. Inter. Bi-Conf. Work. on Agent-Oriented Inf. Sys. (II) (AOIS’04), Speringer, 2004, LNCS, vol. 3508, pp. 68-84. Framework for Analysis of Agent-Goal Models - Horkoff, Yu 17