1. Agile Methods and the CeBASE Method Dan Port, USC Vic Basili, UMD

2. Outline Overview of the CeBASE Method The Challenge of Application to CS577 Our track record Critical Agility Strategies –Risk driven specifications and modeling –Rapidly achieving shared vision, tacit knowledge –Experience Factory and knowledge reuse

3. CeBASE Method Overview We needed to create a unified framework for empirical SE –Reconciled our processes as well as our data definitions We found that EF/GQM and MBASE/Spiral were compatible and complementary –GQM Goals map to Spiral Objectives –GQM Questions & Metrics map to Spiral evaluation of Alternatives –MBASE focuses on projects; EF covers organizations as well Integrated CeBASE Method presented at STC 2000 by Boehm and Vaughn –Led to adoption by several DoD organizations, including Army/DARPA’s biggest software project

4. Org. Improvement Goals –Goal-related questions, metrics Org. Improvement Strategies –Goal achievement models Org. Shared Vision & Improvement Strategy Experience Factory Framework - I

5. Initiatives Planning context Progress/Plan/ Goal Mismatches Experience Base Analyzed experience, Updated models Achievables, Opportunities Org. Improvement Goals –Goal-related questions, metrics Org. Improvement Strategies –Goal achievement models Org. Improvement Initiative Planning & Control Initiative Plans –Initiative-related questions, metrics Initiative Monitoring and Control –Experience-Base Analysis Org. Shared Vision & Improvement Strategy Experience Factory Framework - II

6. Initiatives Planning context Progress/Plan/ Goal Mismatches Experience Base Analyzed experience, Updated models Achievables, Opportunities Org. Improvement Goals –Goal-related questions, metrics Org. Improvement Strategies –Goal achievement models Org. Improvement Initiative Planning & Control Initiative Plans –Initiative-related questions, metrics Initiative Monitoring and Control –Experience-Base Analysis Org. Shared Vision & Improvement Strategy Experience Factory Framework - III Project Shared Vision and Strategy Planning Context Models and data Project experience Org. Goals Project Planning and Control Models and data Progress/Plan/ Goal Mismatches

7. The CeBASE Method 1. Org. Value Propositions (VP’s) a-Stakeholder values 2. Current situation w.r.t. VP’s 3. Improvement Goals, Priorities 4. Global Scope, Results Chain 5. Value/business case models Org-Portfolio Shared Vision 1. Strategy elements 2. Evaluation criteria/questions 3. Improvement plans a. Progress metrics b. Experience base Org. Strategic Plans Organization/ Portfolio: Experience Factory, GMQM 1. Monitor environment -Update models 2. Implement plans 3. Evaluate progress -w.r.t. goals, models 4. Determine, apply corrective actions 5. Update experience base Org. Monitoring & Control Monitoring & Control Context 1. Project Value Propositions a-Stakeholder values 2. Current situation w.r.t. VP’s 3. Improvement Goals, Priorities 4. Project Scope, Results Chain 5. Value/business case models Project Shared Vision Project: MBASE Planning context Plan/Goal mismatches Project Plans Planning Context Initiatives OFB: Progress/Plan/Goal mismatches -shortfalls, opportunities, risks Project vision, goals O-PSV Shortfalls, opportunities, risks; P-OSV Scoping context Shortfalls, opportunities, risks: P-OP Planning Context (O-PP) 1. Monitor environment a-Update models 2. Implement plans 3. Evaluate progress -w.r.t. goals, models, plans 4. Determine, apply corrective actions 5. Update experience base Proj. Monitoring & Control Monitoring & Control context PFB: Progress/Plan/goal mismatches -Shortfalls, opportunities, risks Plan/goal mismatches Monitoring & Control context Project experience, progress w.r.t. plans, goals LCO: Life Cycle Objectives LCA: Life Cycle Architecture IOC: Initial Operational Capability GMQM: Goal-Model-Question-Metric Paradigm MBASE: Model-Based (System) Architecting and Software Engineering -Applies to organization’s and projects’ people, processes, and products Project Plans (PP) 1. LCO/LCA Package -Ops concept, prototypes, rqts, architecture, LCplan, rationale 2. IOC/Transition/Support Package -Design, increment plans, quality plans, T/S plans 3. Evaluation criteria/questions 4. Progress metrics

8. Outline Overview of the CeBASE Method The Challenge of Application to CS577 Our track record Critical Agility Strategies –Risk driven specifications and modeling –Rapidly achieving shared vision, tacit knowledge –Experience Factory and knowledge reuse

9. The CS577 Challenge 20 USC eServices Applications –2 sentence problem statements –USC Information Services clients 100 Graduate Students –30% with industry experience –Largely unfamiliar with each other, Library ops. *Develop LCA packages in 12 weeks Re-form teams from 30-40 continuing students *Develop IOC packages in 12 more weeks –Including 2-week beta test and transition

10. Application of CeBASE Within the constraints of the previous slide, student teams must: –Rapidly assimilate the key stakeholders organizational shared vision –Rapidly converge on an organization shared vision with key stakeholders –Understand and align project with the organizations strategic plans –Formulate and execute feasible project plans –Adapt to frequent client changes –Transition a system in alignment with organization monitoring and control process Impossible?

11. Critical Success Factors for Adoption - I

12. Critical Success Factors for Adoption - II

13. Summary of Results 1996-2000

14. Outline Overview of the CeBASE Method The Challenge of Application to CS577 Our track record Critical Agility Strategies –Risk driven specifications and modeling –Rapidly achieving shared vision, tacit knowledge –Experience Factory and knowledge reuse

15. Risk-Driven Specifications and Activities Basic driving principle for CeBASE activities and specifications (i.e. modeling, model content, degree of detail, etc.) If it’s risky to do something, Don’t e.g. specify firm GUI requirements early If it’s risky not to do something, Do e.g. document shared protocols Seems obvious, but often not explicitly done or managed! –People must be educated to perform effective risk identification, assessment, mitigation, prioritization, tolerance –Varies considerably over the project and people

16. Tactical and Strategic Risk Management CeBASE makes use of both tactical and strategic risk management Tactical Risk Management –Rmall-scale actions serving to contain or respond to risks made or carried out with only a limited or immediate end in view Risk identification and assessment Top-10 risk monitoring (EF) Risk contingency plans Strategic –Risk management as an integrated whole or to a planned effect (e.g. Expected Return on Investment) Risk driven “how much is enough?” approach Risk/value based feasibility assessments (GQM) Risk based development processes (e.g. SAIV)

17. Rapidly achieving shared vision and tacit knowledge Critical challenge is to converge on a shared vision for the project within 12 weeks –Involves many factors such as teambuilding, stakeholder identification, requirements solicitation and negotiation, domain modeling, etc. CeBASE uses many agile approaches to building shared vision and experience factory techniques to utilize tacit knowledge –Win-win requirements negotiation, early prototyping, group planning exercises, stakeholder lists, results chains

18. Results Chain: Hispanic Digital Archive (HDA) Major donor funding Viable HDA Archive Digitize HDA Archive Viable IBM DL package HDA PR, training for USC, community Develop HDA Software Assumption Outcome Initiative Contribution Staff, train HDA Ops/Maint personnel Viable HDA System Digital HDA assets Sustainable HDA Archive World- class Hispanic research, education, outreach support

19. CS577 Experience Factory Teams are supplied with and educated in the use of an experience base with: –Domain model descriptions –Pre-architectures –Specialized WinWin taxonomies –Specialized top-n risks –Previous project examples Each year the experience base is updated –E.g. in 1998 we added “simplifier and complecators” to reduce LCO failure rate

20. Example S&C’s 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 31, 32, 35, 36, 37, 39 Type of Application Simple Block DiagramExamplesSimplifiersComplicators Multimedia Archive  Use standard query languages  Use standard or COTS search engine  Uniform media formats  Natural language processing  Automated cataloging or indexing  Digitizing large archives  Digitizing complex or fragile artifacts  Automated annotation/descrip tion/ or meanings to digital assets  Integration of legacy systems MM asset info Catalog MM Archive query MM asset update query update notification  Rapid access to large Archives  Access to heterogeneous media collections

21. The Results Projects That Failed LCO Criteria -1996: 4 out of 16 (25%) -1997: 4 out of 15 (27%) -1998: 1 out of 20 (5%) -1999: 1 out of 22 (4%) 40% of Student critiques cited S&C’s as helpful (and more since) -In focusing on achievable requirements set within tight schedule -In understanding project risks and tradeoffs

22. Summary Overall success rate 92% –compared with 26% Standish Primarily agile approaches used –Scenarios, prototypes, patterns, metaphors Primarily plan driven approaches used –Risk driven requirements, life cycle architecture, stakeholder roles and responsibilities, feasibility rationale, risk management plan CeBASE not one size fit all –Risk tailoring produces appropriate balance of discipline and flexibility

23. Backup Slides Further CS577 CeBASE Experience Factory examples

24. Example win-win taxonomy

25. Example top-n Risks Multimedia Archive Risks

26. Example previous project data

27. Domain Models

28. Example Domain Model

29. Example Pre- architectures

30. Specialized S&C’s

31. SimplifiersRisks and Trade-offs Generic Uniform Media Formats Specific All video clips are stored using an open file format for video/audio (e.g., MPEG). All film stills are stored using an open image file format (e.g., JPEG). The inverse complicator is to store film clips using streaming video technologies This means that we may have to convert existing digital assets or digitize the original media, which may be costly. A unique file format limits the user base to those who have viewers for that particular file format The chosen file format may not be the most efficient for the various types of media (in terms of compression rates, quality, etc...) Generic Use Standard Query Languages Specific Organize catalog and archive relationally so that queries will be limited to standard search formats,: match exactly by value on any of the fields with or without using boolean combinations (AND, OR, NOT, etc...), or using pattern matching (SQL LIKE keyword) May not be as effective for "discovering" assets in the archive: users must know what they're looking for, in order to search for it Generic Use Standard COTS Specific Use a standard Relational Database Management System (RDBMS) that supports storing multi-media assets A Relational Database Management System may not be most suited for archival of multi-media assets. A Relational Database Management System may have a high initial cost, high implementation, and high administration cost (requires specialized knowledge skills)

32. ComplicatorsRisks and Trade-offs Generic Natural Language Processing Specific Store the information only in one language (e.g., English) and provide dynamic translation into Chinese, Japanese and Korean The inverse simplifier is to store the same information in 4 different languages (English, Chinese, Japanese and Korean). The first approach is a complex, error-prone, expensive natural language processing issue The second approach will require more storage space, in addition to acquiring the translations Generic Digitizing Large Archives Specific Digitizing film clips from the entire collection of films (which grows at a very fast rate of 800 films per year for Indian films alone) If each film clip requires around 10 MB, then the rate of growth of the database will be of 8GB a year (exclusive of catalog information) Generic Integration of "Legacy" Systems Specific Do not require Real-Video plug-in for Web browsers to allow users to view streamed film clips We cannot use more effective multi-media formats, which are becoming standard technologies