Download presentation
Presentation is loading. Please wait.
Published byTerence Rich Modified over 9 years ago
1
1 Convener: Houman Younessi Convener: Houman Younessi Software Engineering Management Software Engineering Management Course # CISH-6050 Lecture 7: Software Process Assessment 07/09/2012
2
2 CISH-6050 - Software Engineering Management AGENDA … Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents
3
3 CISH-6050 - Software Engineering Management SW-CMM Process Assessment SEI SW-CMM Improvement Approach: IDEAL -Initiating -Diagnosing -Establishing -Acting -Leveraging Diagnose process, establish priorities, & act on them SEI SW-CMM Improvement Approach: IDEAL -Initiating -Diagnosing -Establishing -Acting -Leveraging Diagnose process, establish priorities, & act on them
4
4 CISH-6050 - Software Engineering Management SW-CMM: IDEAL Initiating: -Stimulus for improvement -Set context & sponsorship -Establish improvement infrastructure Diagnosing: -Appraise & characterize current practice -Develop recommendations -Document phase results Initiating: -Stimulus for improvement -Set context & sponsorship -Establish improvement infrastructure Diagnosing: -Appraise & characterize current practice -Develop recommendations -Document phase results
5
5 CISH-6050 - Software Engineering Management SW-CMM: IDEAL … Establishing: -Set strategy & priorities -Establish process action items -Plan actions Acting: -Define processes and measures -Plan & execute pilots -Plan, execute, & track installation Establishing: -Set strategy & priorities -Establish process action items -Plan actions Acting: -Define processes and measures -Plan & execute pilots -Plan, execute, & track installation
6
6 CISH-6050 - Software Engineering Management SW-CMM: IDEAL … Leveraging: -Document & analyze lessons -Revise organizational approach Leveraging: -Document & analyze lessons -Revise organizational approach
7
7 CISH-6050 - Software Engineering Management SW-CMM Process Assessment General Classes of SW-CMM Appraisal 1.Software Process Assessment Determine state of organization’s software process 2.Software Capability Evaluations Identify contractors qualified to perform software work General Classes of SW-CMM Appraisal 1.Software Process Assessment Determine state of organization’s software process 2.Software Capability Evaluations Identify contractors qualified to perform software work
8
8 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Software Process Assessment: -Identify improvement priorities within organization -Assessment team uses CMM to guide identifying & prioritizing findings -Findings & KPA guidance used to plan improvement strategy for organization Software Process Assessment: -Identify improvement priorities within organization -Assessment team uses CMM to guide identifying & prioritizing findings -Findings & KPA guidance used to plan improvement strategy for organization
9
9 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Software Capability Evaluations: -Identify risks associated with a project or contract to build high quality on schedule & budget -During acquisition process, capability evaluation may be performed on bidders -Findings of an evaluation may be used to identify risk with using a contractor -Performed on existing contracts to monitor process performance Software Capability Evaluations: -Identify risks associated with a project or contract to build high quality on schedule & budget -During acquisition process, capability evaluation may be performed on bidders -Findings of an evaluation may be used to identify risk with using a contractor -Performed on existing contracts to monitor process performance
10
10 CISH-6050 - Software Engineering Management Software Process Assessment & Capability Evaluation Steps:
11
11 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Common Steps: -Team Selection Select team trained in CMM Knowledgeable in SE & mgmt -Maturity Questionnaire Site reps complete questionnaire -Response Analysis Analyze results of questionnaire Investigation areas = KPAs Common Steps: -Team Selection Select team trained in CMM Knowledgeable in SE & mgmt -Maturity Questionnaire Site reps complete questionnaire -Response Analysis Analyze results of questionnaire Investigation areas = KPAs
12
12 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Common Steps … -On-site Visit Using results analysis, conduct on- site visit to view process areas Using KPAs as guide, question, listen, review & synthesize info Apply professional judgment Document rationale for situations where KPAs not met Common Steps … -On-site Visit Using results analysis, conduct on- site visit to view process areas Using KPAs as guide, question, listen, review & synthesize info Apply professional judgment Document rationale for situations where KPAs not met
13
13 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Common Steps … -Findings At end of on-site period, team produces list of findings Identifies strengths & weaknesses of org’s software processes Software Process Assessment -> Basis for PI recommendations Software Capability Evaluation -> Findings part of risk analysis Common Steps … -Findings At end of on-site period, team produces list of findings Identifies strengths & weaknesses of org’s software processes Software Process Assessment -> Basis for PI recommendations Software Capability Evaluation -> Findings part of risk analysis
14
14 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Common Steps … -KPA Profile Team prepares KPA profile, showing where KPAs satisfied / not satisfied by organization KPA can be satisfied and still have associated findings, as long as findings don’t identify major problems achieving goals of KPA Common Steps … -KPA Profile Team prepares KPA profile, showing where KPAs satisfied / not satisfied by organization KPA can be satisfied and still have associated findings, as long as findings don’t identify major problems achieving goals of KPA
15
15 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Differences: Process Assessments & Capability Evaluation -Results of process assessment or capability evaluation may differ -Assessment / evaluation scope may vary: Different definitions of ‘Organization’ Org may be based on senior management, geo location, common app, profit/loss center, etc. Sample of selected projects Differences: Process Assessments & Capability Evaluation -Results of process assessment or capability evaluation may differ -Assessment / evaluation scope may vary: Different definitions of ‘Organization’ Org may be based on senior management, geo location, common app, profit/loss center, etc. Sample of selected projects
16
16 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Differences … -Motivation, objective, outcome & results ownership differ These factors lead to differences in dynamics of interviews, scope of inquiry, info collected, & results Assessment & evaluation methods are different Assessment training doesn’t prepare team to do evaluation, vice versa Differences … -Motivation, objective, outcome & results ownership differ These factors lead to differences in dynamics of interviews, scope of inquiry, info collected, & results Assessment & evaluation methods are different Assessment training doesn’t prepare team to do evaluation, vice versa
17
17 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Differences … -Process Assessment – performed in open, collaborative environment Commitment from mgmt & staff to do process improvement Objective: surface problems & help improve organization Emphasis on interviews as tool for understanding organization’s software process Differences … -Process Assessment – performed in open, collaborative environment Commitment from mgmt & staff to do process improvement Objective: surface problems & help improve organization Emphasis on interviews as tool for understanding organization’s software process
18
18 CISH-6050 - Software Engineering Management SW-CMM Process Assessment … Differences … -Capability Evaluation – performed in audit-oriented environment Objective tied to monetary considerations Emphasis on documented audit trail that reveals software process actually implemented by organization Differences … -Capability Evaluation – performed in audit-oriented environment Objective tied to monetary considerations Emphasis on documented audit trail that reveals software process actually implemented by organization
19
19 CISH-6050 - Software Engineering Management AGENDA … Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents
20
20 CISH-6050 - Software Engineering Management Software Process Assessments Why do process assessment? -“Why Do Organizations Have Assessments? Do They Pay Off?”, CMU SEI Technical Report, July, 1999 -Panel discussion from SEPG ’99 in Atlanta, GA on March 10, 1999 -Each panelist is one of most active lead assessor for SEI for CMM-Based Appraisals for Internal Process Improvement (CBA IPI) Why do process assessment? -“Why Do Organizations Have Assessments? Do They Pay Off?”, CMU SEI Technical Report, July, 1999 -Panel discussion from SEPG ’99 in Atlanta, GA on March 10, 1999 -Each panelist is one of most active lead assessor for SEI for CMM-Based Appraisals for Internal Process Improvement (CBA IPI)
21
21 CISH-6050 - Software Engineering Management Software Process Assessments … Electronic Data Systems -Why do assessments? Driven by customer: higher maturity to keep customers; extra bidding points for higher maturity; DoD requirements -Do assessments pay off? Yes! Need exec sponsorship, funding Everyone learns, helps org culture Electronic Data Systems -Why do assessments? Driven by customer: higher maturity to keep customers; extra bidding points for higher maturity; DoD requirements -Do assessments pay off? Yes! Need exec sponsorship, funding Everyone learns, helps org culture
22
22 CISH-6050 - Software Engineering Management Software Process Assessments … Reuters -Why do assessments? Framework for continuous process improvement; maturity levels build on previous level Raises awareness in group -Do assessments pay off? Clear view of cost, effort, resource Measure key performance items Method for introducing global change Reuters -Why do assessments? Framework for continuous process improvement; maturity levels build on previous level Raises awareness in group -Do assessments pay off? Clear view of cost, effort, resource Measure key performance items Method for introducing global change
23
23 CISH-6050 - Software Engineering Management Software Process Assessments … U. S. Navy -Why do assessments? Early SEI affiliate Beta test group for Maturity Questionnaire Continued with SEI assessments -Do assessments pay off? Invaluable to organization In 1997 determined 4.1 to 1 ROI Education for new team members U. S. Navy -Why do assessments? Early SEI affiliate Beta test group for Maturity Questionnaire Continued with SEI assessments -Do assessments pay off? Invaluable to organization In 1997 determined 4.1 to 1 ROI Education for new team members
24
24 CISH-6050 - Software Engineering Management Software Process Assessments … Panel Q&A -Why pay cost of doing CBA IPI? -How often do you do full CBA IPI? -How small of organization to do CBA IPI? -How to reduce impact/stress on org when doing CBA IPI? Panel Q&A -Why pay cost of doing CBA IPI? -How often do you do full CBA IPI? -How small of organization to do CBA IPI? -How to reduce impact/stress on org when doing CBA IPI?
25
25 CISH-6050 - Software Engineering Management AGENDA … Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents
26
26 CISH-6050 - Software Engineering Management Software Process Assessments … Blowing the Whistle on Troubled Software Projects -Article by Mark Keil & Daniel Robey -Controversy behind going forward to management when auditors find problems with projects -Case study of IS Auditors 1999 Study 75 Auditors Blowing the Whistle on Troubled Software Projects -Article by Mark Keil & Daniel Robey -Controversy behind going forward to management when auditors find problems with projects -Case study of IS Auditors 1999 Study 75 Auditors
27
27 CISH-6050 - Software Engineering Management Software Process Assessments … Key Points -Auditor obligation vs. risk -Reluctance to blow the whistle -Reluctance to hear the whistle -Conditions that encourage or influence whistle blowing Key Points -Auditor obligation vs. risk -Reluctance to blow the whistle -Reluctance to hear the whistle -Conditions that encourage or influence whistle blowing
28
28 CISH-6050 - Software Engineering Management AGENDA … Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents Process Assessment -SEI SW-CMM Improvement Approach -Do organizational assessments pay off? -Whistleblowers -Reference: SPICE Assessment Documents
29
29 CISH-6050 - Software Engineering Management SPICE Assessment: Reference SPICE Software Process Assessment Information -SPICE documentation available free of charge from Software Quality Institute web site: http://www.sqi.gu.edu.au/spice/suite/do wnload.html SPICE Software Process Assessment Information -SPICE documentation available free of charge from Software Quality Institute web site: http://www.sqi.gu.edu.au/spice/suite/do wnload.html
30
30 CISH-6050 - Software Engineering Management SPICE Assessment: Reference … SPICE Part 3 – Rating Process -Section 4.2: Defining Assessment Input -Section 4.4: Assessing and Rating Process -Section 4.5: Recording the Assessment Output SPICE Part 3 – Rating Process -Section 4.2: Defining Assessment Input -Section 4.4: Assessing and Rating Process -Section 4.5: Recording the Assessment Output
31
31 CISH-6050 - Software Engineering Management SPICE Assessment: Reference … SPICE Part 4 – Guide to Conducting Assessment -Section 4.2: Process Rating Scheme -Section 4.3.1.1: Team-based Assessment -Section 4.4: Assessment Stages (8) -Section 4.5: Success Factors for Process Assessment -Section 5.1: Reviewing Assessment Inputs SPICE Part 4 – Guide to Conducting Assessment -Section 4.2: Process Rating Scheme -Section 4.3.1.1: Team-based Assessment -Section 4.4: Assessment Stages (8) -Section 4.5: Success Factors for Process Assessment -Section 5.1: Reviewing Assessment Inputs
32
32 CISH-6050 - Software Engineering Management SPICE Assessment: Reference … SPICE Part 4 … -Section 5.2: Selecting Process Instances -Section 5.3: Preparing for Team-based Assessment -Section 5.4: Collecting and Verifying Information -Section 5.5: Determine Actual Ratings for Process Instances -Section 5.6: Determining Derived Ratings -Section 5.7: Validating the Ratings -Section 5.8: Presenting Assessment Output SPICE Part 4 … -Section 5.2: Selecting Process Instances -Section 5.3: Preparing for Team-based Assessment -Section 5.4: Collecting and Verifying Information -Section 5.5: Determine Actual Ratings for Process Instances -Section 5.6: Determining Derived Ratings -Section 5.7: Validating the Ratings -Section 5.8: Presenting Assessment Output
33
33 CISH-6050 - Software Engineering Management SPICE Assessment: Reference … SPICE Part 5 – Construction, Selection, & Use of Assessment Tools -Section 4.1: Form and Purpose of an Assessment Instrument -Section 4.2 (and Annexes A, B, C, D): Implementation of Standard Indicators -Section 4.3: Tailoring of Indicators Contained in Assessment Instrument -Section 4.4: Modular Assessment Instruments SPICE Part 5 – Construction, Selection, & Use of Assessment Tools -Section 4.1: Form and Purpose of an Assessment Instrument -Section 4.2 (and Annexes A, B, C, D): Implementation of Standard Indicators -Section 4.3: Tailoring of Indicators Contained in Assessment Instrument -Section 4.4: Modular Assessment Instruments
34
34 CISH-6050 - Software Engineering Management SPICE Assessment: Reference … SPICE Part 5 … -Section 4.5: Capturing and Processing Assessment Data -Section 4.6: Using an Assessment Instrument SPICE Part 6 – Qualifications & Training of Assessors -Section 4.1: Role of the Assessor -Section 5: Assessor Competence SPICE Part 5 … -Section 4.5: Capturing and Processing Assessment Data -Section 4.6: Using an Assessment Instrument SPICE Part 6 – Qualifications & Training of Assessors -Section 4.1: Role of the Assessor -Section 5: Assessor Competence
35
35 CISH-6050 - Software Engineering Management References M. Paulk, B. Curtis, M. B. Chrissis, C. V. Weber, "Capability Maturity Model for Software", Version 1.1, Software Engineering Institute, Carnegie Mellon University, February, 1993. Available at http://www.sei.cmu.edu/publications/documents/93.reports/93.tr.024.html W. S. Humphrey, Managing the Software Process, Addison-Wesley, Reading, MA, 1989 M. Paulk, C. V. Weber, B. Curtis, M. B. Chrissis, The Capability Maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, Boston, MA, 1995 “Process Maturity Profile of the Software Community 2002 Mid-Year Update”, Software Engineering Institute, Carnegie Mellon University, August, 2002. Available at http://www.sei.cmu.edu/sema/pdf/SW- CMM/2002aug.pdf M. Paulk, B. Curtis, M. B. Chrissis, C. V. Weber, "Capability Maturity Model for Software", Version 1.1, Software Engineering Institute, Carnegie Mellon University, February, 1993. Available at http://www.sei.cmu.edu/publications/documents/93.reports/93.tr.024.html W. S. Humphrey, Managing the Software Process, Addison-Wesley, Reading, MA, 1989 M. Paulk, C. V. Weber, B. Curtis, M. B. Chrissis, The Capability Maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, Boston, MA, 1995 “Process Maturity Profile of the Software Community 2002 Mid-Year Update”, Software Engineering Institute, Carnegie Mellon University, August, 2002. Available at http://www.sei.cmu.edu/sema/pdf/SW- CMM/2002aug.pdf
36
36 CISH-6050 - Software Engineering Management References … D. Dunaway, R. Berggren, G. des Rochettes, P. Iredale, I. Lavi, G. Taylor, “Why Do Organizations Have Assessments? Do They Pay Off?”, Software Engineering Institute, Carnegie Mellon University, July, 1999. Available at http://www.sei.cmu.edu/publications/documents/99.reports/99tr012/99tr0 12abstract.html M. Keil, D. Robey, “Blowing the Whistle on Troubled Software Projects”, Communications of the ACM, Vol. 44, No. 4, April 2001, pp. 87-93. PDF file (ACMp87-Keil.pdf) available online via ACM Digital Library when accessed from the RPI at Hartford Library ISO/IEC TR 15504:1998 - Software Process Assessment (SPICE) documentation suite. Available at http://www.sqi.gu.edu.au/spice/suite/download.html D. Dunaway, R. Berggren, G. des Rochettes, P. Iredale, I. Lavi, G. Taylor, “Why Do Organizations Have Assessments? Do They Pay Off?”, Software Engineering Institute, Carnegie Mellon University, July, 1999. Available at http://www.sei.cmu.edu/publications/documents/99.reports/99tr012/99tr0 12abstract.html M. Keil, D. Robey, “Blowing the Whistle on Troubled Software Projects”, Communications of the ACM, Vol. 44, No. 4, April 2001, pp. 87-93. PDF file (ACMp87-Keil.pdf) available online via ACM Digital Library when accessed from the RPI at Hartford Library ISO/IEC TR 15504:1998 - Software Process Assessment (SPICE) documentation suite. Available at http://www.sqi.gu.edu.au/spice/suite/download.html
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.