1. 1 Summary and Analysis of Position Statements Assessment Track Paul E. McMahon (sitting in for Watts Humphrey)

2. 2 Topics Scope –Initial Thoughts on Assessment –Broader View of Assessment (One Chosen) –Another thought to consider on “assessment” Summary of Position Statements –What Should we measure? Tangible Less tangible… Analysis & Recommendations For Track –Brainstorm previous/refine/drive from starting point consensus Discussion

3. 3 Initial Thoughts on Assessment Thinking from CMMI “Assessment” Perspective –“Assessment” dependent on product pieces produced by doing “software engineering” E.g. pieces exist, or don’t –Not on “quality” of those pieces

4. 4 Broader View of “Assessment” (One Chosen) Assessing the quality of whatever the SEMAT product becomes –May be a combination of product and process E.g. could end up including “what engineers do” Note: This is “broader” in that it goes beyond simply “assessing” if you comply or not (existence of some artifact) –This gets at “real quality” side

5. 5 Another Thought to Consider on “Assessment” How does SEMAT “assess” itself? Suggestion--Run SEMAT like a real project Schedules, deliverables from tracks & reviews We need to measure our own progress to meet milestones Can’t wait five years for value to customer

6. 6 What Should We Measure? Tangible Standards, rules-- compliance –Exit/entry criteria –Templates –Specification rules –Content rules –Product measures –Process measures Reuse of existing material Productivity Improvement Intent to stimulate discussion now, and for later track discussion

7. 7 What Should We Measure (continued)? –Other Compliance areas Practices Process models Process Enactment Universals/Kernel –Non-functional Requirements areas Performance Reliability Complexity Reusability Extensibility Scalability Maintainability Reliability Safety –Risks (technical, other)

8. 8 What Should We Measure? Less Tangible Ease of use (usability) Evolve-ability Compliance with Patterns Success of project Process Improvement Skill levels What engineers do People risks (teamwork)

9. 9 What Should We Measure? Less tangible (continued) Improving software development & management Improving education on software development and management Improving consciousness in community To consider for each of the above: –Long term measures –Short term (12 months) measures

10. 10 Analysis & Recommendations For Track Need to work closely/stay aligned with all other tracks, particularly Universals track –As we see where “universal” and other tracks are going we can delve deeper into related “specific’ “assessment/measures” 1 st step for group –Discuss/brainstorm a set of “core” measures 2 nd step for group –Discuss/brainstorm if it makes sense for “domain-specific” core measures –E.g. real-time embedded life-threatening situations

11. 11 Analysis and Recommendations for Track (continued) Suggest this track not discuss –Measures for “kernel/universal” elements until get input from this track Otherwise duplicating their work –Science versus Engineering Leave to Theory Track (although I find very interesting) Consider using “open” community –Helps buy-in –Helps when resources become constrained –Keeps momentum moving, and reduces risk of single point failure on team

12. 12 Discussion… I stop talking and start listening and taking notes…