Evaluate SE Methods, Processes and Tools Technical Task Plan USC Workshop Los Angeles, CA 16 March 2009

Agenda Introduction to the MPT task and adjustments Status of task Planned activities for the workshop 2

SOW Language Look at current SE methods, processes, and tools (MPTs) as they are applied across the DoD acquisition life cycle focusing on three different development environments: individual weapons systems, SoS, and network-centric services. Research will be targeted at improving current/identifying new SE MPTs that will better support the practice of SE in these three environments. Specifically, this task will: 1. Define critical attributes of current SE MPTs across the weapons system, SoS, and network-centric services environments; 2. Identify strengths and weaknesses for these current MPTs and any shortcomings in their application across DoD; 3. Recommend, in priority order, MPTs for further study to innovate or create improved or new MPTs to eliminate identified shortcomings; 4. Upon selection by the government of MPTs recommended in sub-task 3 for further study, perform research to innovate or create improved or new MPTs to eliminate identified shortcomings, thereby advancing the state of practice of SE within the community; and 5. For the improvements delivered in sub-task 4 above, propose a methodology for validating the programs. 3

eWorkshop MPT Sources Select MPTs for Evaluation Describe MPTs Evaluate MPTs MPT Analysis Apply Selection Criteria Complete Detailed Attributes Apply Evaluation Criteria Reports and Recommendations Cumulative MPT Coverage Identified Raw MPTs H H H H M M M M Queue of selected and prioritized MPTs Fully described MPTs to evaluate Evaluated MPTs BPCH Establish Criteria and Validate with Sponsor Repeat H H L L H H H H M M M M H H L L DoD Guideboooks DoD Programs/Reviews Service Repositories Defense Industry Commercial industry To Users Recommended MPTs Improvements Needed Overall Gap Analysis Research Areas MPT Task Overview (As proposed) 4

Initial MPT Attributes 5

Adjustments Scope very broad for task resources Task sponsor initially interested in their particular environmental challenges – Less interested in the ongoing, repetitive evaluation process proposed – Looking for ideas outside traditional MPTs – Broader survey and gap analysis 6

eWorkshop Describe MPTs Identify and Mine Comparative Environments for MPTs MPT Task Changes Select MPTs for Evaluation Initial MPT Candidate Templates Define and Validate Sponsor Environment Challenges and Needs Review sources provided in SoW Review literature and web Broad survey of industry Conduct interviews Revise MPT Attribute Set Identify Promising MPTs and Gaps Conduct open-ended expert workshops Evaluate MPTs MPT Analysis Reports and Recommendations BPCH Repeat To Users 7

Task Status Define Sponsor Environmental Challenges and Needs – Developed and refined initial challenge list Validating Environmental Challenges and Needs – Currently interviewing sponsor personnel 5 people to date Indications are that environment challenge description valid – Revise/extend template as needed – Revise evaluation criteria based on needs assessment Revised attribute set for one-page candidate identification template 8 Italics indicate tasks of the workshop

Environmental Challenges 9

Researcher Template Support data collection on existing MPTs Includes challenge description Based on subset of MPT attributes Intended to winnow down to reasonable number of MPTs to evaluate Primarily used with the open sources review (literature, web) Initial feedback good; waiting to see results 10

Broad Industry Survey To identify potential gaps and gather MPT candidates from broader community based on environmental challenges – Commercial industry – Defense community – Research community Draft survey developed Help needed to get survey to right people – SERC members – CSSE Members 11

MPT Activities during workshop Pilot broad industry survey with ARR members – Collect and review data Discuss environmental challenges – MPT types (Process/governance vs. technical/tools) Discuss gap/MPT identification workshop strategy 12