1. Enterprise Modeling Saad El Beleidy Peyman Jamshidi Jared Kovacs Gabriel Lewis Improving the Analysis of Alternatives (AoA) Process for IT Solutions Government Contract Proposals Presented at:

2. Contents Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions 2

3. Contents 3 Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions

4. IT Solutions Government Contractors 4 Project Sponsor: Civilian and National Security (CNS) Division, Vangent, Inc. Proposal Development is critical to gaining revenue AoA: develop complex IT solution, crucial to proposal 15-25 Proposals per year, of varied sizes Federal Contract Spending decreased $40 billion from 2009 to 2011 Adapted from the Federal Times, 2011

5. 5 Acquisitions Committee Solicitation Proposals Selected Solution Government Contractor Bid Decision Bid Decision Government Entity Requirements Technical Solution Development Proposal Writing Budget & Management Technical Solution Development / AoA Process Level 1 Proposal Development Process Highly competitive: 5-10 competitors for each proposal

6. Process Level 2 Top Level AoA Process 6 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

7. Task Category Classification 1. Labor Intensive: Little expertise required: parsing, documenting. Time Variability = Low (±22%) 2. Decision Making: Expert required: difficult decisions, judgment calls. Time Variability = High (±54%) 3. Experience Recall: Expert required: subjectively making judgments based on previous experience. Time Variability = Medium (± 44%) 4. Networking: Personal dialogue and collaboration with co-workers and others. Time Variability = High (±100%) 7

8. Process Level 2 Top Level AoA Process 8 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

9. Process Level 3 Phase 1: Define Problem Domain 9 Data acquired via knowledge elicitation from stakeholder SMEs Task Category % of Phase Labor Intensive 25% Decision Making 23% Experience Recall 21% Networking31%

10. Process Level 2 Top Level AoA Process 10 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

11. Process Level 3 Phase 2: Define Evaluation Criteria 11 Data acquired via knowledge elicitation from stakeholder SMEs Task Category % of Phase Labor Intensive 42% Decision Making 20% Experience Recall 18% Networking20%

12. Process Level 2 Top Level AoA Process 12 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

13. Process Level 3 Phase 3: Explore Alternate Solutions 13 Data acquired via knowledge elicitation from stakeholder SMEs Task Category % of Phase Labor Intensive 36% Decision Making 15% Experience Recall 13% Networking36%

14. Process Level 2 Top Level AoA Process 14 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

15. Process Level 3 Phase 4: Evaluate Solutions 15 Task Category % of Phase Labor Intensive 38% Decision Making 21% Experience Recall 23% Networking18% Data acquired via knowledge elicitation from stakeholder SMEs

16. Current Issues in AoA 1.Limited and variable availability of past research due to proprietary restrictions 2.Limited and variable applicability of past research once gained 3.Variable difficulty of AoA—need for past research 16

17. Current Issues in AoA 4.Quality of AoA suffers from the information’s lack of availability and applicability 5.One employee handles entire AoA, so potentially parallel processes are conducted in series 6.Time spent for AoA results is an entirely overhead cost 17

18. Key Stakeholder Goals Solutions Architects Perform Analysis of Alternatives – Maximize productivity – Avoid overtime – Maximize labor rewards Capture Managers Manage the transition from opportunity discovery to contract award. Oversees bid strategies, pricing, and teaming – Maximize probability of winning contracts Increase AoA discriminability and quality Proposal Managers Develop and manage the proposal plan and schedule – Maximize proposal/AoA throughput 18

19. Key Stakeholder Interactions 19 Interactions Proposal Demand Solution Demand Tension: Managers and Solutions Architects Limited time and personnel resources to conduct AoAs.

20. Contents 20 Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions

21. Problem Statement During a time of national economic downturn, federal contract spending cuts have led to a decrease in available contract revenue and an increase in competition between government contractors. These factors have increased the time sensitivity of proposal development, specifically in the AoA process. 21

22. Need Statement There is a need for Analysis of Alternatives process improvements to reduce the mean time duration by at least 33%, and the variability by 25%, while maintaining or increasing AoA proposed solution quality and keeping maximum costs below $100,000 per AoA. 22 Need validated with key stakeholders

23. Contents 23 Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions

24. Design Alternative Approach Optimize AoA Staffing Levels – Target Parallel Tasks – Reduce mean time duration Information Management System – Target efficiency increases – Reduce duration and variability 24 AoA Phase Parallel Tasks Define Problem Domain 6 Define Evaluation Criteria 4 Explore Alternate Solutions 5 Evaluate Solutions 4 Task Category % AoAVariability Labor Intensive 35.2% Low ±22% Decision Making 19.2% High ± 54% Experience Recall 26.3% Med ± 44% Networking 19.2% High ±100%

25. Optimize AoA Staffing Levels 1 Additional Solutions Architect collaborates to conduct AoA Reduce mean time duration of AoA – Additional resource to conduct parallel tasks – Increases size of social network – Potential for conflict in making decisions Cost: Approx. $200,000 salary including benefits per year Five year cost: $1,000,000 25 Task Category Labor Intensive Decision Making Experience Recall Networking Efficiency Gain ----10%---+10%

26. Information Management Alternatives 26 Implementing a File Management System – Database searching capability improves availability of information in the AoA – Increases efficiencies of task categories, improves re-use of past research Implementing a Content Management System – Enhanced capabilities greatly improves availability of information in the AoA – Increases efficiencies of task categories, improves re-use of past research Maintaining a Sanitized Repository – Adds new content to the information pool – Changes high-variability task categories to low-variability categories

27. File Management System Benefits – Integrates with current system – Organized file structure – Promotes collaboration – Easily scalable Drawbacks – Requires permissions for file access – Limited search functionality Cost: Intravation Initial Cost: 100 User License GDIT(Parent Company) has 25 user license; $1000 one time fee per active user First Year Cost: $77,000 Annual Maintenance: $1600 Five Year Cost: $83,000 27 Task Category Labor Intensive Decision Making Experience Recall Networking Efficiency Gain +10% +15%+5%

28. Content Management System Benefits – Robust searching and indexing – Preconfigured user roles based upon content–access needs – Authentication, check in/out, tracking – Workflow management Drawbacks – High complexity High learning curve Expensive Technical Support Cost: Documentum Initial Cost: 100 User License Cost: $110,665 System Cost: First Year Cost: $129,000 Annual Maintenance: $19,000 Five Year Cost: $203,000 28 Task Category Labor Intensive Decision Making Experience Recall Networking Efficiency Gain +15% +20%+10%

29. Sanitized Document Repository Benefits – Quality of information improvement Promotes availability Increases applicable content – Virtually eliminates security risks – Provides quicker access to data – Minimal technical support Drawbacks – Low initial benefit Cost Estimate 12 labor hours per AoA Five Year Cost: $150,000 29 Task Category Labor Intensive Decision Making Experience Recall Networking Efficiency Gain +15%+10% +5% VariabilityLowHighMedHigh % of AoA Original  New 35%  40%20%  17%19%  17%26%

30. Contents 30 Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions

31. Simulation Design Monte Carlo Discrete Event Simulation of the AoA 1000 Replications Each Replication covers 25 Proposals (1 year) Key Model Assumptions: 1.Solutions Architects work on one task at a time 2.Only one proposal is being worked on at any given time 3.The four task categories adequately capture AoA time consumption 4.All tasks are of equal importance to the quality of AoA output 31

32. Simulation Design 32 Simulation Model AoA Process Definition Quality Metric AoA Duration Variability AoA Mean Duration Number of Technologies AoA Difficulty Information Availability Information Applicability Task Category Efficiencies

33. Simulation Video

34. :Task Categories (Labor Intensive, Decision Making, Experience Recall, Networking) :Inherent Task Delay :Task Category Efficiency Index (% Expected Time) :Task Category Weight (% of task with specific task category) :Task Category Variability Factor (RV) :Technology Variability (RV) :Number of Technologies/AoAs in the Proposal Task Time Delay Equation 34

35. Task Quality Metric Equation 35 :Availability Factor (RV) :Applicability Factor (RV) :Stakeholder weights

36. Design of Experiment Matrix 36 A1: Staffing Levels A2: Sanitized Repository A3: File Management A4: Content Management RunConfiguration Alternatives A1A2A3A4 1Baseline---- 2A1x--- 3A1, A2xx-- 4A1, A2, A3xxx- 5A1, A2, A4xx-x 6A1, A3x-x- 7A1, A4x--x 8A2-x-- 9A2, A3-xx- 10A2, A4-x-x 11A3--x- 12A4---x

37. Contents 37 Context and Stakeholder Analysis Problem and Need Design Alternatives Simulation Design Results and Conclusions

38. 38 33.00% A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management Simulation Results Percent Decrease in Mean Duration

39. Simulation Results Percent Decrease in Duration Variability 39 25.00% A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management

40. Simulation Results Quality Metric 40 A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management Using a Sanitized Repository is the only proposed alternative that directly affects quality. ~10% Increase in Quality 0.74 Baseline

41. Utility Mean Duration Duration Variability AoA Output Quality Solution Quality 41 Stakeholders’ Utility Function Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method.238.048.143.571

42. Alternative Utility Ranking 42 Rank Alternative Configuration Utility 1A1, A2, A44.25 2A1, A23.95 3A1, A2, A33.90 4A2, A43.90 5A23.54 6A1, A43.49 7A2, A33.28 8A43.18 9A1, A33.14 10A13.09 11Baseline2.50 12A32.37 A1: Optimized Staffing Levels A2: Sanitized Repository A3: File Management A4: Content Management Meet The Need Don’t Meet The Need

43. 43 Sensitivity Analysis Percent Change in Criteria Weight Necessary to Change Utility Rank Alternative Configurations Utility Function Criteria Mean TimeUsabilityIntegrability A1, A2+33%+41%+69% A1, A2, A3+34%+48%+119% A2, A4+60%+34%+60%

44. 44 Cost-Benefit Analysis A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management Meet the Duration and Variability Reduction Need Do Not Meet the Duration and Variability Reduction Need

45. 45 Cost-Benefit Analysis A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management Meet the Duration and Variability Reduction Need Do Not Meet the Duration and Variability Reduction Need More Desirable Closest to Desirable Region

46. 46 Recommendations Optimize Staffing Levels and Maintain a Sanitized Repository (A1, A2) – Percent Reduction in Time Duration: 44% – Percent Reduction in Duration Variability: 38% – Percent Increase in Quality: 10% – Total Utility: 3.95 – Max Expected Cost per AoA: $50,000 – Total Implementation Cost: $230,000/year

47. 47 Recommendations Optimize Staffing Levels, Maintain a Sanitized Repository, and Implement a Content Management System (A1, A2, A4) – Percent Reduction in Time Duration: 52% – Percent Reduction in Duration Variability: 50.00% – Percent Increase in Quality: 10% – Total Utility: 4.25 – Maximum Expected Cost per AoA: $78,000 – Total Implementation Cost: $111,000 plus $249,000/yr (Potential Value, if Cost Reduced)

48. Questions? 48

49. Backup Slides 49

50. Project Plan 50 WBS – Top Level Enterprise Modeling Project 1.0 Project Definition 2.0 Requirements Development 3.0 Solution Development 4.0 Modeling and Testing 5.0 Results Analysis 6.0 Communications and Management Syst 495-Spring 2012

51. Context Syst 490-Fall 2011 51 Output of AoA: Ranked Set of Alternatives AoA output for “Case Management Platforms” (Vangent, Inc) Vendors Considered: Custom Developed Siebel IBM Microsoft Total Scores

52. Syst 495-Spring 2012 52 Stakeholders’ Value Hierarchy Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method AoA Process Improvement Objective Reduce Mean Time Maximize Usability Reduce Time Variability Maximize Integrability Maximize Tailorability Maximize AoA Quality Maximize Tech Support Maximize Scalability.238.048.143.024.071.119.167.190

53. Information Management Alternatives Syst 495-Spring 2012 53 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Retrieve Past Proposal Requirements Maintain List of SME’s Retrieve Past Alternative Material Retrieve Past Criteria from similar proposals Provide reference of past analyses

54. Optimizing Staffing Levels Alternative Syst 495-Spring 2012 54 Time Reduction Parallel Processes Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions

55. Percent Decrease 55 Syst 495-Spring 2012 A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management 33% 25%

56. Alternatives Efficiency Summary Syst 495-Spring 2012 56 Alternative Labor Intensive Decision Making Experience Recall Networking Baseline --- Optimizing Staffing Levels ----10%---+10% Sanitized Repository +15%+10% +5% File Management +10% +15%+5% Content Management +15% +20%+10%

57. Alternatives’ Efficiency Indexes Syst 495-Spring 2012 57 A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management

58. Syst 495-Spring 2012 58 Acquisitions Committee Solicitation Proposal Selected Solution Government Contractor Bid Decision Bid Decision Government Entity Requirements Technical Solution Development Proposal Writing Budget & Management Technical Solution Development Process Level 1 Proposal Development Process Proposal Development is critical to gaining revenue Highly competitive: 5-10 competitors for each proposal

59. Syst 495-Spring 2012 59 Process Level 2 Technical Solution Development 59 Alternatives Analysis Integration of Assets Solicitation Ranked List of Alternatives Assets for Proposal Solution to be Proposed Analysis of Alternatives (AoA) Model developed based on team research and knowledge elicitation from Vangent sponsors

60. Process Level 3 Top Level AoA Process Syst 495-Spring 2012 60 Define the Problem Domain Define Evaluation Criteria Explore Alternate Solutions Evaluate Solutions Solicitation Ranked List of Alternatives High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors