1. University of Southern California Center for Systems and Software Engineering Rapid-Fielding Software-System Development Supannika Koolmanojwong koolmano@usc.edu USC CSSE Annual Research Review March 9, 2010

2. University of Southern California Center for Systems and Software Engineering Outline The Incremental Commitment Model & Rapid- fielding projects Process Pattern Decision Driver Preliminary Results 03/09/2010USC CSSE ARR 20102

3. University of Southern California Center for Systems and Software Engineering Motivation Growing diversity of software systems –No one-size-fits-all process for the full range of software systems Rapid Fielding is a very important Software System Objective –To fit into market windows, respond to competition –Several possible process patterns can be used Some process models provide specific evidence-based and risk based decision points –ICM is the most thoroughly elaborated 03/09/2010USC CSSE ARR 20103

4. University of Southern California Center for Systems and Software Engineering NDI/NCS Growth Trends 03/09/2010USC CSSE ARR 20104 USC e-services project data shows increase in number of projects using NCS from 19% in 2006 to 57% in 2009 Programmableweb.com : -3 new Mashups listed /day -Total 4436 Mashups listed [Programmableweb.com accessed 11/09/09] USC e-services project data shows increase in number of projects using COTS from 28% in ‘97 to 70% in ’02 - Similar results (54% in 2000) were found in the Standish Group’s 2000 survey [Yang 2006]. *

5. University of Southern California Center for Systems and Software Engineering 4 focused ICM Common Process Patterns Process Patterns Use Single Non-Developmental Item (NDI) Agile Architected Agile Formal Methods HW with embedded SW component Indivisible IOC NDI- intensive Hybrid agile/ plan-driven system Multi-owner system of systems Family of systems Brownfield Services- Intensive 5 Market –Driven, Services- Driven, NDI-Driven 03/09/2010USC CSSE ARR 2010

6. University of Southern California Center for Systems and Software Engineering Different Risk Patterns Yield Different Processes 603/09/2010USC CSSE ARR 2010 Architected Agile E.g. Business data processing Use Single NDI E.g. Accounting System NDI-Intensive E.g. Supply Chain Management Services-Intensive E.g. Community Services

7. University of Southern California Center for Systems and Software Engineering 703/09/2010USC CSSE ARR 2010

8. University of Southern California Center for Systems and Software Engineering Definitions of NDI / NCS NDI or Non-developmental Item is an item that is previously developed and available to use. It includes commercial-off-the shelf, open source product, reuse library and customer-furnished package. –Two main categories of NDI are platform-based NDI such as Internet Explorer, Eclipse, MySQL, Apache Tomcat, JDK/JSP application-based NDI such as Microsoft office, Quickbook, BusinessWorks, Barcode Generator, JMP and primopdf. NCS or Net-Centric Services is an online service available to be accessed over the Internet such as Google services, Yahoo services, Google map, Twitter, Ning.com, Gmail, Facebook, Amazon payment, online currency converter and online dictionary. –Net-Centric Services is known as web service, web application, online application, cloud computing, and software-as-a-service. 03/09/2010USC CSSE ARR 20108

9. University of Southern California Center for Systems and Software Engineering NDI, NCS characteristics 9 CharacteristicsNDINCS Platform IndependentYes / NoYes Required Internet AccessYes / NoYes Common StandardNoYes Option of rejecting next release*YesNo Change / upgrade controlClient /Server’s siteServer’s site End user has the latest versionYes / NoYes Database OwnershipYesYes/No * Will you be able to freeze the version you are using? 03/09/2010USC CSSE ARR 2010

10. University of Southern California Center for Systems and Software Engineering Existing Process Guidelines on Process Patterns Process PatternsCurrent process guidelines Architected Agile None Use Single NDI Early user-programming guidance [Scaffidi 2009] focuses on tailoring the selected NDI NDI- Intensive Various process guidelines USC CSSE COTS-Based Application Development Guidelines (2005) CMMI-COTS-Based Systems Services- Intensive None CMMI-SVC focuses on how to develop web services; not how to apply the available services Pure Agile Scrum, XP, Crystal, others Not a good fit for turnkey systems 1003/09/2010USC CSSE ARR 2010

11. University of Southern California Center for Systems and Software Engineering Process Pattern Decision Driver Decision Criteria Importance Architected Agile Use Single NDI NDI-Intensive Services- Intensive Alternatives More than 30% of features available in NDI/NCS 0 – 12 – 33 – 4 Has a single NDI/NCS that satisfies a complete solution 0 – 142 – 3 Very unique/ inflexible business process 2 – 40 – 1 Life Cycle Need control over upgrade / maintenance 2 – 40 – 1 Rapid Deployment; Faster time to market 0 – 142- 42 – 3 Architecture Critical on compatibility 2 – 43 – 41 – 32 – 4 Internet Connection Independence 0 – 4 0 Need high level of services / performance 0 – 40 – 3 0 – 2 Need high security 2 – 40 - 40 – 40 – 2 Asynchronous Communication0 – 4 0 Access Data anywhere 0 – 4 4 Resources Critical mass schedule constraints 0 – 13 – 42 – 32 – 4 Lack of Personnel Capability 0 – 23 – 42 – 42 – 3 Little to no upfront costs (hardware and software) 0 – 22 – 4 3 – 4 Low total cost of ownership0 – 10 – 3 3 – 4 Not-so-powerful local machines1 – 41 – 30 – 43 – 4 11 Note: Decision importance scale varies from project to project Decision Criteria Rating Scale; 0:Very Low; 1:Low; 2: Medium; 3:High; 4:Very High Importance Rating Scale: 1:Low; 2: Medium; 3:High 03/09/2010USC CSSE ARR 2010

12. University of Southern California Center for Systems and Software Engineering An Example of a team that follows the Architected Agile Process Pattern : Shields For Family Project – Develop various reports for LA city-based Family Housing Project 03/09/2010USC CSSE ARR 201012 Decision Criteria Importance Architected Agile Alternatives More than 30% of features available in NDI/NCS 11 Has a single NDI/NCS that satisfies a complete solution 10 Very unique/ inflexible business process 22 Life Cycle Need control over upgrade / maintenance 33 Rapid Deployment; Faster time to market 1 1 Architecture Critical on compatibility 3 2 Internet Connection Independence 3 1 Need high level of services / performance 23 Need high security 22 Asynchronous Communication 2 3 Access Data anywhere 3 3 Resources Critical mass schedule constraints 2 1 Lack of Personnel Capability 2 2 Little to no upfront costs (hardware and software) 2 2 Low total cost of ownership 2 3 Not-so-powerful local machines 2 2

13. University of Southern California Center for Systems and Software Engineering An Example of a team that follows the Architected Agile Process Pattern 03/09/2010USC CSSE ARR 201013 Architected Agile High importance level Low importance level Use single NDI Project Status NDI-Intensive Services -Intensive

14. University of Southern California Center for Systems and Software Engineering The Incremental Commitment Model – Electronic Process Guide (ICM EPG) Using IBM Rational Method Composer to build a software development process guideline mainly describing roles, responsibilities and artifacts. 03/09/2010USC CSSE ARR 201014

15. University of Southern California Center for Systems and Software Engineering Preliminary Results Guidelines \ Average Effort (hours) Paper-based process guidelines ICM EPG Learning15.211.9 Documentation69.246.6 Total Effort209.57175.8 03/09/2010USC CSSE ARR 201015 28% of projects are able to deliver in one semester, instead of two semesters comparing to 12.5% from previous year Clients are highly satisfied with the project results Less effort spent in project development

16. University of Southern California Center for Systems and Software Engineering Process Adoption Statistics # of TeamsResults 8 /14Selected the right process from the beginning 3/14 Selected wrong process due to unclear project scope Selected right process after exploration, more prototyping, found available NDI/NCS 2/14Changed process due to minor project scope changes 1/14Changed process due to infeasible project implementation 03/09/2010USC CSSE ARR 201016

17. University of Southern California Center for Systems and Software Engineering Conclusion For rapid-fielding software projects, this research focuses on developing and evaluating the effectiveness of process guidelines for 4 process patterns. The approach is being used to validate and refine the content of the process patterns 03/09/2010USC CSSE ARR 201017

18. University of Southern California Center for Systems and Software Engineering Differences between NDI and NCS (1/2) Category Non-Developmental Item [ includes open source, customer-furnished software] Net-Centric Services Payment  Non-commercial items usually have no monetary cost  Expensive initial costs, moderate recurring fee, training fee, licensing arrangement-dependent  Not all services are free  Low initial costs, moderate marginal cost, duration depending license Platform  Specific and limited to specific platform / language  Generally supported on a subset of platforms or multiple platforms but with different editions  Platform and language independent  Server and client can work on different platform  Interaction between machines over a network Integration  Generally more tightly coupled  Not very flexible on existing legacy systems when proprietary standard is used  Difficult when platform dependent and different technologies involved  detailed documentation and on-site extensive support  More loosely coupled, Common web standards, flexible to integrate  Requires internet access  Support forums and API documentation available  This integration could be done merely in code, without additional installation of external components Changes  Able to freeze the version, under user control  Designed for specific use so costly for customization and change  Change on server side doesn’t impact the client side  Major releases once in while  Requires end user intervention to upgrade  Changes are out of developers’ control  Not easy to predict change, cannot avoid upgrade  The end-user has the latest version of the service  Change on server side result in client side, not require user intervention  Minor releases frequently (through patching) 03/09/2010USC CSSE ARR 201018

19. University of Southern California Center for Systems and Software Engineering Differences between NDI and NCS (2/2) Category Non-Developmental Item [ includes open source, customer-furnished software] Net-Centric Services Extensions  Only if source is provided and the license permits  Delivered to the end-user by the producer  may not be portable across COTS or compatible with future releases  Extension is limited to data provided by the web services  In-house extension such as wrapper or mashup  Little control over performance overhead Evaluation Criteria  Maintenance, extensibility, scalability, reliability, cost, support, usability, dependency, ease of implementation, maintainability, upgrades, size, Access to source and code-escrow considerations  Upfront costs opposed to subscription  Platform compatibility, Feature controllability  Reliability, Availability, Cost, Available Support, Speed, Predicted longevity of the service provider, release cycle, Bandwidth  Recurring costs and future functionality offered  Standards compatibility  Feature and data controllability Support Services  sometimes available for a fee  Help topics or FAQs would likely not be updated after installation  Upgrades/Patches and data migration support  Sometimes can be customized for specific user  Upgrade through purchasing new releases  generally not available  Help topics would generally be frequently updated; self- learning  Usually not customized for specific user  Patching on service provider’s side; mostly does not require installation on client side Data  Data often stored locally. Backups by the user  Data access is generally fast  Possible variety of proprietary formats, Platform- dependent  May be inflexible for change but more secure  Can process data offline  Data stored on service host’s servers. Backups by the provider. Introduces privacy and data-retention  Data access could be slower since internet based, Process data online  Common XML using web standard protocols  Data from different web services can be used by a single client program 03/09/2010USC CSSE ARR 201019

20. University of Southern California Center for Systems and Software Engineering An Example of a team that follows the “NDI- Intensive” Process Pattern : SPC website Project – Develop a website for an organization by using “wordpress” and some additional custom code 03/09/2010USC CSSE ARR 201020 Decision Criteria ImportanceNDI- Intensive Alternatives More than 30% of features available in NDI/NCS 3 4 Has a single NDI/NCS that satisfies a complete solution 2 3 Very unique/ inflexible business process 1 1 Life Cycle Need control over upgrade / maintenance 2 1 Rapid Deployment; Faster time to market 33 Architecture Critical on compatibility 12 Internet Connection Independence 11 Need high level of services / performance 1 2 Need high security 1 2 Asynchronous Communication 21 Access Data anywhere 33 Resources Critical mass schedule constraints 23 Lack of Personnel Capability 32 Little to no upfront costs (hardware and software) 33 Low total cost of ownership 33 Not-so-powerful local machines 33

21. University of Southern California Center for Systems and Software Engineering An Example of a team that follow the NDI-Intensive Process Pattern 03/09/2010USC CSSE ARR 201021 Use Single NDI Architected Agile NDI - Intensive Services - Intensive High importance level Low importance level Project Status