1. 3 October 2001wscacchi@uci.edu1 Understanding and (Re)Designing Software Development Processes Walt Scacchi Institute for Software Research University of California, Irvine Irvine, CA 92697-3425 USA http://www.ics.uci.edu/~wscacchi/Presentations/JPL-Process-Oct01.ppt

2. 3 October 2001wscacchi@uci.edu2 Recurring problems What is the best way to organize software development project? How to speed up development, reduce costs and improve software quality? How to achieve the quickest development effort, lowest effective development cost, and best available product quality?

3. 3 October 2001wscacchi@uci.edu3 Possible solutions Attain and improve maturity of software development capabilities (image)image Get best people to practice standards-based development process supported by IDEs (Re)Design your software production architecture (SPA) to optimize use of development resources, processes, and people.

4. 3 October 2001wscacchi@uci.edu4 Goals Present an approach for how to optimize software production Identify key concepts, techniques, and tools that enable better optimization Describe optimization transformations from business process redesign studies Describe opportunity areas for exploitation and use

5. 3 October 2001wscacchi@uci.edu5 Definitions and Differences Software production: enterprise processes and resources that produce software Production strategies: business strategies guiding overall approach to building software Production architecture: configuration of enterprise capabilities to enact strategies Optimizing production: minimizing enterprise configuration to maximize strategic options

6. 3 October 2001wscacchi@uci.edu6 (Re)designing software production What first: to-be goal vs. as-is mess? –If you don’t know where you are, any road will do (proverb) –Observation: people at work cannot describe the processes they do with high fidelity (tacit knowledge) –Redesign necessitates understanding as-is, to-be, and here-to-there Creating high-performance work groups –Empowerment, participation, incentivization (resource sovereignty), and recognition W. Scacchi, Redesigning Contracted Service Procurement for Internet-based Electronic Commerce: A Case Study, J. Information Technology and Management, 2(3), 313-334, 2001.Redesigning Contracted Service Procurement for Internet-based Electronic Commerce: A Case Study

7. 3 October 2001wscacchi@uci.edu7 Software Production Strategies Reduce costs Reduce cycle time Improve cash flow Customer satisfaction Increase sales Improve customer service Increase productivity Open new markets Open new “channels” Be innovation leader Increase market share Enable just-in-time service delivery

8. 3 October 2001wscacchi@uci.edu8 Software Production Architecture A composite model that interrelates –software system architecture –software process architecture –development organization architecture –network infrastructure and development tools/environment configuration –documentation architecture –customer-support knowledge base architecture P. Mi and W. Scacchi, A Meta-Model for Formulating Knowledge-Based Models of Software Development, Decision Support Systems, 17(4):313-330, 1996.A Meta-Model for Formulating Knowledge-Based Models of Software Development

9. 3 October 2001wscacchi@uci.edu9 Optimizing Software Production Strategies provide global constraints or opportunities for optimizing software production Constraints and opportunities realized in software production enterprise Constraints and opportunities are distributed across the software production architecture

10. 3 October 2001wscacchi@uci.edu10 Optimizing Software Production Optimization must address composite architecture of software production Local optimization of any component architecture does not guarantee global optimality of software production Diagnostic analyses and transformation heuristics applied to composite architectural models lead to optimization opportunities

11. 3 October 2001wscacchi@uci.edu11 Optimizing Software Production Transformation heuristics classified taxonomically Taxonomy classifies domain-independent and domain-specific hueristics DI transformations applied in any software production setting DS transformations applied to specific component architectures

12. 3 October 2001wscacchi@uci.edu12 Optimizing Software Production DI transformation classes (sample): –Job scope –Worker empowerment –Organization design –Workflow streamlining –Information technology (IT)

13. 3 October 2001wscacchi@uci.edu13 Research grant justification and approval process at Office of Naval Research (c. 1995) W. Scacchi and J. Noll, Process-Driven Intranets: Life Cycle Support for Process Reengineering IEEE Internet Computing, 1(5):42-49, 1997.Process-Driven Intranets: Life Cycle Support for Process Reengineering

14. 3 October 2001wscacchi@uci.edu14 Optimizing Software Production IT transformation sub-classes (sample): –Extend IT-based support to manual process steps –Extend IT-based communication facilities to encourage information sharing activities –Extend IT-based automation to incorporate new kinds of application packages –Extend IT-based integration to interconnect and interrelate existing "islands of automation"

15. 3 October 2001wscacchi@uci.edu15

16. 3 October 2001wscacchi@uci.edu16 As-is vs. to-be process

17. 3 October 2001wscacchi@uci.edu17 Redesign/Optimization Results Reduction in procurement process cycle times of 20X, annual operational savings of $10-15M. –Via transformation and realignment of information systems, business processes, corporate strategy, and work practices in a 1-2 year time frame. –Participatory design, development and refinement of computational models of new work processes, resource configurations and work practices, together as an organizational system. W. Scacchi, Redesigning Contracted Service Procurement for Internet-Based Electronic Commerce: A Case Study, Information Technology and Management, 2(3):313-334, 2001.Redesigning Contracted Service Procurement for Internet-Based Electronic Commerce: A Case Study

18. 3 October 2001wscacchi@uci.edu18 Tools and Techniques Software process redesign case web Knowledge web for software production Process-driven intranets Organizational transformation

19. 3 October 2001wscacchi@uci.edu19 Case study: Software development teamwork Comparative analysis of software specification teamwork (in complex setting) –Five teams, 5-7 members, two-week (part-time) process that incorporates planning, formal notation, automated tool use, reusable assets, documentation tasks, and team shared responsibility.

20. 3 October 2001wscacchi@uci.edu20 What to Understand Work Structures and Shifts: Resource arrangements, historical circumstances, division of labor and expertise, etc. Work Processes: Routine, habitual or emergent patterns of how work flows among people through/onto work structures Work Practices: Behavioral discourse and social dynamics enacted through work processes

21. 3 October 2001wscacchi@uci.edu21 What to Understand Structures are domain independent constructs –Prescriptive/descriptive abstractions Processes are classes of workflow –Descriptive and derived –Prescriptive and composed Practices are instances –Descriptive, historic and situated

22. 3 October 2001wscacchi@uci.edu22 Comparative analysis of software specification teamwork –Six work structure types observed: Negotiated, Integrated, Replicated, Delegated, Prediscriminated and Separated –Three structural shift types observed: anticipated ->, unanticipated -->>, role shift within work structure +.

23. 3 October 2001wscacchi@uci.edu23 Work structures and shifts (data)

24. 3 October 2001wscacchi@uci.edu24 Findings Highest (lowest) quality product (measured by automated tools): T1 (T5) Highest (lowest) productivity (self reported time expended): T5 (T1) Note the coincidental relationship Effectiveness of planning, automated tool use, asset reuse not clearly associated with high(low) quality or high(low) productivity

25. 3 October 2001wscacchi@uci.edu25 Findings Teams falling primarily into Negotiative and Integrative structures had higher quality Teams falling primarily into Delegative, Pre-discriminative or Separative structures had higher productivity Computer supported work environments must account for teamwork structures as a usage parameter.

26. 3 October 2001wscacchi@uci.edu26 Tools and Techniques Software process redesign case web Knowledge web for software production: –Software production ontology –Taxonomy for as-is diagnosis, redesign heuristics –Best practices and lessons learned cross-linked Process-driven intranets Organizational transformation W. Scacchi and A. Valente, Developing a Knowledge Web for Business Process Redesign, Presented at the 1999 Knowledge Acquisition Workshop, Banff, Canada, October 1999.Developing a Knowledge Web for Business Process Redesign

27. 3 October 2001wscacchi@uci.edu27

28. 3 October 2001wscacchi@uci.edu28

29. 3 October 2001wscacchi@uci.edu29 Current field study Understanding open source software practices and processes in different domains –Academic research vs. Commercial development Deep Space Astronomy, Software Architecture Internet infrastructure, Networked computer games To produce and compare case studies using narrative, hypertext, and computational renderings.

30. 3 October 2001wscacchi@uci.edu30 Tools and Techniques Software process redesign case web Knowledge web for software production: Process-driven intranets: –Model, prototype, and enact software production architectures Organizational transformation

31. 3 October 2001wscacchi@uci.edu31 Process-driven intranets Enable rapid configuration of virtual enterprises (VEs) across multiple organizational domains Virtual enterprises for wide-area software development have been demonstrated Software production in VEs supported and enacted via process navigation (“process surfing”) J. Noll and W. Scacchi, Supporting Software Development in Virtual Enterprises, Journal of Digital Information, 1(4), February 1999.Supporting Software Development in Virtual Enterprises

32. 3 October 2001wscacchi@uci.edu32 Process-driven intranets Accommodate organizational autonomy and computer-supported cooperative work Accommodate heterogeneous repositories of development artifacts (source code, etc.) Accommodate use of local tools and development environments

33. 3 October 2001wscacchi@uci.edu33 Process-driven intranets Also see, J. Noll and W. Scacchi, Specifying Process-Oriented Hypertext for Organizational Computing, J. Network and Computer Applications, 24(1):39-61, 2001Specifying Process-Oriented Hypertext for Organizational Computing

34. 3 October 2001wscacchi@uci.edu34 Process-driven intranets

35. 3 October 2001wscacchi@uci.edu35 Tools and Techniques Software process redesign case web Knowledge web for software production Process-driven intranets Organizational transformation –What’s involved? –How long does it take? –How much bang for the buck?

36. 3 October 2001wscacchi@uci.edu36 Organizational transformation Collaborative participation to understand as-is, to-be, and here-to-there SPAs within constraints and contexts. Timeframe: 6-18 months Baseline investment (ROI): –External: Invest $1 to realize $10 savings/gain. –Internal: Staff time, infrastructure usage, etc.

37. 3 October 2001wscacchi@uci.edu37 Conclusions Software production can be optimized Optimizing software production is a strategic option/choice that can be realized Software production architectures and supporting technologies enable optimization Process improvement and production optimization are complementary efforts

38. 3 October 2001wscacchi@uci.edu38 References S. Bendifallah and W. Scacchi, Work Structures and Shifts: A Study in Software Specification Teamwork, Proc. 11th. Intern. Conf. Software Engineering, Pittsburgh, PA, IEEE Press, 260-270, 1989. R. Conradi and A. Fuggetta. Software process improvement: what can be improved? Politecnico di Milano, Dipartimento di Elettronica e Informazione, Submitted for publication, 2001.Software process improvement: what can be improved? W. Scacchi and P. Mi, Process Life Cycle Engineering: A Knowledge-Based Approach and Environment, Intelligent Systems in Accounting, Finance and Management, 6:83- 107, 1997.Process Life Cycle Engineering: A Knowledge-Based Approach and Environment W. Scacchi, Understanding Software Process Redesign using Modeling, Analysis, and Simulation, Software Process--Improvement and Practice, 5(2/3), 183-195, 2000.Understanding Software Process Redesign using Modeling, Analysis, and Simulation J.S. Choi and W. Scacchi, Modeling and Simulating Software Acquisition Process Architectures, Journal of Systems and Software, to appear, 2001Modeling and Simulating Software Acquisition Process Architectures Available at http://www.ics.uci.edu/~wscacchi/publications.htmlhttp://www.ics.uci.edu/~wscacchi/publications.html

39. 3 October 2001wscacchi@uci.edu39

40. 3 October 2001wscacchi@uci.edu40