Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating.

Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating and Integrating Agile and Plan-Driven Methods Based on Balancing Agility and Discipline: A Guide for the Perplexed, B. Boehm and R. Turner, Addison Wesley, 2004

Outline Tutorial learning objectives
Survey of assumptions about participants General software trends and implications Sources of perplexity about agile, plan-driven methods Overview of agile and plan-driven methods XP, TSP “days in the life” Comparisons of differences, strengths, weaknesses Risk-based balance of agility and discipline Small, medium, large examples Observations and Way Forward Hands-on exercise Conclusions; review of learning objectives 4/19/04

Tutorial Learning Objectives
Practitioners and managers Understand agile, plan-driven method strengths, difficulties Learn risk-based approach to tailor hybrid methods Practice in formulating organizational strategy Researchers Understand research issues, opportunities Techniques for achieving rapid quality Empirical analyses Educators Understand differences in agile, plan-driven approaches Understand teaching challenges, opportunities Survey courses, project courses 4/19/04

Assumptions About Participants - Surveyed at Tutorial
Generally familiar with plan-driven methods Waterfall, incremental, spiral, RUP, PSP/TSP Software CMM, CMMI, SPICE, ISO I2207 Less familiar with agile methods XP, Scrum, Crystal, DSDM, FDD Representative of mixed domains Large/small projects, research, education Variety of software applications domains Increasing need for high software dependability Increasing software complexity, speed of change 4/19/04

Information Technology Trends
Current/Future Trends Everything connected (maybe) Rapid requirements change COTS capabilities determine rqts. No control over COTS evolution Ever-decreasing cycle times Outsourced jobs Failures critical Complex, adaptive, emergent systems of systems Adaptive process models Traditional Development Standalone systems Stable requirements Rqts. determine capabilities Control over evolution Enough time to keep stable Stable jobs Failures noncritical Reductionist systems Repeatability-oriented process, maturity models 4/19/04

Background Two approaches to software development
Disciplined (SW-CMM, document-based, heavy process) Agile (XP, tacit knowledge, light process) Both have strengths and weaknesses Agile and disciplined proponents are believers Many of us are perplexed 4/19/04

Key Definitions Agile method – Discipline – (per Webster)
one which fully adopts the four value propositions and twelve principles in the Agile Manifesto. Discipline – (per Webster) control gained by enforcing obedience or order; orderly or prescribed conduct or pattern of behavior; self-control. Plan-driven – a description for disciplined methods (order is often defined in plans) Plan – (per Webster) a method for achieving an end (a process plan); an orderly arrangement of parts of an overall design (a product plan). We assume that such plans are documented. 4/19/04

The Agile Manifesto We are uncovering better ways of developing
software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. 4/19/04

Plan-Driven Methods Are Evolving Too
Evolutionary acquisition and spiral development U.S. DoD Directive , Instruction Risk-driven level of detail Of processes: peer reviews vs. Critical Design Reviews Of products: GUI prototypes vs. specifications Of heavyweight methods: Team Software Process, Rational Unified Process 4/19/04

Sources of Perplexity Distinguishing method use from method misuse
Citing method misuse vs. intended use Claiming XP use when simply “not documenting” “CMM Level 4 Memorial Library” of 99 2-inch binders Overgeneralization based on the most visible instances XP is Agile; CMM is Plan-Driven Multiple definitions Quality: customer satisfaction or compliance? Claims of universality The pace of IT change is accelerating and Agile methods adapt to change better than disciplined methods therefore Agile methods will take over the IT world. Software development is uncertain and the SW-CMM improves predictability therefore all software developers should use the SW-CMM 4/19/04

Sources of Perplexity (2)
Early success stories Chrysler project that successfully birthed XP was later cancelled Cleanroom has never made it into the mainstream Purist interpretations (and internal disagreements) “Don't start by incrementally adopting parts of XP. Its pieces fit together like a fine Swiss watch“ "An advantage of agile methods is that you can apply them selectively and generatively." "If you aren't 100% compliant with SW CMM Level 3, don't bother to bid" "With the CMMI continuous interpretation, you can improve your processes in any order you feel is best." 4/19/04

Various Agile Methods Available
Adaptive Software Development (ASD) Agile Modeling Crystal methods Dynamic System Development Methodology (DSDM) * eXtreme Programming (XP) Feature Driven Development Lean Development Scrum 4/19/04

XP Principles – I Philosophy: Take known good practices and push them to extremes “If code reviews are good, we’ll review code all the time” “If testing is good, we’ll test all the time” “If design is good, we’ll make it part of everybody’s daily business” 4/19/04

XP Principles – II “If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality” “If architecture is important, everybody will work defining and refining the architecture all the time” “If integration testing is important, then we’ll integrate and test several times a day” 4/19/04

XP Principles – III “If short iterations are good, we’ll make the iterations really, really short – seconds and minutes and hours, not weeks and months and years” “If customer involvement is good, we’ll make them full-time participants” 4/19/04

XP: The 12 Practices -Kent Beck, Extreme Programming Explained, Addison Wesley, 1999
The Planning Game Small Releases Metaphor Simple Design Testing Refactoring Pair Programming Collective Ownership Continuous Integration 40-hour Week On-site Customer Coding Standards -Used generatively, not imperatively 4/19/04

XP Characteristics 4/19/04

The Team Software Process (TSP) -Watts Humphrey, Introduction to the ISP, Addison Wesley, 2000
Scale-up of Personal Software Process (PSP) To about 20-person teams Strong emphasis on planning and control 21 scripts, 5 roles/activities, 21 forms Risk-driven tailoring can make process more agile 4/19/04

Example TSP Role Script
4/19/04

TSP Characteristics 4/19/04

Days In The Life: XP and TSP
Comparison of two development teams Product: Tool to process complex sales report generated by legacy system Prototype delivered Task is to enhance and port prototype 8 month projected schedule Estimated size: 20,000 SLOC 4/19/04

Background Information
TSP/PSP Training Each member has 125-hour PSP for Engineers 2 Members 5-day launch coach training Tools and environment Web-based TSP data collection tool OO tools Individual offices and conference room Project planning 4-day launch session Develop/tailored all processes 180 tasks identified and plan accepted by stakeholders XP Training 2 members attended 40-hour XP workshop In-house presentations and mentoring Tools and environment Automated test suite development tool OO tools Bullpen and Conference room/lounge Project planning 1-day customer exploration 2-day follow-up to prioritize Agreed on 2-week iteration length, 3-iteration release length, and release schedule 4/19/04

Normal and Crisis Days Used Friday as common day
Crisis days are plan-breakers New requirements Short schedule Please read the handouts and we’ll discuss 4/19/04

Comparison Differences Similarities Bottom Line
Depth and scope of metrics Specificity of process Level and scope of reporting Customer interface Similarities Collaborative teams Well-defined roles Spiral, risk- and increment-driven process Measurement Test first (test-driven) approach Bottom Line Both were able to confidently push-back when necessary 4/19/04

Finding Middle Ground A pragmatic view
Both approaches have “home grounds” A broad range of environments and needs Trends require better handling of rapid change Processes should be the right weight for the job We believe risk-based analysis is the key to finding middle ground 4/19/04

Contrasts and Home Grounds
Comparison is difficult, but we found 4 areas where there are clear differences Application Project goals, size, environment; velocity Management Customer relations, planning and control, communications Technical How the software is developed Personnel The type and competency of developers and stakeholders 4/19/04

Application Characteristics
Primary goals A: Rapid value, responsiveness to change P: Predictability, stability, high assurance Size A: Small to medium P: Large Environment A: Turbulent, high change P: Stable, requirements defined Project velocity A: Code, learn, and fix P: Architect for parallel, incremental development 4/19/04

Management Characteristics
Customer Relations A: Dedicated, collocated; where feasible P: Contractual; increasingly evolutionary A: Trust through working software and participation P: Trust through process maturity evaluations Planning and Control A: Means to an end P: Anchor processes, communication Project Communications A: Tacit, interpersonal knowledge P: Explicit, documented knowledge 4/19/04

Technical Characteristics
Requirements A: Adjustable, informal stories P: Formally baselined, complete, consistent specifications Development A: Simple Design P: Architecture-based design Testing A: Automated, test-driven P: Planned, requirements-driven 4/19/04

Technical Characteristics (2) Cost of Change: Beck, Li
4/19/04

Technical Characteristics (3) Cost of change: Poppendieck
At least two cost-escalation curves Lean development: Architect to defer high-stakes decisions – e.g. COTS Easier to change an unmade decision Most Changes Changes in High-stakes Constraints Time Cost of Change 4/19/04

Technical Characteristics (4) Cost of Change: TRW
Beck Li 4/19/04

Technical Characteristics (5) How Much Architecting?
Beck 10,000 KSLOC Sweet spot 100 KSLOC Liu 10 KSLOC 4/19/04

Personnel Characteristics
Customers A: CRACK customers throughout development P: CRACK customers early CRACK: Collaborative, Representative, Authorized, Committed, and Knowledgeable Developers A: Heavy mix of high caliber throughout P: Heavy mix early with lower capability later Culture A: Many degrees of freedom (Thrives on chaos) P: Clear policies and procedures (Thrives on order) Collocated customers are difficult to find/keep/keep current on either side 4/19/04

Personnel Characteristics (2) Modified Cockburn Levels
Level Characteristics 3 Able to revise a method (break its rules) to fit an unprecedented new situation 2 Able to tailor a method to fit a precedented new situation 1A With training, able to perform discretionary method steps (e.g., sizing stories to fit increments, composing patterns, compound refactoring, complex COTS integration). With experience can become Level 2. 1B With training, able to perform procedural method steps (e.g. coding a simple method, simple refactoring, following coding standards and CM procedures, running tests). With experience can master some Level 1A skills. -1 May have technical skills, but unable or unwilling to collaborate or follow shared methods. 4/19/04

Summary of Home Grounds
Characteristics Agile Plan-driven Application Primary Goals Rapid value; responding to change Predictability, stability, high assurance Size Smaller teams and projects Larger teams and projects Environment Turbulent; high change; project-focused Stable; low-change; project/organization focused Management Customer Relations Dedicated on-site customers, where feasible; focused on prioritized increments As-needed customer interactions; focused on contract provisions; increasingly evolutionary Planning/Control Internalized plans; qualitative control Documented plans, quantitative control Communications Tacit interpersonal knowledge Explicit documented knowledge Technical Requirements Prioritized informal stories and test cases; undergoing unforseeable change Formalized project, capability, interface, quality, forseeable evolution requirements Development Simple design; short increments; refactoring assumed inexpensive Architect for parallel development; longer increments; refactoring assumed expensive Test Executable test cases define requirements Documented test plans and procedures Personnel Customers Dedicated, collocated CRACK* performers CRACK* performers, not always collocated Developers At least 30% full-time Cockburn level 2 and 3 experts; no Level 1B or -1 personnel** 50% Cockburn Level 3s early; 10% throughout; 30% Level 1B’s workable; no Level -1s** Culture Comfort and empowerment via many degrees of freedom (thriving on chaos) Comfort and empowerment via framework of policies and procedures (thriving on order) * Collaborative, Representative, Authorized, Committed, Knowledgeable ** These numbers will particularly vary with the complexity of the application 4/19/04

Two Case Studies Show how that agile and plan-driven methods can be extended Provide examples of success and lessons learned ThoughtWorks Lease Management Agile extended with plan-driven CCPDS-R Plan-driven overlaid with agile concepts 4/19/04

Thoughtworks Lease Management
XP replaced ineffective traditional development Problems when project moved beyond XP assumptions The effort to develop or modify a story really does not increase with time and story number Trusting people to get everything done on time is compatible with fixed schedules and diseconomies of scale Simple design and YAGNI scale up easily to large projects Disciplined practices enabled XP to scale up High-level architectural plans to provide essential big-picture information More careful definition of milestone completion criteria to avoid “finishing” but not finishing Use of design patterns and architectural solutions rather than simple design to handle foreseeable change 4/19/04

CCPDS-R USAF Command Center Processing and Display System Replacement for early missile warning Government contract environment required plan-driven approach, project needed agility Practices implemented to provide agility mapped well to the Agile Manifesto Individuals and Interactions over Processes and Tools Milestone content was redefined Architecture was organized around the performers’ skill levels Working Software over Comprehensive Documentation Later PDR demonstrated working software for high-risk areas Customer Collaboration Over Contract Negotiation Used COCOMO for cost and schedule negotiations Responding to Change Over Following a Plan Automated plans Architecture for re-use saved effort 4/19/04

CCPDS-R (2) Cost of Change
Beck 4/19/04

Five Critical Decision Factors
Size, Criticality, Dynamism, Personnel, Culture 4/19/04

Comparing the Case Study Projects
4/19/04

Misconceptions 4/19/04

Using Risk to Balance Discipline and Agility - Overview
4/19/04

Risks Used in Risk Comparison
Environmental risks Technology uncertainties Many stakeholders Complex system-of-systems Legacy compatibility Agility risks Scalability Use of simple design Personnel turnover Too-frequent releases Not enough agile-capable people Plan-driven risks Rapid change Need for rapid results Emergent requirements Not enough discipline-capable people 4/19/04

Three Examples Agent-based systems Small – Event Managers application
Medium – SupplyChain.com application Large – National Information System for Crisis Management (NISCM) application Each example results in a development strategy based on risk analyses 4/19/04

Event Managers Project
Small startup company Diverse set of smaller agent-based planning applications This project: support for managing the infrastructure and operations of conferences and conventions Widening variety of options and interdependencies, makes an agent-based approach highly attractive 4/19/04

Event Managers Profile
4/19/04

Event Managers Strategy
LCA 4/19/04

SupplyChain.com Profile
Turnkey agent-based supply chain management systems Distributed, multi-organization teams of about 50 people Parts of applications are relatively stable, while others are highly volatile Architectures are driven by a few key COTS packages that are also continually evolving 4/19/04

SupplyChain.com Profile
4/19/04

SupplyChain.com Strategy
4/19/04 LCO LCA

NISCM Profile Broad coalition of government agencies and critical private-sector organizations Support cross-agency and public-private sector coordination of crisis management activities Adaptive mobile network, virtual collaboration, information assurance, and information integration technology Private-sector system-of-systems integration contractor 4/19/04

NISCM Profile 4/19/04

NISCM Strategy 4/19/04 LCO LCA

Risk Profiles Across Examples
4/19/04

Summary of Observations
No silver bullet Clear agile and plan-driven home grounds Increased demands for agility, velocity, quality Balanced methods are emerging Build methods up vs. tailor down Methods are important; people more so Values Communications Expectations management 4/19/04

No Silver Bullet Brooks’ werewolf concerns Agile methods
Complexity, conformity, changeability, invisibility Agile methods Handle changeability and invisibility Miss on complexity and conformity Plan-driven methods Handle conformity and invisibility Miss on changeability and complexity 4/19/04

Future Applications Need Both
Historically Many small, non-critical, well-skilled, agile culture, rapidly evolving projects Many large, critical, mixed-skill, ordered culture, stable projects In the future Large projects are no longer stable Maintenance of extensive process and product plans will become too expensive Complexity and conformity werewolves are waiting for agile projects Attributes of both approaches will be needed 4/19/04

Balanced Methods are Emerging
Agile methods Crystal Orange DSDM FDD Lean Development Plan-Driven methods Rational Unified Process CMMI Hybrid Boehm-Turner Risk-based Code Science/Agile Plus 4/19/04

Build up – Don’t Tailor Down
Plan-driven methods traditionally Define heavyweight process guidelines Advocate tailoring Treat mass of processes as security blanket Agilists traditionally Begin with the minimum Add as needed (and justified by cost-benefit) Start small; extend only where necessary 4/19/04

Maybe its not the methods!
Agile movement has echoed a long line of warning calls Success of agile may be due as much to people factors as to technology Valuing people over processes is the most important factor in the manifesto I know I saw something about that in the process somewhere… 4/19/04

People Of the people, by the people, for the people
Separation of concerns is increasingly harmful A few quotes… The notion of “user” cannot be precisely defined, and therefore it has no place in computer science or software engineering. Dijkstra Analysis and allocation of the system requirements is not the responsibility of the software engineering group, but it is a prerequisite for their work. The Capability Maturity Model for Software. Software engineering is not project management. Tucker 4/19/04

Values Reconciling values is a critical people-oriented task
Stakeholders value different things Software engineering is usually value-neutral Every requirement, object, test case, defect equally important Process improvement and plan-driven methods are inwardly-focused Aimed at productivity improvement Not higher value to customer Agilists attention to prioritization and negotiation are promising 4/19/04

Communications Face it, most engineers can’t talk, much less write
Need to bridge the customer-developer communications gap IKIWISI* reigns Rapid change increases need for solid communication Few sources of guidance Cockburn’s Agile Software Development is a good starting place *I’ll know It When I See It 4/19/04

Expectations Management
Differences between successful and troubled projects is often expectations management SW developers have problems with EM Strong desire to please Little confidence in prediction Over confidence in abilities Most significant common factor in highly effective plan-driven or agile teams EM means not setting up unrealistic expectations Process mastery Preparation Courage Developers seem to like Sisyphean tasks… 4/19/04

Research Challenges and Opportunities
Techniques for achieving rapid quality Risk/value-driven process lightening Agile with lightweight V&V overlay Capitalizing on domain knowledge Reuse, component-based approaches Automating quality enhancement Empirical analyses Distributed agile methods Hybrid agile/plan-driven methods Effects of individual practices Effects of scale, personnel turnover 4/19/04

Size Distribution 60% of projects in agile home ground
60% of effort in plan-driven home ground 4/19/04

Development Methods 4/19/04

Do Both Assure Success? TSP and agile methods are indeed silver bullets and always succeed spectacularly well. People tend to report success more than they do failures. The pioneer projects are being done by high-capability early adopters of new methods. The pioneer projects are experiencing some Hawthorne effects of performing extra well while being in the spotlight. The projects are being compared to particularly poorly performing past projects. High improvement numbers make options worth exploring 4/19/04

Reifer Scorecard for XP
4/19/04

Pair Programming Results
Short studies are suspect; pairs take about 10 hours to jell 4/19/04

Educational Approaches
Survey courses Materials here plus excursions into practice Try, compare inspections and pair programming Try, compare refactoring vs. use of design patterns Project courses Some practices need tailoring to student constraints Steep learning curve; precede with survey course 4/19/04

Experimental XP-Like Course
Goals: Create an improved version of Agile COCOMOII Use XP practices; supplement where needed Gather software process data Characteristics: Part-time, non-collocated student team Part-time customer Personnel with moderate platform/applications experience 4/19/04

4/19/04

XP + situational common sense => Lightweight method
Planning variants Brainstorming +modified win-win -> e-Story/Task cards Bi-weekly meetings Stories prioritized -> iteration requirements Design variants Brainstorming:: group input:: team buy-in Architecture determined from shortfalls of previous arch. Top level design :: iteration designs Iteration design :: fulfills iteration requirements Prototype GUI, draft class design: pair-work 4/19/04

Electronic Story/Task Cards
4/19/04

XP + situational common sense => Lightweight method
Coding variants: Coding standards:: drafted, team consensus, will evolve as needed Customer available bi-weekly for feedback/report Tests are being written before any code, all code is unit & integration tested Quality guidelines: Digital archives of artifacts Simple, short, risk-based documentation 4/19/04

Organizational Way Forward
Convene key stakeholders to determine organizational goals and strategies Rebalance agility and discipline in this organizational context 4/19/04

Determining Organizational Goals and Strategies
Current software organization’s status and needs Key stakeholders’ needs and trends How to cope with future trends The increased pace of change and need for agility The increased concern with software dependability and need for discipline Your ability to satisfy your stakeholders’ evolving value propositions and to keep up with your toughest competitors The increasing gap between supply and demand for Cockburn Levels 2 and 3 people Your ability to cope with existing and emerging challenges COTS integration, evolving Internet/Web capabilities, distributed and mobile operations, agent coordination, multimode virtual collaboration, and outsourcing jobs 4/19/04

Collins’ Good to Great - HarperBusiness, 2001
Bureaucratic Organization Start-up Organization Hierarchical Organization Great Organization Low High Ethic of Entrepreneurship Culture of Discipline 4/19/04

Rebalancing Agility and Discipline
Graph “typical project” on 5D chart -Now; 5 years ahead Develop best home ground continuous process improvement strategy Treat anomalies as risks to be managed Develop incremental mixed strategy. Try on pilot project Integrate, execute process, people, technology plans. Monitor and control using Balanced Scorecard. In home ground with some anomalies Highly mixed agile/plan driven profile In agile or plan-driven home ground 4/19/04

Hands-on Exercise Graph your organization’s “typical project” on 5D charts Now; 5 years from now Identify major needs for organizational change With respect to balancing agility and discipline With respect to future trends, environmental risks Identify most critical first steps for improvement Identify pilot project for applying first steps Application, staffing Needs for education, teambuilding, culture change Summarize in bulleted strategic plan 4/19/04

Now 4/19/04

In 5 Years 4/19/04

Conclusions Plan-driven and agile methods both aim to
Satisfy customers Meet cost and schedule parameters Home grounds exist, but the need/opportunity for integration is expanding We recommend a self-assessment of your organization and business Locate your place in the home ground space Identify areas of change and how they might move your location Look for ways to integrate methods to better respond to change People concerns dominate method concerns 4/19/04

Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating.

Similar presentations

Presentation on theme: "Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating.

Similar presentations

Presentation on theme: "Barry Boehm University of Southern California Richard Turner OUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline: Evaluating."— Presentation transcript:

Similar presentations

About project

Feedback