Agile Software Development: What’s Really Going On Scott W Agile Software Development: What’s Really Going On Scott W. Ambler Practice Leader Agile Development

Scott Ambler - Background Practice Leader Agile Development Fellow – International Association of Software Architects www-306.ibm.com/software/rational/bios/ambler.html

Presentation Overview Warning! Agile Software Development (ASD) Agile Adoption Rates Going Beyond the Extreme Rhetoric

Warning! I’m spectacularly blunt at times Many new ideas will be presented Some may not fit well into your existing environment Some will challenge your existing notions about software development Some will confirm your unvoiced suspicions Don’t make any “career-ending moves” Be skeptical but open minded

Agile Software Development (ASD) What is ASD? How it’s different Mythbusters Why does ASD Work? Some Common Practices

What is Agile? Core principles An iterative and incremental (evolutionary) approach performed in a highly collaborative manner with just the right amount of ceremony to produce high quality software in a cost effective and timely manner which meets the changing needs of its stakeholders. Core principles “Fits just right” process Continuous testing and validation Consistent team collaboration Rapid response to change Ongoing customer involvement Frequent delivery of working software Agile can mean different things to different people, so it can be helpful to create some common ground to avoid misunderstandings. As IBM Rational has worked with companies who are undertaking Agile Development, we’ve made some key observations: Agile software development can be significantly different from one organization to another. Agile is not a “one size fits all” proposition Agilists do a lot more testing, in fact they often write a test before they write sufficient production code to fulfill that test, then they iterate Agilists work very closely together and ideally work closely with their stakeholders Changing requirements are seen as a competitive advantage, as long as you can act on them, not as something that you need to prevent Agilists deliver working software every iteration, providing concrete evidence of actual progress on the project. Daily builds, or even multiple times a day are highly desirable Shorter iterations are desirable, from as short as one-to-two weeks, 4 weeks as a common recommendation, although up to 8 weeks will occur at scale

How Agile is Different Focus on collaboration: Less paperwork and more conversation Stakeholders actively involved Focus on working software: Greater feedback makes agile projects easier to manage Less documentation is required Less bureaucracy Agilists are generalizing specialists: Less hand offs between people Less people required Specialists find it difficult at first to fit into the team Agile is based on practice, not theory: This is a significant change from traditional You need to see how agile works in practice to truly understand it

Mythbusters Myth Reality No Documentation Undisciplined No Planning Not Predictable Does Not Scale Is a Fad Silver Bullet RUP isn’t agile Not Fixed Price Reality Agile Documentation Requires great discipline Just-in-time (JIT) planning Far more predictable Eclipse is agile It’s quickly becoming the norm It requires skilled people RUP is as agile as you make it Agile provides stakeholders control over the budget, schedule, and scope

Why Agile Works www.ambysoft.com/essays/whyAgileWorksFeedback.htm

Some Common Practices Regular Deployment of Working Software Pair Programming Refactoring Continuous Integration Test Driven Development (TDD) Shared Code Ownership Active Stakeholder Participation

Have you Adopted Agile? Number of Projects Run Success Rates Agile Adoption Rates* Have you Adopted Agile? Number of Projects Run Success Rates *Figures from an April 2007 Survey to be summarized in the August 2007 issue of Dr. Dobb’s Journal

Has Your Organization Adopted One or More Agile Techniques?

Number of Agile Projects Run

% of Successful Agile Projects

Going Beyond the Extreme Rhetoric Modeling and Documentation Rational Unified Process (RUP) Testing Database Refactoring Database Regression Testing Governance

Agile Model Driven Development (AMDD) Project Level www. agilemodeling Agile Model Driven Development (AMDD) Project Level www.agilemodeling.com/essays/amdd.htm

Agile Documentation Document the stable, not the speculative Agile documents: Maximize stakeholder ROI Describe “good things to know” Have a specific customer and facilitate the work efforts of that customer Are sufficiently accurate, consistent, and detailed

Rational Unified Process (RUP)

Agile Testing www.ddj.com/dept/debug/196603549?cid=Ambysoft Regression testing is critical to the success of evolutionary (iterative and incremental) development Acceptance tests are considered to be primary requirements artifacts Unit tests are considered to be detailed design artifacts

Database Refactoring A database refactoring is a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. Examples: Move Column, Rename Table, and Replace Blob With Table. A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers. Important: Database refactorings are a subset of schema transformations, but they do not add functionality. www.agiledata.org

Database Testing www.agiledata.org/essays/databaseTesting.html

Agility at Scale: “Right-Sizing” Governance Pragmatic Governance Body Staged Program Delivery Manage Project Pipeline By Business Value Scenario-Driven Development Iterative Development Adapt The Process Risk-Based Milestones Continuous Improvement Embedded Compliance Simple And Relevant Metrics Continuous Project Monitoring Organization & Meetings Processes Mission & Principles Measures Other industries has gone through a major paradigm shift over the last 50 years. In the 70s, Toyota launched the Toyota Manufacturing Systems which revolutionized how the car industry is run, with concepts such as self-organized teams, continuous improvements (kai-zen), and more agile governance structures, where e.g. any factory worker was allowed to stop production if they say a quality problem. Similarly, Walmart has lead a revolution in the retail industry where you have much more of a collaborative governance model with supplier and buyer working hand in hand to enable an adaptive model enabling inventories to be adjusted rapidly to meet demands and fluctuations caused by hurricanes or holidays. In the same way, modern military has moved from a strict command and control to a structure of steer by objective and self governed teams with accountability. A platoon is given the authority to adapt to reality and find the best possible path to accomplish established objectives. We are starting to see the same paradigm shift in the software industry, and we have captured a set of practices that captures this new approach to governance, or "right-sizing governance". INTRO TO THIS SLIDE: Traditional governance is often based on a command-and-control mindset, something that doesn’t work well for intellectual workers. Traditional governance is often described as trying to herd cats. You can’t herd cats. To govern Agile projects, and in an effective manner in general, you want to: Take a collaborative approach with IT professionals Involve them with decisions, respect their expertise The key is that governance will likely happen whether you want it to or not. The key is to be PROACTIVE about governance instead of REACTIVE. That way you will have greater influence over how the organization will address these issues. DETAILED DISCUSSION OF Practices/Patterns: Pragmatic Governance Body – cost effective techniques, focus on enabling development teams, not controlling them Staged Program Delivery – roll out the program in chunks / increments. Subprojects must sign up to release date. If subproject misses, it skips to the next release, but this minimizes the impact to customers. Manage Project Pipeline by Business Value – invest in the projects that make the most sense, with less focus on “cool technology” Iterative Development – incremental approach to development. Build a system in time-boxes, not as a single-big bang effort Adapt the Process – one process size does not fit all, tailor the process to meet a team’s exact needs. Processes must be evaluated and evolve over time. Shouldn’t have a “because we’ve always done it that way” mentality Risk-based Milestones – huge feature of RUP, Mitigate business, technical, … risk early in the lifecycle Continuous Improvement – Identify and act on lessons learned throughout the project, not just at the end Embedded Compliance – Build in compliance into your day-to-day process, do not have a separate compliance process which causes unnecessary overhead. Automation is critical Simple and Relevant Metrics – Automate metrics collection, minimize the number of metrics collected, know why you’re collecting them Continuous Project Monitoring – Use automated metrics to monitor projects, identify potential issues and work with teams to resolve them early Integrated Lifecycle Environment – Automate as much of the drudge work as possible, tools and processes should fit together effectively throughout the lifecycle. Valued Corporate Assets – People should follow guidance, reuse assets, and conform to common infrastructure because these things add value, not because they’re forced to do so. Make it easier to follow existing guidelines rather than creating their own. Flexible Architectures – SOA, components, objects, patterns, … lead to greater levels of consistency, reuse, enhancability, and adaptability Self-Organizing Teams – IT professionals are intelligent people who can determine their own strategies for working, for planning their work within established parameters (such as iteration boundaries) and are provided the right coaching. Align HR Policies with IT Values – Hiring, retaining, and promoting technical staff requires different strategies than non-technical staff. Promote rewards for people to learn new skills. Limit bureaucracy, and put incentives/rewards in place for timely delivery or other key accomplishments. Align Organization Structure With Architecture – Distributed teams motivate you to have partitioned architectures Roles & Responsibilities Policies & Standards Self-Organizing Teams Align HR Policies With IT Values Align Organization Structure With Architecture Integrated Lifecycle Environment Valued Corporate Assets Flexible Architectures

Scott W. Ambler www-306.ibm.com/software/rational/bios/ambler.html Keep In Touch! Scott W. Ambler www-306.ibm.com/software/rational/bios/ambler.html

References and Recommended Reading www.agilealliance.com www.agilemodeling.com www.agiledata.org www.ambysoft.com www.databaserefactoring.com www.enterpriseunifiedprocess.com Ambler, S.W. (2002). Agile Modeling: Effective Practices for XP and the UP. New York: John Wiley & Sons. Ambler, S.W. (2003). Agile Database Techniques. New York: John Wiley & Sons. Ambler, S.W. (2004). The Object Primer 3rd Edition: AMDD with UML 2. New York: Cambridge University Press. Ambler, S.W. and Sadalage, P.J. (2006). Refactoring Databases: Evolutionary Database Design. Reading, MA: Addison Wesley Longman, Inc. Larman, C. (2004). Agile and Iterative Development: A Manager’s Guide. Reading, MA: Addison Wesley McGovern, J., Ambler, S.W., Stevens, M., Linn, J., Sharan, V., & Jo, E. (2003). The Practical Guide to Enterprise Architecture. Prentice Hall PTR.