Walking the Tight-Rope Software Project Management in Real Life Urbán Zoltán, Várszegi György 2004

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi2 Introduction  There are well established theories about project management. In practice they are easiest to follow for »small-sized independent (demo) projects »government financed mega-projects  The real challenge is to manage multiple projects in parallel in a competitive environment under time, budget and resource pressure  Using well-established general practices can still reduce the risks

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi3 Who we are  ScanSoft, Inc. is a public company based in Boston, MA. With nearly 800 employees worldwide, it is the market-leading supplier of speech and imaging solutions.  ScanSoft-Recognita Rt. in Budapest is its only imaging development hub. Size of its engineering is about 90 heads  The ratio between development and QA is close to 2:1

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi4 Who we are  Our product portfolio is: »OmniPage: stand-alone OCR »PDF Converter Pro: opening and creating PDF files »PaperPort: document management system »OmniPage SDK: imaging development toolkit »OmniForm: form designer and data acquisition solution

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi5 Engineering Process  We have two basic lines of work: »Retail (boxed) products –a classic project management concept to deliver major product versions »Customization projects –a relatively high number of minor projects controlled by available resources and priorities defined by sales  We will concentrate on the first type in this presentation

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi6 Roles  For retail product projects the customer is not the contracting party for the development »Product Delivery Team (PDT) –Defines and delivers the product –A cross-functional group of area representatives  Program manager: the coordinator to drive toward team consensus  Product manager, marketing, international  Project manager, QA Lead, documentation, engineering  Technical support  Manufacturing »Product Approval Committee (PAC) –Approves, finances and supervises the project –Senior management of the company

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi7 Engineering Phases / Milestones  Strategic vision »Kick-off PAC review  Product definition »Product and market definition PAC review  Product design »Design PAC review  Coding »Alpha acceptance

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi8 Engineering Phases / Milestones  Alpha »UI freeze »Beta readiness PAC review »Beta acceptance  Beta »First release candidate (RC0) »Launch PAC review »Release to manufacturing (RTM)  Sustaining »RTM of localized versions, patches, point releases

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi9 PAC reviews  Regular PAC reviews »Planned delimiters of the phases »PDT demonstrates the current state and the successful termination of the previous phase »PDT presents a detailed plan for the rest of the project –Engineering: deliverables (25 varieties), schedule (100 date points), resource plan, technical overview, quality plan, documentation, localization, 3 rd party components, risks »PAC decision can be –GO (contract for the next phase) –Redirect (major change in the course of the project) –Retry the review (inadequate input from PDT) –Cancel the project

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi10 PAC reviews  Exception reviews »Any material change in the course of the project that affects the “contract” for the current phase should prompt an exception review –Unexpected major market changes –Missed milestones –Budget overrun »Usually PDT (Program Manager) initiates them »The review materials and the PAC decision criteria are basically the same as at the regular reviews

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi11 Life Cycle / Definition, Design  Our approach to these phases is iterative  Initial Marketing Requirements Document »Product marketing creates it. MRD0 for a 60 man-year project is typically less than 10 pages with about 60 new product features with very few details.  Functional Specification »Engineering has pretty wide freedom in implementation details »Feature-based (short and comprehensive descriptions, technical details, licensed components, test methodology, benchmarking, use cases, risks) »Incremental for existing product lines

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi12 Life Cycle / Definition, Design  Feature matrix »Important communication and decision-making tool –To set realistic project goals (feedback to MRD) –To follow the coding progress until Alpha (weekly updates) »Required resources for each feature –Rows: features with owners and links to the feature descriptions –Columns: resource names –Cells: necessary man weeks (special skills considered) »Available resources –Considering holidays, other projects, project management »Padding 25-50% –for target features, unforeseen events, underestimated features

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi13 Life Cycle / Definition, Design  Feature matrix (cont.) »Priority of the feature –Minimal (committed) / Target (may be realized) –Dropped (by marketing or a target feature during coding)  Feature meetings »Useful to understand the feature implementation details and their effects »3-4 features discussed daily with –Project manager –Director –Developer(s) implementing the feature –QA Lead –Technical writer

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi14 Life Cycle / Coding  Coding (Implementation) »High risk items first –New 3 rd party items or new versions of them should be definitely taken care of first »Experts with unique knowledge –Very effective and very vulnerable »Technology test group within development –But unit tests are not a general rule »No obligatory coding standard »Currently code review only for new hires –Extension under preparation –Planned after Alpha, based on bug statistics

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi15 Life Cycle / Coding  Coding (cont.) »Unified installer configurations »Avoid sharing components run-time between separate products –It increases complexity a lot  Version control »Using unified folder structures  Build process »In-house automated nightly build process with labeling, binary versioning and error reporting (including automated test script results after Alpha)

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi16 Life Cycle / Coding  QA preparation »Test Case List –A list of all the project-related planned / implemented Test Cases »Test Case –One suite of manual test steps to check a specific item of product functionality  We write them in the form of test automation scripts »Test Matrix –Plan / report about the performed test steps  who performed the test, which test case, when, how long it took, operating system, build id, bugs reported

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi17 Life Cycle / Coding  QA preparation (cont.) »Number of Test Cases –No theoretical upper limit. A higher number yields better coverage, but raises costs. –We plan weekly test cycles. Test cases are sized to fit all functionality tests under one operating system into one week for one tester. –We usually test on 5 operating systems »Basically finalized by mid-Alpha –Usually tuned even later based on the test results, external beta test reports, random tests

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi18 Life Cycle / Coding  The trend of the number of man-weeks necessary to implement the minimum and target features predicts Alpha date

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi19 Life Cycle / Alpha  Milestones »Alpha acceptance test –Run on Alpha-candidate(s) –Test cases to check feature availability –Benchmarked parameters with Alpha thresholds  About 25 parameters: accuracies, speeds, stability, leak etc. »Marketing review –Last-minute call for marketing to tune the UI and the features –Resources reserved for an iteration cycle »UI Freeze –Finalize program resources then user guide and help  These items are usually on the critical path for the localized releases –Localization starts

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi20 Life Cycle / Alpha, Beta  The primary goal is to detect and fix bugs  Based on years of experience in three different companies, both the Alpha and the Beta phase should be 10 weeks long  Compression disorganizes the process and finally results in at least the same amount of time  Bug track database is used with a bug fixing workflow »Allowed state transitions per role, change history, on- line reports on the intranet

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi21 Life Cycle / Alpha, Beta  Tools / QA »Usually 3 computers for each tester »Disk images for each platform and hardware »Test automation –It has become more relevant recently –Tests are run on about 20 machines 24 by 7 –Script development  Starts only in Alpha because it is very sensitive to structural changes  Implemented test steps are commented out from the test cases so manual testers do not perform them  Pretty expensive to prepare them for all kinds of failure situations and to give a meaningful test report  During script development a high portion of bugs is detected  Scripts pay off in the regression tests of the sustaining phase

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi22 Life Cycle / Alpha, Beta  Tools / Developers »Test machines with disk images and multi-OS emulators are used to reproduce bugs without disturbing QA »Special software tools to fix hard-to-isolate problems –Memory over- and underwrites –Memory leaks –Threading problems »Using common components for –Logging (run-time selectable level) –Setting debug variables run-time –Letting system-level exceptions be caught in JIT debugger

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi23 Life Cycle / Beta  Beta acceptance test »Run on Beta candidate(s) »Using all test cases »Benchmarked parameters with Beta thresholds  External Beta cycles »Managed by technical support »Base test scripts prepared by QA »We usually have 2-3 external beta cycles »Important to get test results from –Real-life, non-clean software environments –Machines with software and hardware components not available in  house

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi24 Life Cycle / Beta The trend of benchmarked parameters »Predicts dates when Alpha/Beta/RTM thresholds can be met »Especially useful considering historical data trends e.g.: improvement in the last 10 weeks

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi25 Life Cycle / Beta Statistical analysis of the bugs »Most useful before RC »Comparison with historical data results in more reliable estimates e.g. turning point in open bugs happened 12 weeks before RTM for the previous version

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi26 Life Cycle / Beta  Bug meetings »Low-priority bugs are requested to be deferred to reduce the number of changes and the associated risk »The PDT (mainly technical support) decides to accept or refuse the request »Daily meetings in the last test cycle(s)  Release candidate »Very important milestone –Does every participant believe that the build, with all its known problems, could be released, if the next test cycle does not detect any new bug? »Check-in only through the project manager »Tests performed from the final media

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi27 Life Cycle / Sustaining  Deliverables »Localized versions, trials, point releases, patches  Team »Separate small team of 2-3 developers »Very QA-intensive period. Typically 5 operating systems and max. 2-3 languages tested in parallel.  Archival »Seems to be simple so long as no-one tries to recreate the build (tools, instructions, environment) »Current version control system is not reliable enough »Sources on DVD and masters on CD kept safe in 2 copies geographically separated

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi28 Project control  ISO 9001:2000 and Tick’IT certification »External audits twice a year »Internal audits 2-3 times a year »Control documents on the intranet  Software Development Handbook »Describes the development process »Specifies locations for tools, documents, source code, defect database etc. »Very useful for new hires  Our project quality plan is expressed in a life- cycle form on the intranet with all the milestones and the placeholders for the required documents

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi29 Project control  Build reports »Informs development and QA about the location and status of the current build and the known issues  Test reports »QA summarizes its findings in the form of a test report about important builds (Alpha, Beta, RC)  PDT conference calls once a week  Detailed project status reports every second week

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi30 Deviations  Schedule compression »Usually a time-to-market requirement  Losing resources »Mainly to a higher priority project  Creeping features »Changes in market conditions or late discoveries about cross-effects may cause features to creep  3 rd parties and outsourcing »For economical reasons suppliers with much inferior quality systems may be selected

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi31 Deviations  Dealing with deviations »Cutting administrative corners “temporarily” –No process revisions –Skipping reports –Not updating project documents –Putting archival tasks aside »Design –Feature bargaining »Coding –Using padding and dropping (target) features »Alpha and Beta –Shortening test cycles (but not reducing their number) –Hiring more temps (typically QA)

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi32 Conclusions  Our engineering process is far from being optimal from a project management point of view »Other presentations try to describe such ideal methods »Listening to these, you’d probably feel guilty as we did for not following all their instructions and advice  However, we believe our approach is closer to optimal economically  The evidence for this may be the number of our successfully delivered projects  We know that behind the implementation and operation of each small step towards an ideal plan there lies “blood, sweat and tears”