Beta Testing: The Contractor’s Perspective Trns·port User Group Meeting October 2005

Overview What is beta testing? User participation Testing standards Beta objectives Roles and responsibilities Beta support Benefits Success factors

What is Beta Testing? Product testing and assessment that is performed by real users in real environments and using real data prior to production release. Beta testing is required by AASHTOWare ® standards for all new development, unless exception is granted.

User Participation Customer input into a product is critical to building a quality system. Beta testing provides opportunity for testing in the user environment with real user data and user scenarios.

Testing Standards Unit Testing: Informal unit tests by the programmers Build Testing: Combine and test the aggregated units System Testing: Test software as a whole Alpha Testing: Contractor tries to break the product, validate against requirements, review documentation Beta Testing: Licensees use the product as a production application with real data Acceptance Testing: meet the final requirements

Beta Objectives Validate software functionality and installation in real environments Test according to agency work flows Test with high volume of data Uncover defects and assess overall quality Prepare production-ready software

Key Roles Task Force (TTF) – Oversight of all software testing phases. Approves beta test completion. Technical Review Team (TRT) – Oversight of beta progress. Guidance on issue resolution. Led by a Task Force member. Beta Sites – Conduct testing according to test plan and agency workflows. Test specific releases and platforms. End User Designee signs off at beta end. Contractor – Provides software, documentation, test plans and beta support. Resolves issues and delivers production software.

Contractor’s Responsibilities Provide installation support, on-site if needed Provide remote support Deliver software updates during the beta period, if needed Resolve all reported defects Report status to the TRT and TTF Finalize production software

Beta Site’s Responsibilities Schedule beta resources and assign test leads Establish test environment Conduct testing according to test plans Conduct testing of agency workflows Report defects to contractor Report final test results to the TRT and TTF

Beta Plan Document Beta schedule Beta release plan –Software –Test plans, data, test cases –Documentation Beta support process –Issue reporting and tracking – Collaboration site Acceptance Process

Beta Progress Flow

Beta Support Tools Project collaboration space (Groove) – Meeting agendas –Project artifacts –Prototypes and test plans Issue reporting and Tracking –Simple web forms –TeamTrack ® – a future standard Other collaboration tools –Preview web sites –List Serves and Beta Test Results Form

Groove collaboration tool is used on most projects

Beta reporting web page

TeamTrack tool will standardize beta issue reporting

Benefits of Beta Testing Validation of requirements in user environments Early detection of defects User participation and ownership Early installation and configuration Close collaboration with the TRT and project team Reduction in warranty, maintenance, and support

Success Factors Participation of agencies with the right Trnsport platforms and module versions in production Beta readiness and completion of testing Adequate duration of beta test Multiple beta participants High level of communication Effective collaboration tools

Questions ? Questions ?