Testing Web to Wireless …and Back March 2001 Presenter: Shirley DeLewis, CQA, CSTE, CPSA

© DMR Consulting 2 Confidential Information n What Were we Testing? n What’s the Product? n Architecture n Overall Test Strategy n Team Responsibilities n Test Environment n Test Strategy n Test Plans n Summary n Questions n Contact Information Agenda

© DMR Consulting 3 Confidential Information n A multi-tiered architecture with multiple components n Wireless Hand-held PC running Windows CE n Web-Based (Dispatcher) Work Order Management Application n Work Flow Engine n High Performance Middleware (because of Wireless Packet Transfers) n RiMMS (dynamic route optimization & scheduling) n Databases (SQL Server, LDAP, Oracle) n NT Servers What were we testing?

© DMR Consulting 4 Confidential Information What’s the Product: ARROW n ARROW n ARROW schedules work orders for field technicians based upon customer need, company needs, location, and technician availability n Work orders are dispatched dynamically, wirelessly, to the technician in the field n Field technicians complete their work orders, keep track of their time, record all important information, and wirelessly transfer completed work order back up to the central office

© DMR Consulting 5 Confidential Information Architecture

© DMR Consulting 6 Confidential Information Overall Test Strategy n Unit Testing/Peer Reviews by Development n PenCentra Hand-Held Field Testing plus Data Center Operations and Procedures tested by Operations Group n BUT WHAT DID THE TEST TEAM DO? n Full System Integration and Functional Testing by Test Team in Test Environment n Internal DMR Field Tests in San Diego & Dallas/Fort Worth n Customer Acceptance Testing (End-to-End with Connectivity to Client Test Lab) n Extensive Parallel Field Tests

© DMR Consulting 7 Confidential Information Test Team Responsibilities n Develop Overall Testing Strategy n Define Test Groups, Units and Test Cases n Implement Problem Tracking Tool/Procedures n Create Test Plans and Schedules n Establish/Build Test Environment n Develop Daily Build Environment/Processes n Coordinate Testing with Development Team n Discover, Report and Monitor Defects & Issues n Verify Requirements were Fully Tested n … TEST! And RETEST!

© DMR Consulting 8 Confidential Information Test Environment

© DMR Consulting 9 Confidential Information Six Parts to be Tested Scheduling Automatically Creates appointments - linked to dispatch, route optimization Dispatch Automatically sends the technician into the field, handles all normal operations, humans used for exceptions only - linked to scheduling tool and route optimizer Hand-Held Device Receives Wireless data and Instructions from Hosted application, workers perform tasks and send completed work orders--automatically updated, stored & forward - linked to all software and to support system Enterprise Connections Route Optimization Linked to dispatch tool, and scheduling tool, sends the right tech. To the right place, at the right time, doing the right things, reshuffles schedule Hosted Facility Hosts the scheduling, dispatching, routing, and database. Houses all servers, TI connections to Carrier - links scheduling, routing, dispatching, hand-helds, and reporting software 4. Reporting Screen Powerful web-based reporting screen for management of entire system. Takes information from hosting site and allows management to make critical decisions viewing: inventory, A/R, HR work process, dispatch, exception routing, issue management, etc

© DMR Consulting 10 Confidential Information How we Managed Testing the Parts n Established High Level System Integration & Verification Test Groups and Units n Developed Testing Strategy for Each Functional Segment: n Hand-Held Wireless PenCentra Hardware Tests n Dispatcher (Web) User Interface n Field Technician (Hand-Held Wireless) In-House Testing n Middleware (wireless communication piece) included: – Work Flow Engine and XML Gateway n Stress Testing (200 Hand-Held utilized) n Routing Optimization (RiMMS) Testing included – Associated mapping and address database n Operations-Infrastructure Tests n Management Reporting (about 30 reports)

© DMR Consulting 11 Confidential Information Test Plans

© DMR Consulting 12 Confidential Information Integration of Test Groups n Tracking Work Orders from the Web Interface to the Wireless Hand-held Unit in our Test Environment n Extensive Parallel Field Tests n Team members sent to San Diego and Dallas/Fort Worth to work with client dispatchers & field technicians n End-to-End Client Acceptance Test n From Client databases through our systems and back to verify accurate data transmissions utilizing client test lab – Performed as part of system Pilot phase – Also tested our databases—SQL7 to Oracle

© DMR Consulting 13 Confidential Information Master Test Plan Table of Contents Master Test Plan Table of Contents 1.INTRODUCTION n 1.1Purpose 1.1Purpose n 1.2Document Structure 1.2Document Structure n 1.3Project Overview 1.3Project Overview n 1.4System Overview 1.4System Overview n 1.5Testing Scope 1.5Testing Scope 2.ORGANIZATION & RESPONSIBILITIES n 2.1DMR Organization 2.1DMR Organization n 2.2Test Staffing and Assignments 2.2Test Staffing and Assignments n 2.3Additional Personnel & Assignments 2.3Additional Personnel & Assignments n 2.4Test Team Responsibilities 2.4Test Team Responsibilities 3.TESTING APPROACH n 3.1Environment and Tools 3.1Environment and Tools –3.1.1Test Environment3.1.1Test Environment –3.1.2Identification of Requirements3.1.2Identification of Requirements –3.1.3Test Data3.1.3Test Data –3.1.4Daily Builds3.1.4Daily Builds –3.1.5Testing Tools3.1.5Testing Tools –3.1.6Version Control3.1.6Version Control 1 n 3.1.7Problem Reporting & Change 3.1.7Problem Reporting & Change Control 4. DEFINITION OF TEST COMPONENTS 5.TESTING STRATEGIES n 5.1Unit Testing 5.1Unit Testing n 5.2PenCentra Field Testing 5.2PenCentra Field Testing n 5.3Full System Integration and 5.3Full System Integration and Functional Testing n 5.3.1Dispatcher Testing 5.3.1Dispatcher Testing n 5.3.2PenCentra In-House Testing 5.3.2PenCentra In-House Testing n 5.3.3Middleware & Stress testing 5.3.3Middleware & Stress testing n 5.3.4RiMMS testing 5.3.4RiMMS testing n 5.4Operations Infrastructure Test 5.4Operations Infrastructure Test n 5.5End-to-End Acceptance Testing 5.5End-to-End Acceptance Testing n 5.6Parallel Field Test 5.6Parallel Field Test n 5.7Management Reporting 5.7Management Reporting APPENDIX A - A Guide to PR-Tracker APPENDIX B - Issues and Triage Process APPENDIX C - P410G Integration & Verification Test Groups

© DMR Consulting 14 Confidential Information User Interface Test Plans n Specific Web Interface (Dispatcher) Test Plan n Written to ensure Web interface screens mapped to functional specs n Updated as change requests came in from client and changes made to interface n Specific Hand-Held Units Test Plan n One overall Test Plan to ensure full work order lifecycle covered n Eight individual Test Plans—one for each type of work order

© DMR Consulting 15 Confidential Information Other Test Plans n Middleware Test Plan n Test Cases and Procedures to isolate specific functionality requirements of this component n Real World Stress from production model handheld field computers n Client side heartbeat for configurable continuous load n RiMMS Test Plan n Data verification through ARROW to RiMMS relay utilizing SQL Server Stored Procedures n Typical business scenarios n Verification of RiMMS automation

© DMR Consulting 16 Confidential Information And yet More Test Plans n End-to-End Testing n Migration of data from client database through work order pipeline to ARROW and back using both the ARROW test environment and the client test lab n Work Order creation, completion, cancellation, and rescheduling n Work Order Parallel Field Testing n Designed to train/coach field personnel in use of system while testing, using local SMEs n Typical business scenarios run through hand- held units and dispatcher web site while simultaneously using paper system work orders and running a comparison of data accuracy

© DMR Consulting 17 Confidential Information Summary n Extensive variety of Web to Wireless and Infrastructure technical testing solutions n Took full team cooperation n Managers n System and Functional (Business) Architects n Developers n Operations Group n Client Support n And of course, the Test Group & extra folks as available n 30 member Team Work made it happen!!!

© DMR Consulting 18 Confidential Information QUESTIONS? Wed Hosted ASP Dispatcher Solution: Software Hardware ARROW Application Mobile Devices Installation Help Desk Hosting Field-Force Automation Management Services Management Information Work Order Closure Work Order Dispatch, Routing, Scheduling Field Technician

© DMR Consulting 19 Confidential Information Contact Information n Shirley DeLewis, CQA, CSTE, CPSA n Director, Software Quality Assurance & Test n DMR Consulting, Bellevue, Washington n n