1. Common Test Project CMU Presentation Jason Weighley Manager Test Engineering

2. © Bombardier Inc. or its subsidiaries. All rights reserved. Background  Bombardier Transportation – RCS Division – Provides CBTC signaling for Metro and People Mover applications –Signaling system consists of several sub systems (Comprised of Hardware / Software) which require changes from project to project –All sub systems are tested in a factory environment (Separate then integrated) Tested procedures and cases based on functional requirements and prepared by a test engineering group. –Safety sub systems are also tested / inspected by independent group (VNV). This group also prepares its own test cases for functional / safety requirements.

3. © Bombardier Inc. or its subsidiaries. All rights reserved. Why do we need COMMON tests?  Sharing a common test suite (tools) is the first requirement towards improving schedule, budget, and quality – and is already being addressed via the effort of the test group.  Common tests will give us a clearly defined quality gate that must be cleared before code can be released to FAT, V&V and/or the field. Stable test environment must be achieved before developers begin to use.  As each new “issue/bug” is discovered, additional test can be added to the test cases thereby ensuring that each new issue/bug will continue to be tested – despite the fact that people may forget about them over time.  If we have (increased) common test coverage, we will be able to provide faster turnaround time with less effort.  Quality and Repeatability will increase. Cost and schedule turnaround will decrease.

4. © Bombardier Inc. or its subsidiaries. All rights reserved. Why do we need SHARED test cases?  Maintaining 2-3 sets of test cases is costing us 2-3 times as much budget – this is a cost issue.  Sharing test cases will ensure that the expertise and knowledge of each engineer will go into the process to be shared by all others.  Quality and Repeatability will increase. Cost and schedule turnaround will decrease.

5. © Bombardier Inc. or its subsidiaries. All rights reserved. All of the tests are really the same! Most of our effort goes here We do NOT do enough of this!

6. © Bombardier Inc. or its subsidiaries. All rights reserved. What exactly does “shared tests” mean?

7. © Bombardier Inc. or its subsidiaries. All rights reserved. Current Approach

8. © Bombardier Inc. or its subsidiaries. All rights reserved. Proposed Approach 1.Management tool to track progress 2.Built-in Quality Gates (V&V and Factory test cannot start unless the automatic scripts pass) 3.No duplication of effort results in 2/3 reduction of total test time 4.Better consistency 5.Automated test yields faster turn-around time after changes 6.Higher confidence that all requirements were implemented and tested Scripts must be continuously updated as the software is debugged.

9. © Bombardier Inc. or its subsidiaries. All rights reserved. 1.Improving the efficiency of the project testing is the key towards reducing the product cost! 2.Finding defects earlier (moving to the left-side of the “V” diagram) 3.Reducing the test overlap between the FAT, Safety, Field, and Development groups 4.Reducing the turn-around time for regression testing (Development, FAT, Safety) Summary Statements

10. © Bombardier Inc. or its subsidiaries. All rights reserved. Plan to implement and Work Required  Record VNV and Test Engineering test cases used for vehicle and wayside development on Metro Madrid into DOORS (cases to be provided by cognizant engineers)  Update links to functional requirements for ATP related tests to VNV and Test Engineering test cases  Review Test Case linking with Cognizant engineers  Based on linking Identify common test cases from Metro Madrid (Current base project for several new China projects)  Review Common Test Cases with Cognizant Engineers  Repeat above for Shenzhen based on Project requirements and add/modify common tests  Update / Generate automated testing for China Projects based on Common Test  Test automated test scripts to verify script is working correctly.