1. Daily Tests - SAST March 5 2008, © Ascom1 Daily Tests of Embedded Systems

2. Daily Tests - SAST March 5 2008, © Ascom2

3. 3 Why and What have Ascom done with ’Daily Tests’? Try to reduce time from Error to Identification in the software development process. Fault Failure Detection Identification Bug Tracker Error

4. Daily Tests - SAST March 5 2008, © Ascom4 James Bach’s words on the subject For technical reasons why you rationally repeat tests... Recharge Intermittence Mutation Retry Benchmark For business reasons why you rationally repeat tests... Inexpensive Importance Enough Mandated Indifference/avoidance

5. Daily Tests - SAST March 5 2008, © Ascom5 Where are daily automatic tests in the test process of Ascom Integration test System test Daily test Module test You are here Testers Software engineers DMZ

6. Daily Tests - SAST March 5 2008, © Ascom6 Types of tests Static Code Tests Lines of code ToDo’s Lint warnings Compiler warnings Functional Tests & Profiling Software upgrade Browse MMI Change parameter through MMI Call Send alarm Message reception Roaming Bearer Handover Non-functional Tests (Performance) Calls Messaging Alarm Permutations of above while stationary or moving

7. Daily Tests - SAST March 5 2008, © Ascom7 Evolution of test interface Production test State reporting Event synthesis Profiling Parameter Setup Level Time

8. Daily Tests - SAST March 5 2008, © Ascom8 Change Benefit Benefit of automated test as a function of the changes in the test Browse MMI Grade of Service Tests Lines of code Change Parameter Trough MMI

9. Daily Tests - SAST March 5 2008, © Ascom9 The good and the bad about testing during implementation YesterdayToday

10. Daily Tests - SAST March 5 2008, © Ascom10 Example – DECT-telephone software CVS Checkout Compilation Files Web Download Software Execute Tests Create Report Analyze Execute Report Analyze

11. Daily Tests - SAST March 5 2008, © Ascom11

12. Daily Tests - SAST March 5 2008, © Ascom12 Found bugs (during 3 months) Process starvation. Uninitialized variable resulting in dangerous pointers. Index pointer outside array. Parameter not set correctly through MMI. Timing problems between tasks. Heap overflow. Stack overflow. Memory leakage. Logical error in state machine.

13. Daily Tests - SAST March 5 2008, © Ascom13 Improvements in the development process More frequent check-ins, which minimizes delays in integrations. Less broken builds, which helped teams to move on to other tasks. More feedback, which increases actions & learning. Watch out for spam feedback, which causes people to ignore messages.

14. Daily Tests - SAST March 5 2008, © Ascom14 Conclusions + Helps developers improve code quality. + Labeled software for testers on a daily basis. + Helps break anti-patterns in the development process. + Gives Benchmarks/KPIs for test- and development process. + Rapid detection and identification of major errors. + Good for detecting run-time errors.

15. Daily Tests - SAST March 5 2008, © Ascom15 Conclusions cont. - Adds labour. - Changing code – changing tests. - Needs daily supervision. - Very limited test coverage. - Does not replace other types of tests (Module- Integration- System test) + Does not replace the need for testers.

16. Daily Tests - SAST March 5 2008, © Ascom16 Ok, I’m hooked. What do I need to get started?  Code  Compiler  Scripting skills  File area  Web server  Devices to test  Sufficient hooks for event synthesis in SW  Possibility to exchange SW (in HW)  State & event reporting from SW  Profiling statistics  Basic knowledge

17. Daily Tests - SAST March 5 2008, © Ascom17 Tools and Links http://www.satisfice.com/repeatable http://www.satisfice.com/articles/test_automation_snake_oil.pdf http://en.wikipedia.org/wiki/Test_automation PC-lint

18. Daily Tests - SAST March 5 2008, © Ascom18 THANK YOU! Peter Kaltenbrunner peter.kaltenbrunner@ascom.se 031 – 55 95 97