1. Agere Systems - Optimay GmbHMay, 2003 - 1 - UMTS terminal testing: A practical perspective Olaf Bergengruen May, 2003

2. Agere Systems - Optimay GmbHMay, 2003 - 2 - „We will not survive without a proper regression test system“, an Optimay engineer

3. Agere Systems - Optimay GmbHMay, 2003 - 3 - Overview Virtual testing Logical view (what we want to test) Why virtual testing Requirements to the test system 3GPP approach Test model Test cases Optimay‘s implementation GSM/GPRS virtual test system GSM/GPRS/UMTS virtual test system

4. Agere Systems - Optimay GmbHMay, 2003 - 4 - Test environment / logical view (conformance testing, FTA)

5. Agere Systems - Optimay GmbHMay, 2003 - 5 - Test environment / logical view (2)

6. Agere Systems - Optimay GmbHMay, 2003 - 6 - Why virtual test environment We need a SW only environment (no specific HW) HW is expensive or does not exist HW debugging is difficult and time consuming SW developers don‘t want to be bothered with HW issues or don‘t have the specific skills We need to be much faster than real time Developers want to use standard tools (compilers, editors, debuggers)

7. Agere Systems - Optimay GmbHMay, 2003 - 7 - Requirements to the virtual test system Test system shall be much faster than real time Support the design of test scenarios at a high level of abstraction All test suites need to run ‚over night‘: GSM/GPRS tests (GSM 11.10), UMTS test (3GPP 34.123), STK tests (GSM 11.10-4), MMI test cases and possibly other suites (TCP/IP, WAP,...) Test system shall be available to anybody, at anytime, on any PC

8. Agere Systems - Optimay GmbHMay, 2003 - 8 - GSM/GPRS test architecture

9. Agere Systems - Optimay GmbHMay, 2003 - 9 - GSM/GPRS test architecture

10. Agere Systems - Optimay GmbHMay, 2003 - 10 - Great approach: L1 simulated interface Basic communication with the Tester via three functions: RxRadioBlock, TxRadioBlock, TimerTick These functions can be extended as needed to simulate DSP code or other HW related functionality MS is the master of system ‚ticks‘, about 100 times faster than real time Tester and MS are tightly coupled which allows debugging using standard tools

11. Agere Systems - Optimay GmbHMay, 2003 - 11 - Debugging GMM code... static void GMMDoRoutingAreaUpdate ( GM_IE_UPDATE_TYPE RauType, BOOLEAN FirstAttempt ) { MMRAUpdPending = FALSE; GMRauType = RauType; GMStopT3302( ); GMStopT3311( ); if ( ! FirstAttempt ) { GMT3330Expiries = 0; if ( GMActState == GMActDetachInit ) { GMStopT3321( ); GMDetachPending = TRUE; } if ( GMTriggerAction == GM_GS_ATTACH_COMP ) { /* Reinitialise the trigger action variable. */ GMTriggerAction = GM_NO_TRIGGER_ACTION; /* Confirm the end of the ATTACH procedure to GS */ SEND( GSGMEstCnf ) ( ); }

12. Agere Systems - Optimay GmbHMay, 2003 - 12 - Highlights of our GSM/GPRS virtual test system Extremely efficient performance (around 1300 tests in 20 minutes) Includes testing of DSP code About 30 man-years development effort (including test case development)

13. Agere Systems - Optimay GmbHMay, 2003 - 13 - Drawbacks of our GSM/GPRS simulation Tester uses proprietary (in house) test script languages and encoders which are difficult to maintain It is difficult to write test cases Tester is not nicely layered as shown in the previous slide, so it is difficult to extend and maintain Summarizing: the system can not be extended to implement an UMTS System Simulator

14. Agere Systems - Optimay GmbHMay, 2003 - 14 - 3GPP test model (TS 34.123-3)

15. Agere Systems - Optimay GmbHMay, 2003 - 15 - Why we need a Test Model Main reason for us: If we comply with the 3GPP Test Model we can use the TTCN test cases developped here at ETSI / STF 160 In general: Clear and stable interfaces to the SS enables manufacturers to develop the test equipment It reduces ambiguities within test case scenarios It enables outsourcing of testing activities and SW re- use It provides a common consens or understanding of the complete system to manufacturers and operators

16. Agere Systems - Optimay GmbHMay, 2003 - 16 - A test case (1/2)... Configure SS Configure PHY / L1 Configure MAC Configure RLC Schedule and send System Information Blocks Test case preamble Bring UE into initial state (e.g. CS and PS registration) —Perform Location Update procedure —Perform GPRS Attach procedure

17. Agere Systems - Optimay GmbHMay, 2003 - 17 - Test case body Stimulate the UE, e.g. —Send X message to the UE, or —Change cell power levels, or —Trigger the UE via AT command to initiate a call Verify responses —Match messages from the UE to expected values —Assign a verdict (PASS, FAIL, INCONC) Test case postamble Complete signalling to bring UE into stable state A test case (2/2)...

18. Agere Systems - Optimay GmbHMay, 2003 - 18 - So, how to design a Virtual Tester which matches the 3GPP model and our requirements ?

19. Agere Systems - Optimay GmbHMay, 2003 - 19 - Problems designing an overall GSM/GPRS/UMTS Test Env. The clean 3GPP model is not enough for a complete MS test environment We need to re-use tools, tracers, and most of all test cases developed for GSM in the new environment We can not afford to develop the UMTS test suite in house, we need to re-use the 3GPP tests The L1 and DSP simulation strategies for GSM do not match those of UMTS

20. Agere Systems - Optimay GmbHMay, 2003 - 20 - Current approach (a compromise) Multi-threaded system: The proper MS, the GSM SS, the UMTS SS and the SIM simulator All communicating via shared buffers The tick-master is the MS, when all jobs for current frame are completed, it issues a ‚tick‘ and all System Simulators prepare data (radio blocks) for next frame. Each SS will send a ‚tick-ack‘ to the MS when it is ready for next frame When all SSs are done: the MS L1 engine will fetch or store a radio block

21. Agere Systems - Optimay GmbHMay, 2003 - 21 - GSM/GPRS/UMTS Test Environment

22. Agere Systems - Optimay GmbHMay, 2003 - 22 - Summary A Virtual Test Environment is an essential tool for development for quality assurance 3GPP test model A rough overview of our implementation

23. Agere Systems - Optimay GmbHMay, 2003 - 23 - Thank you.