1. 1 Open Accessible Coded Test for 3GPP Mobile Conformance Shicheng Hu Leader of MCC task 160 / 272 Leader of MCC task 160 / 272 16th Dec. 2004 Open Source Application & Standards Workshop

2. 2 About myself Born in Shanghai, ChinaBorn in Shanghai, China Home city, Nuremberg, GermanyHome city, Nuremberg, Germany Nationality: FrenchNationality: French Having worked for ETSI > 11 yearsHaving worked for ETSI > 11 years Protocol Testing Competence CentreProtocol Testing Competence Centre Leading mobile testing for 3GPPLeading mobile testing for 3GPP shicheng.hu@etsi.orgshicheng.hu@etsi.org

3. 3 ETSI Open Test Standards ETSI has a large pool of open test standardsETSI has a large pool of open test standards Coded abstract test suites in TTCN languageCoded abstract test suites in TTCN language IPR belongs to ETSIIPR belongs to ETSI Open access, free of chargeOpen access, free of charge ETSI has led the workETSI has led the work Internet testInternet test –VoIP – H.225, SIP test Fix network testFix network test –ISDN / BISDN, Supplement Service test, IN test Wireless testWireless test –DECT, TETRA, Hipper-LAN/MAN/Access test Mobile testMobile test –GSM –3GPP mobile TTCN-coded test is the latest open test standard

4. 4 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

5. 5 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

6. 6 3GPP complex & New Hope Huge investment for 3GPP deploymentHuge investment for 3GPP deployment 3GPP system provides high bit rate access, 0.38, 2, 10, … Mbits/s3GPP system provides high bit rate access, 0.38, 2, 10, … Mbits/s Capable of providing wide range of servicesCapable of providing wide range of services 3GPP is a complex3GPP is a complex –Three complicated radio technologies, FDD, TDD, GERAN –Enhanced (U)SIM applications User equipment (UE) integrates service access, the technologies and USIM in an intelligent, sexy, secure, easy-use … mobile toyUser equipment (UE) integrates service access, the technologies and USIM in an intelligent, sexy, secure, easy-use … mobile toy –Let the toys in the hand of everybody for playing/use everywhere at anytime –They should help to make money in a smart way for all 3G Operators, manufacturers, service / contents providers …

7. 7 3GPP System Bottleneck 3GPP provides a system solution for a generation3GPP provides a system solution for a generation 3GPP system components:3GPP system components: –Core networks –Infrastructures –Mobiles –USIM cards Bottleneck of the component chain is mobiles (UEs)Bottleneck of the component chain is mobiles (UEs) Mobiles are the Hi-Tech products ofMobiles are the Hi-Tech products of –Radio technology –Microelectronics –Protocol stacks –Software platform –Entertainment electronics Mobile test is always the last step before system deploymentMobile test is always the last step before system deployment

8. 8 Conformant UE = Interoperability Conformant UE = Interoperability UE Interoperability towards access technologies is the key issue for mobilityUE Interoperability towards access technologies is the key issue for mobility Conformant (U)SIM against different UEsConformant (U)SIM against different UEs Nobody can make money without an interoperable UENobody can make money without an interoperable UE –This truth is well understood by the 3GPP industry Conformant UE = Interoperability (of UE in 3G networks)Conformant UE = Interoperability (of UE in 3G networks) Solution: conformance testSolution: conformance test –Very detailed test specifications –Based on the 3GPP core specs –Low level test –Before commercial products entering market –Voluntary declaration, but certified –One of the most important UE quality benchmarks

9. 9 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

10. 10 Scope of Conformance Test Scope of Conformance Test USIM functional test (TS 31.121)USIM functional test (TS 31.121) –Identification, PIN, account charging, network priority, phonebook UICC test (SIM electrical test) (TS 102.230)UICC test (SIM electrical test) (TS 102.230) –power on/off, 1.8v/3v detection, presence detection –bit/character transmitting duration, command processing and error handling, … Acoustic test (TS 26.132)Acoustic test (TS 26.132) –Loudness rating, idle noise, echo … Radio Frequency & Radio Resource Management test (TS 34.121)Radio Frequency & Radio Resource Management test (TS 34.121) –Many characteristics tests of transmitter & receiver –Many difficult performance tests Protocol and signalling test via radio interface (TS 34.123)Protocol and signalling test via radio interface (TS 34.123) –Heart of conformance –Most difficult part –Unambiguous & less tolerant (leading to verdict Pass or Fail) –Time consumed at specification

11. 11 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

12. 12 How to Reach 3GPP mobile Protocol /signalling Conformance Need rigorous test specs in TTCN (coded tests)Need rigorous test specs in TTCN (coded tests) –Decided by 3GPP in 2000 –Black box test method –A commercially available UE (UMTS mobile) under test Use ETSI Specialist Task Force (STF) tool and facilities to accelerateUse ETSI Specialist Task Force (STF) tool and facilities to accelerate Request serviceRequest service Long term commitments from 3GPP and the test industryLong term commitments from 3GPP and the test industry –The project takes several years –Costs millions –Costs millions Share the costs and ensure the fundingShare the costs and ensure the funding –3GPP funding from Organisation Partners Main source, nearly 800k (61 mm) /year)Main source, nearly 800k (61 mm) /year) –Voluntary contributions as cash from the marketing partners Range from 65k - 150k (5 – 12 mm) /yearRange from 65k - 150k (5 – 12 mm) /year –Man-power as free-of-charge Range from 15 - 18 mm /yearRange from 15 - 18 mm /year

13. 13 3G Test industry commitments The TTCN test cases are implemented in the System Simulators in different platformsThe TTCN test cases are implemented in the System Simulators in different platforms –Secure the implementation commitments before starting the project SS vendors organise TC verifications and regression testsSS vendors organise TC verifications and regression tests –To make implemented TC running –A running TC has a high real value –A run TC will continue running after changes Early identify the clients who deploy the TTCN test casesEarly identify the clients who deploy the TTCN test cases –Prioritising TCs –Scheduling the deployment –Certifications

14. 14 Clients of 3GPP test Global Certification Forum (GCF)Global Certification Forum (GCF) PCS Type Certification Review Board (PTCRB )PCS Type Certification Review Board (PTCRB ) –GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs TD-SCDMA Industry Alliance (TDIA)TD-SCDMA Industry Alliance (TDIA) –TDIA is the major clients for TD-SCDMA (LCR TDD) test specs GCF WI-10 selected 400 TCs as high priorityGCF WI-10 selected 400 TCs as high priority –Divided in 4 packages –Progressing sequentially one by one (milestone 80%) UMTS mobile certification will be started if 80% of each package are validatedUMTS mobile certification will be started if 80% of each package are validated

15. 15 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

16. 16 Who writes TTCN TCs MCC task 160 created 2000MCC task 160 created 2000 13 experts with excellent 3G + TTCN knowledge13 experts with excellent 3G + TTCN knowledge –from 4 organisation partners Currently from 3 partnersCurrently from 3 partners MCC task 272 split out from MCC task 160 for 2G/2.5G -> 3G TCsMCC task 272 split out from MCC task 160 for 2G/2.5G -> 3G TCs 8 ATSs have been generated8 ATSs have been generated –NAS CS and PS signalling test –RRC test –3G 2G/2.5G interRAT tests –L2 tests (RLC, MAC …) –Radio access bearer test –SMS test 4 formal + 17 interim working deliveries / year4 formal + 17 interim working deliveries / year

17. 17 Methodology for 3G test Test Cases (TTCN) 34.123-3, 51.010-5 PICS ( capability applicability ) 34.123-2, 51.010-2 Test Purposes, 34.123-1 Descript (prose), 34.108, 51.010-1 Test Report logging and analysis Core specs 25.331, 25.321, 2 24.008, 44.018, 44.060 implementation testing SS Mobile Under Test compilation Executable Test Suite (e.g., C++)

18. 18 How to approve TC Test cases are approved by e-mailTest cases are approved by e-mail –No quality compromise –Two weeks consultation period –TC approval one by one A golden rule: TC must run at least on one SS platform against two independent commercial mobilesA golden rule: TC must run at least on one SS platform against two independent commercial mobiles –SS vendors raise CR for a new TC approval –Log files and radio conditions as support information for approval Reason behindReason behind –TTCN TC, SS and UE are developed at the same time –None of them is perfect Taken two years to approve the GCF prioritised TCs for 3G interoperabilityTaken two years to approve the GCF prioritised TCs for 3G interoperability –100 TC approved in 2003, –250 TC approved in 2004 –The development of these TCs took also two years beforehand All SDOs of 3GPP OP own IPR of 3GPP ATSsAll SDOs of 3GPP OP own IPR of 3GPP ATSs –Open access to TTCN codes

19. 19 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

20. 20 Result 400 TCs in 8 ATSs, 25 MB codes400 TCs in 8 ATSs, 25 MB codes –Platform independent –Running on > 4 platforms We have reached the GCF 80% milestoneWe have reached the GCF 80% milestone –3G mobile certification starting at beginning of 2005 –The light is green for mass 3G mobiles entering the markets –4 types of 3G mobiles are ready to go Open the real 3G test businessOpen the real 3G test business –4 validated 3G SS platforms available in the markets –4 test houses in Europe opening 3G test services –3G TTCN experts becoming hot in the labour market We know the price of each 3G coded TCWe know the price of each 3G coded TC –10 k /TC –3G investment for 4 years: 4 mil. for 400 TCs All TTCN codes can be downloaded free of chargeAll TTCN codes can be downloaded free of charge These codes are continuously maintained and deliveredThese codes are continuously maintained and delivered –Every three weeks for 3G members –Every three months for everybody

21. 21 Contents Why 3GPP testWhy 3GPP test What to testWhat to test How to reachHow to reach Who worksWho works ResultResult ConclusionsConclusions

22. 22 Conclusions It is a successful software projectIt is a successful software project 3GPP investment starts the value return3GPP investment starts the value return The 3G mobile bottleneck has goneThe 3G mobile bottleneck has gone key for successkey for success –Use ETSI tool having created STFs –TTCN teams work closely with test industry