1. ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006
Anthony Wiles
Manager
ETSI Protocol and Testing Competence Centre

2. ETSI Testing Projects - ICT Workshop, Brazil March 2006
The ETSI PTCC
Protocol and Testing Competence Centre
In-house team of experts providing direct support to ETSI Technical Bodies (for over 12 years)
Development of protocols and profiles and the application of modern specification techniques (35%)
‘Standards Engineering’
Validation of standards (5%)
E.g., testbed activities (IPv6, SIP/H.323)
Development of test specifications (60%)
Conformance tests
Interoperability tests
Network Integration tests
Others (PHY, performance …)
ETSI Testing Projects - ICT Workshop, Brazil March 2006

3. Typical ETSI Test Specifications
Cellular: GSM and 3G (UMTS) terminals
Now includes IMS
WiFi: HiperMAN, HiperACCESS, WiMax
VoIP: H.323, SIP, SIGTRAN
Service Creation: OSA/Parlay (API, IDL)
IPv6: Core, Security, Mobility, v4-v6
Cordless phones: DECT
Radio communications: TETRA, DMR
Access terminals: FSK, SMS
Broadband: ISDN, DSL
Smartcards: Readers, cards, security modules
Intelligent Transport Systems (ITS): DSRC
Future: More Security, more ITS, GRID ...
ETSI Testing Projects - ICT Workshop, Brazil March 2006

4. IPv6 Test Specifications STF 276
Joint ETSI – eEurope project
Supported by the ETSI membership
Mainly vendors
Require IPv6 test specifications using ETSI methods and test languages
Designed to be part of 3GPP and TISPAN NGN IPv6 testing strategy (where applicable)
European Commission:
Mandate for IPv6 device interoperability
Amplify Europe’s voice in IPv6 testing
Liaising with the IPv6 Logo Program
For more info visit:
ETSI Testing Projects - ICT Workshop, Brazil March 2006

5. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Two Phases (2005 – 2007)
Phase 1: Core IPv6
Total value approx. 0,5M€
Jan – Dec 2005
Phase2: IPSec, Mobile IPv6, IPv4-6 Transitioning
Total value approx. 1,0M€
Jan 2006 – May 2007
Provided as
Funding
Provision of experts
Hosting costs
Travel
All technical work done at ETSI by STF276
Currently 11 participants in the STF
ETSI Testing Projects - ICT Workshop, Brazil March 2006

6. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Outputs
IPv6 Testing Framework
Published (update due end 2006)
IPv6 Requirements Catalogue
On line as part of IPT Library
Test specifications
Interoperability test specifications
Conformance test specifications
Library of TTCN-3 test code freely available
On-line, extensible and reusable
All tests are validated (probably 30% of the effort)
On ETSI IPv6 Testbed
By member companies
For more info visit:
ETSI Testing Projects - ICT Workshop, Brazil March 2006

7. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Results Phase 1
Over 1100 Core IPv6 Requirements identified
Including 3GPP
300 Core IPv6 conformance tests
fully validated
120 Core IPv6 Interoperability Tests
All these test specifications including TTCN-3 code are free and open source
ETSI Testing Projects - ICT Workshop, Brazil March 2006

8. ETSI Testing Projects - ICT Workshop, Brazil March 2006
ETSI Testing Projects - ICT Workshop, Brazil March 2006

9. 3GPP UMTS (terminal) Test Specifications (STF 160)
Nobody can make money without interoperable handsets (UE)!
UE Interoperability towards access technologies is the key issue for mobility
Conformant UE = Interoperability (of UE in 3G networks)
Good experience from GSM
Solution: conformance testing
Very detailed test specifications
Based on the 3GPP core specs
Before commercial products enter the market
Voluntary declaration, but certified
One of the most important UE quality benchmarks
ETSI Testing Projects - ICT Workshop, Brazil March 2006

10. Methodology for 3G Testing
Test Report
logging and analysis
testing SS
Mobile Under Test
compilation
Executable Test Suite
(e.g., C++)
implementation
Core specs
25.331, , , ,
PICS (capability applicability) ,
Test Purposes, Descript (prose), ,
Test Cases
(TTCN) ,
ETSI Testing Projects - ICT Workshop, Brazil March 2006

11. 3G Test Industry Commitment is Vital
The TTCN test cases are implemented in the System Simulators (SS) on different platforms
Must secure the commitment of the SS vendors
SS vendors organise verification and regression testing
Each and every test case is validated
Bug-free tests have a high real value
Important to ensure that a TC will execute correctly even after updates
Early identification of clients who deploy the tests
Prioritising TCs
Scheduling the deployment
Certifications
ETSI Testing Projects - ICT Workshop, Brazil March 2006

12. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Funding
Test specs developed at ETSI by STF160
Duration of several years
Shared costs to ensure the funding
3GPP funding from Organisation Partners
Main source, approx 1M€ (61 mm) /year)
Voluntary contributions from the marketing partners
Range from 65k€ - 150k€ (5 – 12 mm) /year
Man-power (free-of-charge)
Range from mm /year
ETSI Testing Projects - ICT Workshop, Brazil March 2006

13. Typical 3GPP UMTS UE Tests
Protocol and signalling tests via radio interface
Largest part of the testing process
Full conformance tests
USIM functional tests
Identification, PIN, account charging, network priority, phonebook
UICC tests (SIM electrical test)
power on/off, 1.8v/3v detection, presence detection
bit/character transmitting duration, command processing and error handling
Acoustic tests
Loudness rating, idle noise, echo …
Radio Frequency & Radio Resource Management tests
Characteristics and performance tests of transmitter & receiver
ETSI Testing Projects - ICT Workshop, Brazil March 2006

14. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Users of 3GPP Tests
Global Certification Forum (GCF)
PCS Type Certification Review Board (PTCRB )
GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs
TD-SCDMA Industry Alliance (TDIA)
TDIA is the major clients for TD-SCDMA (LCR TDD) test specs
GCF WI-10 selected 400 TCs as high priority
Divided in 4 packages
Progressing sequentially one by one (milestone 80%)
UMTS mobile certification will be started if 80% of each package are validated
ETSI Testing Projects - ICT Workshop, Brazil March 2006

15. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Approval Process
Test cases are approved by
No quality compromise
Two week consultation period
TCs are approvedone by one
Each test must run on at least one SS platform against two independent commercial mobiles
Log files and radio conditions as support information for approval
Reason for this
TTCN TC, SS and UE are developed at the same time
None of them is perfect
ETSI Testing Projects - ICT Workshop, Brazil March 2006

16. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Results
400 TCs in 8 ATSs, 25 MB code
Platform independent
Running on > 4 platforms
We have reached the GCF 80% validated milestone
3G mobile certification starting at beginning of 2005
The light is green for mass 3G mobiles entering the markets
4 types of 3G mobiles are ready to go
Begun the real 3G test business
4 validated 3G SS platforms available in the markets
4 test houses in Europe opening 3G test services
3G TTCN experts becoming hot in the labour market
We know the price of each 3G coded TC
10 k€ /TC
3G investment for 4 years: 4 mil.€ for 400 TCs
All TTCN code can be downloaded free of charge
Code is continuously maintained and delivered
Every three weeks for 3G members
Every three months for everybody
ETSI Testing Projects - ICT Workshop, Brazil March 2006

17. VoIP-related Test Specifications STF 269, STF 270 and others
SIP (RFC 3261)
Over 400 conformance test cases
Fully validated (executables available on commercial tools)
STF 270 is doing the maintenance
H.225
About 300 conformance test cases
No ongoing maintenance
SIGTRAN
Test purposes for M3UA, M2UA, SCTP
TTCN-3 available commercially (not ETSI)
IMS SIP (together with SDP)
3GPP RAN5 (STF 160)
TTCN-3 will be used
Work started March 2006
Based on the generic SIP test suite (see above) augmented to concentrate on 3GPP SIP specific requirements
ETSI Testing Projects - ICT Workshop, Brazil March 2006

18. ETSI – WiMAX Test Specifications STF 252
WiMAX Forum Board meeting in January 2005
Agreed to harmonize the WiMAX/HiperMAN base specifications
Agreed to harmonize the WiMAX/HiperMAN test specifications
Agreed to exploit ETSI’s testing expertise
Agreed to accelerate the work using the ETSI STFs
Agreed that validation of the test specifications needed
complex interplay of all parties involved
WiMAX Forum, ETSI BRAN, equipment vendors, ETSI STF, test tool developers, test labs, etc.
ETSI WiMAX Co-operation agreement signed in 2005
ETSI Testing Projects - ICT Workshop, Brazil March 2006

19. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Results
Wave 1: Joint test specifications approved and published
Completed Dec. 2005
PICS – Protocol Implementation Conformance Statement
TSS&TP – Test Suite Structure and Test Purposes
ATS – Approx 60 test cases implemented, validated and used in certification
First WiMAX products have been certified
Wave 2: New drafts reaching approval
PICS – new updated version with corrections
TSS&TP – new draft with close to 1400 Test Purposes
ATS – new draft with close to 300 additional test cases developed and entering validation
ETSI Testing Projects - ICT Workshop, Brazil March 2006

20. ETSI Testing Projects - ICT Workshop, Brazil March 2006
IPT Library STF 295
IP Testing Library
Developed by ETSI TC MTS
All MTS IP and IP-related conformance and IOP tests
PICS, TPs, Test Suites etc.
Due to be published April 2006
SIP, IPv6, H.323, H.248
Includes IPv6 Requirements Catalogue
Linked to TPs and TTCN-3 code
Powerful search and selection possibilities
E.g., allows test suite tailored to specific search criteria
ETSI Testing Projects - ICT Workshop, Brazil March 2006

21. Screen Shot of IPT Library
ETSI Testing Projects - ICT Workshop, Brazil March 2006

22. PlugtestsTM ETSI Interoperability Service
Created in 1999, specialized in the organization of test events
Open to all organizations
Experts from ETSI and other technical communities
Skilled ETSI Staff managing test events
In 6 years, 66 Events
3500 engineers
1400 Companies
Usefulness & Service Satisfaction ranked at / 5
ETSI Testing Projects - ICT Workshop, Brazil March 2006

23. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Why?
Provides useful feedback to the standardization process
Accelerates time to market via quick product debugging
Improves both quality and features of implementations
Helps to ensure end users’ satisfaction
Reduces deployment costs
Allows networking with the market key players
COST & TIME SAVING!
ETSI Testing Projects - ICT Workshop, Brazil March 2006

24. ETSI Testing Projects - ICT Workshop, Brazil March 2006
Upcoming events in 2006
Q1 – 2006
SIM/Handset
Lawful Interception
3rd WiMAX Plugfest
Digital Identity
Q2 – 2006
IPv6 Asia
4th SQTE EU round
NGN
Mobile Application
Q3 – 2006
OSA Parlay
UWB
SIGTRAN
Q4 – 2006
Human Factors (HF)
IPv6 French Summit & Interop (FR)
Powerline
RFID
ITS ecall
ETSI Testing Projects - ICT Workshop, Brazil March 2006

25. Anthony Wiles ETSI Protocol and Testing Competence Centre
Thank You!
Phone +33 (0)
Website
Anthony Wiles
ETSI Protocol and Testing Competence Centre