1. Overview of ETSI Testing Methodology
Anthony Wiles
Manager
ETSI Protocol and Testing Competence Centre

2. ETSI European Telecommunications Standards Institute
Sophia Antipolis, France The home of ICT standardization
ETSI Testing Methodology - SINTESIO Seminar 2006

3. ETSI Testing Methodology - SINTESIO Seminar 2006
Interoperability is …
The ultimate aim of ICT standardisation
The red thread running through the entire standards development process, it’s not an isolated issue
Not something to be somehow fixed at the end
The ability of systems and products to interwork is fundamental
ETSI approach
ETSI produces with the required degree of parallelism
Base Standards and
Test Specifications
Base standards should be designed with interoperability in mind
Profiles designed to reduce potential non-interoperability
Protocol conformance and Interoperability statements
Two complementary forms of testing
Conformance testing
Interoperability testing (formal IOT)
Testing provides vital feedback into the standardization work
ETSI Testing Methodology - SINTESIO Seminar 2006

4. Poor Interoperability is Expensive
In the past, interoperability failures meant:
Bad publicity in trade magazines
Embarrassment for the manufacturer
Annoyance of the end customer
Today, interoperability failures in the field means:
Front page headlines in the Financial Times
Fall in manufacturers stock price
Loss of investor confidence
Unrecoverable damage to brand name
Irretrievable loss of customers
We can no longer afford to get it wrong!
ETSI Testing Methodology - SINTESIO Seminar 2006

5. ETSI Testing Methodology - SINTESIO Seminar 2006
Specification and Testing is Part of ETSI’s Strategy (to get it right!)
ETSI membership believes in methodical testing
A good example
GSM/UMTS is growing by 1 Million users per day!
Can you imagine what would happen if users were experiencing interoperability problems ?
More than 1 Million Euros invested per year for writing formal test specifications
We have experienced what it means to get it right!
A bad example
We have not always got it right!
For GPRS we did not take a formal approach to testing
First market products did not interoperate
We have learnt that testing is not cheap, but it pays in the long run
ETSI Testing Methodology - SINTESIO Seminar 2006

6. ETSI Resources for Interoperability
Technical Committee MTS (Methods for Testing and Specification)
Development of methodologies, techniques and languages (including TTCN3)
ETSI’s Protocol and Testing Competence Centre (PTCC)
Supports ETSI committees on the application of formal techniques in standards on a daily basis
Development of test specifications (conformance and interop)
ETSI Plugtests™ Service
Validation of standards and prototypes through interoperability events
ETSI has other recognized resources specialized in different testing areas however unlike but very complementary
Methods for Testing and Specification specialized in the development of methodologies, technique and languages
Protocol and Testing Competence Centre dedicated to support ETSI TBs on the application of fomal techniques in standards as well as the development of test specifications (conformance and interoperability)
Plugtests for the validation of standards and prototypes through interoperability
ETSI Testing Methodology - SINTESIO Seminar 2006

7. ETSI Testing Methodology - SINTESIO Seminar 2006
How Does the PTCC Help?
Assist ETSI Technical Bodies on the use of state of the art techniques for
Specification, validation and testing
Good working practices (Standards Engineering)
Pragmatic and flexible approach
Based on experience
Help to develop usable methodologies
For ETSI’s current and emerging needs
Knowledge transfer
Quality through Continuity
ETSI Testing Methodology - SINTESIO Seminar 2006

8. ETSI Testing Methodology - SINTESIO Seminar 2006
Who PTCC Supports
Technical Bodies (TB)
Technical Committees
ETSI Projects
Partnership Projects etc.
Chairmen, Rapporteurs, Individuals
Working Groups (WG)
STFs (Specialist Task Forces)
Experts are seconded from the ETSI membership
25-30 experts in 2006
Produce test specifications (including validation)
Short projects
e.g., 2 man-months maintenance of VoIP tests
Long projects
e.g., UMTS testing 58 man-months per year over 3-5 years
15 – 20 PTCC STFs per year
Responsible for approx. 2,7M€ of expert resource per year
ETSI Testing Methodology - SINTESIO Seminar 2006

9. Typical Standards Development Process in ETSI
Industry
Certification
Iterative feedback
Products mature from prototypes to commercial products
(Unit) Conformance Testing
Interoperability Testing
time
Conformance Test Specifications
Interoperability Test Specifications
Iterative feedback
ETSI: Development of base standards
ETSI Testing Methodology - SINTESIO Seminar 2006

10. PTCC Support for Specification Techniques
PTCC provides support to the ETSI TBs on the use of modern specification techniques in ETSI deliverables:
UML (Unified Modeling Language)
ASN.1 (Abstract Syntax Notation)
XML (Extensible Markup Language)
MSC (Message Sequence Charts)
SDL (Specification and Description Language)
Etc.
TTCN (Test and Test Control Notation)
ETSI Testing Methodology - SINTESIO Seminar 2006

11. Different Kinds of ETSI Test Specifications
Robustness
Conformance
Interoperability
Performance
Network Integration
RF/EMC
ETSI Testing Methodology - SINTESIO Seminar 2006

12. Typical ETSI PTCC Areas of Testing
Cellular: GSM and 3G (UMTS) terminals
Now includes early IMS
WiFi: HiperMAN, HiperACCESS, WiMax
VoIP: H.323, SIP, SIGTRAN
Service Creation: OSA/Parlay (API, IDL)
IPv6: Core, Security, Mobility, v4-v6
Cordless phones: DECT
Radio communications: TETRA, DMR
Access terminals: FSK, SMS
Broadband: ISDN, DSL
Smartcards: Readers, cards, security modules
Intelligent Transport Systems (ITS): DSRC
Early NGN
Future: More Security, more NGN, GRID ...
ETSI Testing Methodology - SINTESIO Seminar 2006

13. ETSI Testing Methodology - SINTESIO Seminar 2006
Progressive Testing
All engineering disciplines accept that testing should be applied to progressively complex units
From individual components to entire systems
This demands a number of different testing solutions
There is no silver bullet
In the two extremes
Just testing the components is not enough
Just testing the system is not enough
Limitations are economic, not technical
What can I afford to test?
What can I afford not to test?
What should be covered by standardised test specifications
What is the right kind of testing for the job
What kind of ‘thing’ is being tested? Device? System? Service?
What is the most economic method(s)?
What resources are (or must be made) available?
Tools, test scripts, testbeds etc.
ETSI Testing Methodology - SINTESIO Seminar 2006

14. ETSI Testing Methodology - SINTESIO Seminar 2006
Conformance Testing
Is Black-Box testing
Stimulation and Response
Tester
Device
Point of Control and Observation (PCO)
Reqs.
ETSI Testing Methodology - SINTESIO Seminar 2006

15. Conformance Testing Tests is Usually Layer-byLayer
IUT = Implementation Under Test L3
Upper Tester
API
MMI L3 L2
PHY
SUT = System Under Test 1 2 3 4 5 6 7 8 9 * # l a t i g d
Test
System
SUT
Lower Tester i i t a l
ETSI Testing Methodology - SINTESIO Seminar 2006

16. Characteristics of Conformance Testing (1)
Is unit testing
Tests a single ‘part’ of a device (e.g., a protocol layer)
Often incrementally – layer-by-layer
Tests against well-specified requirements
For conformance to the requirements in a base specification or profile (standard)
Usually limited to one requirement per test
Tests at a 'low' level
At the protocol (message/behaviour) level (bits)
Or service primitive, API, Interface
Tests over standardised interfaces
May not be available to ‘normal’ user
Requires a test system (and executable test cases)
Can be expensive (e.g., radio-based test system)
Tests under ideal conditions
ETSI Testing Methodology - SINTESIO Seminar 2006

17. Characteristics of Conformance Testing (2)
High control and observability
Means we can explicitly test error behaviour
Can provoke and test non-normal (but legitimate) scenarios
Can be extended to include robustness tests
It is usually automated and tests are repeatable
Conformance Testing is DEEP and NARROW
Thorough and accurate but limited in scope
Gives a high-level of confidence that key components of a device or system are working as they were specified and designed to do
ETSI Testing Methodology - SINTESIO Seminar 2006

18. Limitations of Conformance Testing
Does not prove end-to-end functionality (interoperability) between communicating systems
Conformance tested implementations may still not interoperate
This is often a specification problem rather than a testing problem! Need minimum requirements or profiles
Does not test a complete system
Tests individual system components, not the whole
A system is often greater than the sum of its parts!
Does not test functionality
Does not test the user’s ‘perception’ of the system
Standardised conformance tests do not include proprietary ‘aspects’
Though this may well be done by a manufacturer with own conformance tests for proprietary requirements
ETSI Testing Methodology - SINTESIO Seminar 2006

19. Conformance Testing Methodology ISO/IEC 9646 (parts 1-7)
Test Report
logging and analysis
Base Standard
(or Profile)
implementation
Product
Implementation Under Test
Test System
testing
compilation
Executable Test Suite
(e.g., C++)
Industry
validation
Req. checklist
ICS
Test Purposes
TSS & TP
Test Suite (Test Cases)
ATS
ETSI Testing Methodology - SINTESIO Seminar 2006

20. Implementation Conformance Statement (ICS)Q#
ICS Question
Ref
Status
Support Q1
Support of Feature F1
[x] Clause a
OPTIONAL Q2
Support of Feature F2
[x] Clause b
MANDATORY : Qn
Support of Message M1
[y] Clause c
CONDITIONAL Qk
Support of message Parameter P1
[y] Clause d
The ICS is a checklist of the capabilities supported by the Implementation Under Test (IUT)
Provides an overview of the features and options that are implemented in any given product
The ICS can be used to select and parameterise test cases and can be used as an indicator of basic interoperability between different products.
Conditional support e.g., Qn must be supported if Q1 supported then otherwise not.
ETSI Testing Methodology - SINTESIO Seminar 2006

21. ETSI Testing Methodology - SINTESIO Seminar 2006
Profile ICS Q#
ICS Question
Ref
Status
Profile Status
Support Q1
Support of feature F1
[x] Clause a
OPTIONAL
MANDATORY Q2
Support of feature F2
[x] Clause b : Qn
Support of Message M1
[y] Clause c
CONDITIONAL Qk
Support of parameter P1
[y] Clause d
N/A
A profile ICS reflects how a (protocol) profile restricts the scope of a base standard by making certain options mandatory or excluding certain options.
ETSI Testing Methodology - SINTESIO Seminar 2006

22. Selecting Tests with the ICSQ#
ICS Question
Ref
Status
Support Q1
Support of Feature F1
[x] Clause a
OPTIONAL P Q2
Support of Feature F2
[x] Clause b
MANDATORY : Qn
Support of Message M1
[y] Clause c
CONDITIONAL Qk
Support of message Parameter P1
[y] Clause d O
A filled-in ICS can be used to select tests. For example, a test for an OPTIONAL feature which is not supported in a given product will be deselected from the test suite.
ETSI Testing Methodology - SINTESIO Seminar 2006

23. Parameterizing Tests with eXtra Information for Testing (IXIT)Q#
IXIT Question
Ref
Allowed Values
Value Q1
Network address
[x] Clause a
Valid IP address Q2
Value of Timer T1
[x] Clause b s 5s :
A filled-in IXIT can be used to parameterize tests. The Value column is used to specify explicit IUT or test system dependent values.
ETSI Testing Methodology - SINTESIO Seminar 2006

24. The Requirements Catalogue
Each Requirement is categorized as follows (example from IPv6):
Requirement type:
Mandatory (MUST, MUST NOT)
Recommended (SHOULD, SHOULD NOT)
Optional (MAY, MAY NOT)
Requirement target:
E.g., Host, Router, Node (Host or Router)
Requirement text
Functional groupings
E.g., Process Fragmented packet, Generate ICMPv6 Error Typetc.
A scalable database containing all requirement elements
Web interface
Browsing by function
User-defined search
Links to RFC and related test specification
ETSI Testing Methodology - SINTESIO Seminar 2006

25. ETSI IP Testing Library
ETSI Testing Methodology - SINTESIO Seminar 2006

26. Conformance Test Purposes
Test Purposes (TP) are natural language, precise descriptions of the purpose of the test in relation to a particular (base standard) requirement
Define the functionality being tested—the WHAT
Do not define HOW to test the function
Grouped into a logical structure (Test Suite Structure)
TSS & TP
One Requirement may spawn several TPs
A conformance TP is written on the protocol level
Specified in
Natural language, or
ETSI’s Test Purpose Language (TPLan)
They are not test code
ETSI Testing Methodology - SINTESIO Seminar 2006

27. TPLan Example for Conformance
TP id : TP_COR_0047_01
Summary : ‘hop limit of one'
RQ Ref : RQ_COR_0047
Config : CF_02_C
TC Ref : TC_COR_0047_01
ensure that {
--Stimulus
when { IUT receives ‘Ipv6 packet' from ‘Host'
containing ‘IPv6 Header'
indicating ‘Hop limit' set to ‘1‘ }
--Expected response
then { IUT sends ‘ICMPv6 Time Exceeded' to ‘Host‘
containing ‘ICMP code' set to ‘ZERO‘ } }
ETSI Testing Methodology - SINTESIO Seminar 2006

28. Conformance Test Cases
Detailed TTCN-3 test script that implements test purpose
can be compiled and executed
Is the HOW to test not WHAT to test
Composition
a preamble
test body (i.e., implementation of the Test Purpose)
A postamble
Assigns test verdicts
Handles unexpected behaviour as well as the behaviour in the test purpose
Can be distributed over parallel test components
Can be entirely automated
Configurable at run-time, e.g., SUT address
ETSI Testing Methodology - SINTESIO Seminar 2006

29. ETSI Testing Methodology - SINTESIO Seminar 2006
Abstract Test Suite
The Test Suite (ATS) is the collection of all Test Cases
Most ETSI Test Suites are written in TTCN
Predominantly in TTCN-2
Progression to TTCN-3 for new work
Not RF testing (generally not physical layer)
TTCN-3
Testing and Test Control Notation
Allows tests to be specified in detail (test code) that is independent of the (eventual) real test system on which it may be run
i.e., TTCN-3 will run on any test system that supports a standardised TTCN-3 execution environment
ETSI Testing Methodology - SINTESIO Seminar 2006

30. ETSI Testing Methodology - SINTESIO Seminar 2006
What is TTCN-3?
Internationally standardized testing language
Look and feel of a regular programming language
Allows unambiguous implementation of tests
Independent of execution environment due to separate test system interface standards
Wide range of applications from mobile (radio) communications to Internet to software
Good tool support
Good book:
ETSI Testing Methodology - SINTESIO Seminar 2006

31. Example TTCN-3 Test Case
testcase TC_COR_0047_01() runs on Ipv6Node system EtherNetAdapter {
f_cf02Up(); // Configure test system for HS->RT
// No preamble required in this case
f_TP_HopsSetToOne(); // Perform test
// No postamble required in this case
f_cf02Down(); // Return test system to initial state }
function f_TP_HopsSetToOne() runs on Ipv6Node {
var Ipv6Packet v_ipPkt;
var FncRetCode v_ret := f_echoTimeExceeded( 1, v_ipPkt );
if ( v_ret == e_success and v_ipPkt.icmpCode == 0 )
{ setverdict(pass);}
else { setverdict(fail); }
function f_echoTimeExceeded(in UInt8 p_hops, out Ipv6Packet p_ípPkt ) runs on Ipv6Node return FncRetCode {
var Ipv6Packet v_ipPacket; var FncRetCode v_ret;
ipPort.send( m_echoReqWithHops(p_hops) ); alt {
[] ipPort.receive( mw_anyTimeExceeded ) -> value p_ipPkt
{ return e_success } [] ipPort.receive { return e_error } }
ETSI Testing Methodology - SINTESIO Seminar 2006

32. Executable Test Suites
Executable test suites are compiled from the abstract
Either direct to binary, or
More commonly to some intermediate programming language (C++, Java, etc.)
ETSI does not deliver executables, but
We try to ensure all code can be compiled
And validated by executing the tests against a real implementation on at least one test system
5 test systems in the case of UMTS, for example
ETSI Testing Methodology - SINTESIO Seminar 2006

33. (N-protocol specific) Underlying Protocol Stack
TTCN-3 Test System
The Industry
PICS etc
(reqs. catalogue)
Parameterisation
Selection
Control / Logging
User
TCI
TTCN-3 Control Interface
Test Suite in TTCN-3
(source)
TTCN-3 Test Suite
(object)
Compilation
ENCODER
DECODER
(N-protocol specific)
TRI
TTCN-3 Runtime Interface
Adaptation Layers
Underlying Protocol Stack
(N-1)
Connection to the SUT
ETSI Testing Methodology - SINTESIO Seminar 2006

34. Interoperability Testing
End-to-End Interoperability of devices
What is being tested?
Assignment of verdicts?
Need to distinguish between
Formal interoperability testing
And informal interoperability events used to validate/develop technologies
Device1
Devicen
Device2
ETSI Testing Methodology - SINTESIO Seminar 2006

35. Interoperability Testing Tests Functionality Between DevicesQE
EUT 1 2 3 4 5 6 7 8 9 * # 1 2 3 4 5 6 7 8 9 * #
Means of Communication (MoC)
Interoperability Testing (
of terminals)
SUT
API
MMI L3 L2
PHY
QE = Qualified Equipment
EUT = Equipment Under Test
ETSI Testing Methodology - SINTESIO Seminar 2006

36. Characteristics of Interoperability Testing
Is system testing
Tests a complete device or a collection of devices
Shows that (two) devices interoperate
Albeit within a limited scenario
Tests at a ‘high’ level (as perceived by users)
Tests the ‘whole’, not the parts
e.g, protocol stacks + applications
Tests functionality
Shows function is accomplished (but not how)
Does not necessarily require a test system
Uses existing interfaces (standard/proprietary)
Interoperability Testing is BROAD and SHALLOW
Less thorough but wide in scope
Gives a high-level of confidence that devices (or components in a system) will interoperate with other devices (components)
ETSI Testing Methodology - SINTESIO Seminar 2006

37. Limitations of Interoperability Testing
Does not prove interoperability with other implementations with which no testing has been done
A may interoperate with B and B may interoperate with C. But it doesn’t necessarily follow that A will interoperate with C.
Combinatorial explosion
Does not prove that a device is conformant
Interoperable devices may still interoperate even though they are non-conformant
Cannot explicitly test error behaviour or unusual scenarios
Or other conditions that may need to be forced (lack of controllability)
Has limited coverage (does not fully exercise the device)
Not usually automated and may not be repeatable
ETSI Testing Methodology - SINTESIO Seminar 2006

38. Interoperability Testing Methodology ETSI TS 102 237
Test Report
logging and analysis
Base Standard (or Profile)
implementation
Product 1
Qualified Equipment
Product 2
Equipment Under Test
testing
Interop Test Cases
Interop Test Purposes
IFS (functional checklist)
ETSI Testing Methodology - SINTESIO Seminar 2006

39. Interoperability Test Purposes
Define the function being tested—the WHAT
Do not define HOW to test the function
Grouped into a logical structure (Test Suite Structure)
One Requirement may spawn several TPs
An interoperability TP is on the functional level
Specified in ETSI’s Test Purpose Language (TPLan)
ETSI Testing Methodology - SINTESIO Seminar 2006

40. TPLan Example for Interoperability
TP id : TP_COR_1719_02
Summary : 'EUT sends packet to All-RT LL M/cast address'
RQ ref : RQ_COR_1719
Config : CF_021_I
TD ref : TD_COR_1719_02
with { QE1 'configured as a router'
and QE2 'configured as a router'}
ensure that {
when { EUT is requested to
'send packet to All-Routers Link-Local M/cast addr' } then { QE1 indicates 'receipt of the packet'
and QE2 indicates 'receipt of the packet' } }
ETSI Testing Methodology - SINTESIO Seminar 2006

41. Interoperability Test Descriptions
Specify detailed steps to be followed to achieve stated test purpose
Steps are specified clearly and unambiguously without unreasonable restrictions on actual method:
Example:
Answer incoming call
NOT
Pick up telephone handset
Written in a structured and tabulated natural language so tests can be performed manually
ETSI Testing Methodology - SINTESIO Seminar 2006

42. Example Interoperability Test Description
ETSI Testing Methodology - SINTESIO Seminar 2006

43. Automated IOP Testing Using TTCN-3
Controller
(TTCN-3)
Test
Driver
(TTCN-3)
Test
Driver
(TTCN-3)
AT Commands
AT Commands
ETSI Testing Methodology - SINTESIO Seminar 2006

44. ETSI Testing Methodology - SINTESIO Seminar 2006
IOT with Conformance Verifiction (aka NIT - Network Integration Testing)
ETSI Testing Methodology - SINTESIO Seminar 2006

45. ETSI Testing Methodology - SINTESIO Seminar 2006
Example NIT for IMS
ETSI Testing Methodology - SINTESIO Seminar 2006

46. Conformance and Interoperability Testing are Complementary
ETSI experience
As you move up a system stack the emphasis should change from conformance to IOT
Lower layer protocols, infrastructure
Emphasis on conformance
Middleware, enablers
Combination of Conformance + IOT
Services, applications, systems
Emphasis on IOT
Conformance testing as a pre-requisite to IOT
Interop Testing
Conformance Testing
Interoperability
ETSI Testing Methodology - SINTESIO Seminar 2006

47. ETSI Testing Methodology - SINTESIO Seminar 2006

48. TTCN-3 Standards http://www.ttcn-3.org
ES (Z.140)
TTCN-3 Core Language
ES (Z.141)
TTCN-3 Tabular Presentation Format (TFT)
TR (will eventually be ES ) (Z.142)
TTCN-3 Graphical Presentation Format (GFT)
ES (Z.143)
TTCN-3 Operational Semantics ES
TTCN-3 Runtime Interface (TRI) ES
TTCN-3 Control Interfaces (TCI)
ES and upwards
Using ASN.1, XML, IDL, C/C++, with TTCN-3
ETSI Testing Methodology - SINTESIO Seminar 2006

49. TTCN-3 can be used for Conformance and Interoperability Testing
ETSI Testing Methodology - SINTESIO Seminar 2006

50. ETSI Testing Methodology - SINTESIO Seminar 2006
Benefits of TTCN-3
Specifically designed for testing
Concentrates on the test not the test system
Commonly understood syntax and operational semantics
Constantly maintained and developed
Off-the-shelf tools and TTCN-based test systems are readily available
Single language for many (all?) testing activities
Education and training costs can be rationalized
Maintenance of test suites (and products) is easier
Allows the application of a common methodology and style, both on a corporate level and within standardization
ETSI Testing Methodology - SINTESIO Seminar 2006

51. Main Capabilities of TTCN-3
Dynamic concurrent testing configurations
Various communication mechanisms (synch and asynch)
Data and signature templates with powerful matching mechanisms (including regular expressions)
Specification of encoding information
Display and user-defined attributes
Test suite parameterization
Control of Test Case execution and selection mechanisms
Control of complex test configurations
Assignment and handling of test verdicts
Harmonized with ASN.1 (XML and IDL coming)
Different presentation formats
Well-defined syntax, static - and operational semantics
ETSI Testing Methodology - SINTESIO Seminar 2006

52. The Core Language and Other Presentation Formats
Text format
Core format is text based (most popular)
TTCN-3 can be edited or viewed in other formats
Tabular format (for TTCN-2 people)
Graphical format (good for visual overview)
Other standardized formats in the future?
Proprietary formats possible
PresentationFormat3
PresentationFormatn
Graphical Format
Tabular Format
TTCN-3 Core Language
ETSI Testing Methodology - SINTESIO Seminar 2006

53. Example Core (Text) Format
function PO49901(integer FL) runs on MyMTC {
L0.send(A_RL3(FL,CREF1,16))
TAC.start
alt {
[] L0.receive(A_RC1((FL+1) mod 2)) {
TAC.cancel
verdict.set(pass) }
[] TAC.timeout {
verdict.set(inconc)
[] any.receive {
verdict.set(fail)
ETSI Testing Methodology - SINTESIO Seminar 2006

54. Example Graphical Format
ETSI Testing Methodology - SINTESIO Seminar 2006

55. Example Tabular Format
ETSI Testing Methodology - SINTESIO Seminar 2006

56. Use of TTCN-3 With Other Languages
TTCN can be integrated with other languages
Fully harmonized with ASN.1 (1997)
Harmonized with other languages
IDL, XML, C/C++
ASN.1 Types & Values
IDL Types & Values
Other types & Valuesn
XML Types & Values
TTCN-3 Core Language
ETSI Testing Methodology - SINTESIO Seminar 2006

57. Major Elements of a TTCN-3 Test Suite
Definition of data types for
Protocol Messages and
Information elements (fields, parameters)
Internal data (e.g., for computation)
Test coordination messages etc.
Predefined simple types
Integer, boolean, float
bitstring, hexstring, octetstring
charstring, universal charstring
... and complex types
record, record of, set, set of
union, enumerated
... and special types such as
verdict, address
Actual test data (values) used during testing
Message templates
(Remote) signature templates
Matching mechanisms
values, lists, wildcards,
regular expressions etc.
Encoding information
Test components
Test ports
Connections
between test components
to System Under Test
Configurations
static
dynamic
test cases
test steps
default behaviour
management of test components
sending/receiving messages
verdict assignment
computation (e.g., checksums)
timers and timeouts
test execution and control
etc.
Test Suite
Test Data Types
Actual Test Data
Test System Architecture
Test Behaviour
ETSI Testing Methodology - SINTESIO Seminar 2006

58. Simple TTCN-3 Architecture
TTCN-3 based Test
System
Implementation Under Test
Black-Box
Point of Control and Observation (PCO)
Test System
IUT
stimulus
response
Test port
Test port
ETSI Testing Methodology - SINTESIO Seminar 2006

59. TTCN-3 Architecture – Test Components
Test System
Internal Communication
Parallel Test
Component
(PTC)
Parallel Test
Component
(PTC)
IUT
Main Test
Component
(MTC)
Communication to IUT
ETSI Testing Methodology - SINTESIO Seminar 2006

60. ETSI Testing Methodology - SINTESIO Seminar 2006
TTCN-3 Module
module Example {
// Definitions part
control
// Control part }
} with {encode "BER"}
Module (…)
Module Definitions
Module Control
Attributes
ETSI Testing Methodology - SINTESIO Seminar 2006

61. ETSI Testing Methodology - SINTESIO Seminar 2006
Definitions Part
module Example {
// Port and Test Component definitions
// Defintions of message types
// Templates for instances of actual messages
// Test Case definitions }
module Example {
group TestArchitecture
// Port and Test Component definitions }
group MessageTypes
// Defintions of message types
group ActualMessages
// Templates for instances of actual messages
group TestCases
// Test Case definitions
ETSI Testing Methodology - SINTESIO Seminar 2006

62. Specifying Test Messages
type record MsgType {
integer msgId
charstring msgName,
charstring msgInfo }
template MsgType INVITE {
msgId 0
msgName "INVITE",
msgInfo "For a good Brazilian Dinner" }
template MsgType ACCEPT {
msgId 1
msgName "ACCEPT",
msgInfo * }
ETSI Testing Methodology - SINTESIO Seminar 2006

63. ETSI Testing Methodology - SINTESIO Seminar 2006
TTCN-3 Behaviour
testcase TC1() runs on PTC1 {
P.send(INVITE)
T1.start(30E3)
alt
[] PCO.receive(ACCEPT)
{verdict.set(pass)}
[] PCO.receive(DECLINE)
{verdict.set(inconc)} [] PCO.receive
{verdict.set(fail)}
[] T1.timeout }
INVITE
ACCEPT or DECLINE P
ETSI Testing Methodology - SINTESIO Seminar 2006

64. ETSI Testing Methodology - SINTESIO Seminar 2006
The Control Part
module Example {
// Definitions part
params { boolean Q1; : }
control
if Q1 then execute (TC1())
} with {encode "BER"}
Module (…)
Module Definitions
Module Control
Attributes
ETSI Testing Methodology - SINTESIO Seminar 2006

65. Thank You! Phone +33 (0)4 92 94 43 26 e-mail ptcchelp@etsi.org
Website