1. SIMPL-T: SDL Intended for Management and Planning of Tests By
Qing Li, Robert Probert, William Skelton, Yiqun Xu
SAM’04 Workshop, Ottawa, Canada
June, 2004

2. Outline Background and Motivation Approach Assessment
Contributions and Future Work
SAM'04

3. Relative Error Correction Cost in a Software Life Cycle
Background and Motivation
Relative Error Correction Cost in a Software Life Cycle
Specification is at the earliest stage in a software lifecycle
Its correctness and completeness are critical to subsequent development
It must be validated against user requirements
Specification must be tested!
SAM'04

4. No Existing Formal Language is Suitable for Testing SDL Specifications
Background and Motivation
No Existing Formal Language is Suitable for Testing SDL Specifications
TTCN
MSC
UML
URN/UCM
LOTOS
SDL
SAM'04

5. Background and Motivation
SDL Task Force
The graphical representation, ensuring auto-layout is possible
Test capabilities, such as SDL based test scripts
ASN.1(1994) support, including encoding/ decoding of PDUs
Associated methodology issues, such as maximum integration of tool chain
SAM'04

6. Statement of Research Problem
Background and Motivation
Statement of Research Problem
To define and investigate the applicability of a simple, useful and efficient language for describing tests of SDL specifications
A simple test language for SDL specifications
Defined as a minimal extension of existing SDL features
To provide essential test specification functionality
SIMPL-T
-- SDL Intended for Management and Planning of Tests
SAM'04

7. Approach Basic Testing Concepts Key Requirements
Suitability of SDL for Test Specification
SIMPL-T – SDL with Extensions
SAM'04

8. SDL & TTCN Overlap Extensions to SDL Approach SDL Overlap TTCN TTCN
SDL and TTCN has overlap
TTCN is sufficient to test SDL Spec.
Only part of TTCN is needed to test SDL Spec.
After identify the SDL testing features, extensions to SDL can be defined
Extensions to SDL
TTCN
SDL
SAM'04

9. Test Architecture (ITU-T Z.500)
Basic Testing Concepts
Test Architecture (ITU-T Z.500)
Basic testing concepts are defined in ITU-T Z.500 and ISO 9646
Key requirements of testing language can be derived from these standards
SAM'04

10. Test Architecture (ISO 9646)
Basic Testing Concepts
Test Architecture (ISO 9646)
Lower Tester
Upper Tester U
IUT L
Underlying Service Provider
* Test Configuration
SAM'04

11. Test Architecture Tester – Run test suite IUT Connection – PCOs
Basic Testing Concepts
Test Architecture
Tester – Run test suite
IUT
Connection – PCOs
Communication Channels
SAM'04

12. Test Case and Test Suite
Basic Testing Concepts
Test Case and Test Suite
Test Suite
Test Case
Test Purpose
Test Case Behaviour
Sending a stimulus to the IUT
Specifying expected response
Store and Transfer data
Take alternative actions
Repeated test steps or actions
SAM'04

13. Observations Check the responses Measure the timing of response
Basic Testing Concepts
Observations
Check the responses
Measure the timing of response
Assign Verdict
SAM'04

14. Key Requirements Test Architecture – Tester and SUT
Approach
Key Requirements
Test Architecture – Tester and SUT
Connection between the Tester and the SUT
Communication between the Tester and the SUT
Organization and Management of Tests
Sending Stimuli to the IUT
Receiving Response from the IUT
Storing and Transferring Data
Flow Control
Test Step Repetition
Checking Responses and Matching Mechanism
Measuring the Timing of Responses.
Assigning and Handling of Verdict
SAM'04

15. Suitability of SDL for Test Specification
Approach
Suitability of SDL for Test Specification
Based on the key requirements, this thesis investigated:
Which SDL features can be used for testing
How they can be used
Identified what extensions are needed
SAM'04

16. Extensions Organization and Management of Tests Checking Responses
Approach
Extensions
Organization and Management of Tests
Checking Responses
-- “Input Via” and Matching mechanism
Assigning and Handling of Verdicts
New constructs for test suite, test group, test case, and test purpose are defined
Existing “input” construct is enhanced to “input via”
Matching mechanism is defined
New constructs for Verdict is defined; Verdict handling mechanism is defined
SAM'04

17. Organization and Management of Tests
Approach
Organization and Management of Tests
Testsuite_Definition ::= “TESTSUITE” TestsuiteName “;”
[ Gate_Definition ]
[ Testsuite_Component ]
“ENDTESTSUITE;”
Gate_Definition ::= “GATE” GateName “;”
[ In_Signal_List ] “;”
[ Out_Signal_List ]”;”
In_Signal_List ::= Signal_Identifier
[ “,” In_Signal_List ]
Out_Signal_List ::= Signal_Identifier
[ “,” Out_Signal_List ]
Testsuite_Component ::= ([Signal_Definition]
[Signal_List_Definition] [Timer_Definition] ……
[Test_Group_Definition]
[Test_Case_Definition] )
[Testsuite_Component ]
Test_Group_Definition ::= “TESTGROUP” TestGroupName “;”
Test_Case_Definition_List
Example syntax of “Testsuite”
SAM'04

18. New INPUT VIA Construct
Approach
New INPUT VIA Construct
STATE S1;
INPUT A VIA Gate1;
NEXTSTATE S2;
SAM'04

19. Specifying Expected Values of Parameters inside INPUT
Approach
Specifying Expected Values of Parameters inside INPUT
SAM'04

20. Matching Mechanism Unmatched Signal Handling: Disregard by default
Approach
Matching Mechanism
Unmatched Signal Handling:
Disregard by default
Explicitly use “Save” construct when necessary
SAM'04

21. Matching Mechanism Overlapped Signal Handling:
Approach
Matching Mechanism
Overlapped Signal Handling:
the same signal arriving from different gates/channels;
-- > They are not considered as overlap in SIMPL-T
(2) the parameters carried by the same signal have different values and the values have overlap
-- > They are not allowed in SIMPL-T
SAM'04

22. An Example of a SIMPL-T Test Case
Approach
An Example of a SIMPL-T Test Case
SAM'04

23. An Example of a SIMPL-T Test Case (Cont.)
Approach
An Example of a SIMPL-T Test Case (Cont.)
SAM'04

24. The Strengths and Limitations of SIMPL-T Comparing to TTCN
Assessment
The Strengths and Limitations of SIMPL-T Comparing to TTCN
+ Strength - Weakness = Same / Not needed
To assess SIMPL-T, we compare it to the international standard testing language TTCN:
SIMPL-T has many advantages over TTCN
SIMPL-T also has weakness
“/” – Many TTCN features are not needed in SIMPL-T, #11 and # 12 are examples
Many features are equally supported by TTCN and SIMPL-T:
Concurrency:
TTCN -- Through Main Test component /Parallel Test Components (MTC/PTCs)
SIMPL-T – Inherited from SDL
SAM'04

25. Ordering Problem Two or more signals can arrive in arbitrary order
Assessment
Ordering Problem
Two or more signals can arrive in arbitrary order
The order is irrelevant,
The test language does not have a mechanism to specify this situation
SIMPL-T
-- solve it using “save” construct
SAM'04

26. Contributions Submitted to the SDL Task Force
Contributions and Future Work
Contributions
Submitted to the SDL Task Force
Defined a simple, easy to learn test language
Create a potential for lower cost tools
Lead to more interest in SDL and testing
SAM'04

27. Future Work Concurrency Defaults Extensions for larger applications
Contributions and Future Work
Future Work
Concurrency
Defaults
Extensions for larger applications
Defaults” is not an essential feature in testing, so it is not discussed; but it makes a test language easier to use, so left as future work
For larger applications, extensions may be needed
SAM'04