International Telecommunication Union Geneva, 9(pm)-10 February 2009 Providing testability for ITU Recommendations Ostap Monkewich, OMCI ITU-T Workshop on “New challenges for Telecommunication Security Standardizations" Geneva, 9(pm)-10 February 2009

International Telecommunication Union Geneva, 9(pm)-10 February Theme “What is missing from your Recommendations that is needed for testing?”

International Telecommunication Union Geneva, 9(pm)-10 February Why do we need to test? To show the customer that the product meets the requirements To show that the product is likely to interoperate with other vendors To demonstrate that Recommendations were implemented inside the product To improve the quality of ITU-T Recommendations

International Telecommunication Union Geneva, 9(pm)-10 February Need high-quality test results Repeatable by any competent test laboratory Can be used to compare with test results obtained for similar products Recognized by all and accepted in all geographical market regions

International Telecommunication Union Geneva, 9(pm)-10 February What kind of testing? Conformance Testing verifies if the implementation does what the Recommendation says it is supposed to do ProductTest Suite (Gold Standard) Interoperability Testing checks if two implementation communicate at functionalities level N = 6 products or 15 pairs Each product is tested 15 times N = 100 or ~5000 pairs Each product is tested 5000 times Functionalities Every pair

International Telecommunication Union Geneva, 9(pm)-10 February Frequently Asked Question Why not do only interoperability testing? Answer: We don’t know what to do when two implementations do not interoperate: If we change something – are we closer to the Recommendation or farther from it? Non-conforming changes destroy interoperability with other vendors Serious problems will not be discovered when observing functionalities only example: turning lights on and off in a new house is a good “interoperability” or functionality test. The house may burn down at a later time because the wiring did not conform to the standard

International Telecommunication Union Geneva, 9(pm)-10 February Sources of Interoperability Problems Recommendations Errors in Recommendations Ambiguities in natural language Unverified or invalid behaviours described Implementers’ different interpretations of text Requirements in text, more than one meaning No standardized questionnaire for supplier Incompatible Recommendations/implementations Different choices of options Device incompatibilities Different host system configurations

International Telecommunication Union Geneva, 9(pm)-10 February In addition to Base Recommendation Requirements Clause Extracted from text, need no interpretation Implementation Conformance Statement (ICS) Proforma Supplier declares what pieces were implemented from Recommendation Implementation eXtra Information for Testing (IXIT) Proforma Identifies non-standardized items, how architecture, interfaces are packaged Test Suite Structure Logically groups test cases Test Purposes one test purpose/verdict per test case Abstract Test Suite Set of tests that covers the Recommendation

International Telecommunication Union Geneva, 9(pm)-10 February Recommendations to support testing Base Recommendation R1 R2 R3 Rn TP1 TP2 TP3 TPn TC1 TC2 TC3 TCn ICS Answers TC1 TC3 TCn Requirements Clause TSS &TPATS Execution Test Cases ICS TSS & TP - Test Suite Structure and Test Purposes TP - Test Purpose ATS: - Abstract Test Suite TC - Test Case ICS: - Implementation Conformance Statement R - Requirement IXIT Information IXIT ICS Questions ICS Proforma

International Telecommunication Union Geneva, 9(pm)-10 February Requirements – NGN Draft Rec. Y.2702 (R-40)Each user/subscriber associated with an application service subscription is required to be uniquely addressable (R-41)It is required that it be possible for the end user to access a service simultaneously multiple times and/or from multiple devices. (R-42)It is required that it be possible to support multiple subscription profiles for an individual end-user. (O-1)It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the end user to uniquely identify the NGN provider. (O-2)It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the user to authenticate and authorize the NGN provider. Sections and – NGN User authentication for service access and network attachment

International Telecommunication Union Geneva, 9(pm)-10 February Implementation Conformance Statement (ICS) Proforma IndexTextStatusRef.Val.Support When User Equipment (UE) sends HTTP Request does Service Provider (SP) return http Response with M M O IV ) IV ) IV x) _Yes _No On receipt of HTTP Request from UE does SP send a redirect http Response with in the URL When UE sends HTTP Request does SP inform the Identity Provider (IdP) of the receipt Appendix IV to NGN Draft Rec. Y.2702 on Identity management (IdM)

International Telecommunication Union Geneva, 9(pm)-10 February Examples of Test Purposes in SIP Security Testing Ensure that the IUT, after sending the initial REGISTER request to the Registrar, ignores Registrar OK response by sending a second REGISTER request Ensure that the IUT re-sends the initial REGISTER request on receipt from the Registrar of a 401 Unauthorized response in which WWW-Authenticate header does not contain the nonce= field

International Telecommunication Union Geneva, 9(pm)-10 February Implementation Conformance Statement (ICS) ICS = ICS Proforma with answers A list of requirements and options claimed to have been implemented Used for Shopping for products to match for interoperability selecting Test Cases from test suite for test execution ICS

International Telecommunication Union Geneva, 9(pm)-10 February Test Suite Structure Test Suite Test Case Test Group

International Telecommunication Union Geneva, 9(pm)-10 February Test Case Structure Test Case Test Event Test Step

International Telecommunication Union Geneva, 9(pm)-10 February Conclusion First, Conformance testing then Interoperability testing High-quality Recommendations are needed Base Recommendations require additional parts to produce widely accepted test results Methodology Recommendations for testing are available – attached charts

International Telecommunication Union Geneva, 9(pm)-10 February Additional Slides All you need to develop high-quality Recommendations and Test Specifications

International Telecommunication Union Geneva, 9(pm)-10 February Conformance and Interoperability Testing Methodology Recommendations X General Concepts X Abstract Test Suite Specification X (Superseded by Z.160/170 series) X Test Realization X Requirements on Test Laboratories and Clients X Protocol Profile Test Specification X Implementation Conformance Statements Supplement 1 to X.290 series - Generic approach to interoperability testing Supplement 2 to X.290 series - Interoperability testing framework and methodology

International Telecommunication Union Geneva, 9(pm)-10 February Testing and Test Control Notation (TTCN-3) Z.161 (Z.140 Revised): Core Language Z.162 (Z.141 Revised): Tabular Format Z.163 (Z.142 Revised): Graphical Format Z.164 (Z.143 Revised): Operational semantics Z.165 (Z.144 Revised): Runtime interface Z.166 (Z.145 Revised): Control interface Z.167 (New): Using ASN.1 with TTCN-3 Z.168 (New): The IDL to TTCN-3 mapping Z.169 (New): Using XML Schema with TTCN-3 Z.170 (New): TTCN-3 documentation tags

International Telecommunication Union Geneva, 9(pm)-10 February Specification and Description Language Z.100 Overview of SDL-2008 Z.101 Basic SDL-2008 Z.102 Comprehensive SDL-2008 Z.103 Shorthand notation and annotation in SDL-2008 Z.104 Data and action language in SDL-2008 Z.105 SDL-2008 combined with ASN.1 modules Z.106 Common Interchange Format for SDL- 2008

International Telecommunication Union Geneva, 9(pm)-10 February Abstract Syntax Notation One (ASN.1) X.680 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Specification of basic notation X.680-X.693 (07/02) Information Technology - Abstract Syntax Notation One (ASN.1) & ASN.1 encoding rules X.681 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Information object specification X.682 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Constraint specification X.683 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Parameterization of ASN.1 specifications

International Telecommunication Union Geneva, 9(pm)-10 February Abstract Syntax Notation One (ASN.1) X.690 (07/02)Information technology - ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER) X.691 (07/02) Information technology - ASN.1 encoding rules: Specification of Packed Encoding Rules (PER) X.692 (03/02) Information technology - ASN.1 encoding rules: Specification of Encoding Control Notation (ECN) X.693 (12/01) Information technology - ASN.1 encoding rules: XML encoding rules

International Telecommunication Union Geneva, 9(pm)-10 February Supporting Recommendations Z.120 Message Sequence Chart (MSC) Z.150 User Requirements Notation - Language Requirements and Framework Z.151 User Requirements Notation - Language Definition Z.110 Application of Formal Description Techniques Z.450 Quality aspects of protocol-related Recommendations