Towards Interoperability Test Generation of Time Dependent Protocols: a Case Study Zhiliang Wang, Jianping Wu, Xia Yin Department of Computer Science Tsinghua University (Globecom2004)

2 Outline Background Motivation Our proposed method Conclusion and Future work

3 Background: Protocol Testing Conformance testing – Basic method of protocol testing – Test whether an implementation conforms to its protocol specification – Test a single protocol entity Interoperability testing – Complement of conformance testing – Test whether two or more protocol implementations can communicate with each other correctly and inter-operate as a whole system to perform functions specified in protocol specifications – Test two or more protocol entities

4 Background: Protocol Testing Protocol Specification Test Generation Abstract Test Suite Test Implementation Implementation Processes Protocol Impl. SUT Test Execution Test Verdict

5 Background: Time-Dependent Protocols Real-life network protocols – Behaviors of input and output – Their time of occurrence – Time dependent! Conformance testing of time-dependent protocols – Protocol modeling: Timed Automaton and its variants (e.g. TIOA) – Many researches have been done Interoperability testing of time-dependent protocols – No work considered time constraints – Existing methods are difficult to be applied in the real-life interoperability testing This paper studied interoperability test generation for time dependent protocols

6 Background: Neighbor Discovery Protocol One of the basic protocols in IPv6 protocol set Should be supported in all IPv6 implementations Corresponding functions in IPv4 – ARP (Address Resolution Protocol) – ICMP router discovery – ICMP redirect function Using 5 ICMP packet types: – RS: Router Solicitation – RA: Router Advertisement – NS: Neighbor Solicitation – NA: Neighbor Advertisement – Redirect

7 Outline Background Motivation Our proposed method Conclusion and Future work

8 Motivation System (Protocol) Modeling –Single protocol entity: TIOA (Timed Input Output Automata) –Two or more protocol entities: CTIOA (Communicating TIOA) An TIOA set Channels between TIOAs Test Generation Method Generate grid automaton for each TIOA Calculate a composite product of grid automata Apply existing non-timed interoperability test generation methods Problem: state space explosion !! An improved method calculate the composite automaton of TIOAs using reachability analysis method Generate interoperability test sequences Analyze the executability of derived test sequences

9 Outline Background Motivation Our proposed method –Formal model and protocol specification –Test architecture –Test generation –Executability analysis Conclusion and Future work

10 Formal Model Communicating Timed Input Output Automata (CTIOA) –a set of TIOAs M: M i = S Mi : set of states; L Mi : set of input and output actions: (“!a”, “?b”); s Mi 0 : initial state; C Mi : set of clocks; T Mi : set of transitions: –a set of channels C: C ij represents the communicating channel from TIOA M i to M j. Semantic of channels: the output of TIOA M i can be transferred to the input of M j via channel C ij An assumption: time taken for an IO action via a channel is approximately equal to 0 (Normal network load) MjMj MiMi C ij a !a ?a Predicate of clock values Clock set to be reset IO action

11 Protocol Specification Router discovery function in ND Protocol –Two types of nodes: Router and Host –Protocol specification uS_RA: Unsolicited Router Advertisements S_RA: Solicited Router Advertisements RS: Router Solicitations Clock set: Router node: {t11, t12} Host node: {t21, t22} Channel set: {C RH, C HR } 12 Transition Examples: ?RS[t11>=3]/{t12} 13 ?RS[0<t11<3]

12 Test Architecture PO: Point of Observation Control line: e.g. Telnet

13 Test Cases Generation (overview) Calculate the composite automaton of TIOAs using reachability analysis method Generate interoperability test sequences Analyze the executability of derived test sequences

14 Reachability Analysis Global Reachability Graph Synchronizing Transitions and ________________________________________________________ and

15 Reachability Analysis An example of Global Reachability Graph (Tree) –DFS-based method Model

16 Test Cases Generation(overview) Calculate the composite automaton of TIOAs using reachability analysis method Generate interoperability test sequences Analyze the executability of derived test sequences

17 Test Generation An example (b) (a)

18 Test Cases Generation(overview) Calculate the composite automaton of TIOAs using reachability analysis method Generate interoperability test sequences Analyze the executability of derived test sequences

19 Executability Analysis Executability: satisfies all time constrains IOTS = ST 1 ·ST 2 ·...·ST n [Theorem 1] t ij k must be represented by a linear combination of tt s (s = 0,1,…,k-1) and d ij (initial local clock values). Global Clocks Local Clocks C 1 ({d ij |i, j}, tt 0 ) ∧ C 2 ({d ij |i, j}, tt 0, tt 1 ) ∧... ∧ C n ({d ij |i, j}, tt 0, tt 1,..., tt n-1 ) C 1 ({t ij 1 |i, j}) ∧ C 2 ({t ij 2 |i, j}) ∧... ∧ C n ({t ij n |i, j}) Local Clocks C({d ij |i, j}, {tt k |k}) Global Clocks

20 Executability Analysis (cont) Problem 1: Executability Determination Given (1) IOTS = ST 1 ·ST 2 ·...·ST n ;(2) each initial local clock values d ij. Determine if IOTS is executable. C 1 ({d ij |i, j}, tt 0 ) ∧ C 2 ({d ij |i, j}, tt 0, tt 1 ) ∧... ∧ C n ({d ij |i, j}, tt 0, tt 1,..., tt n-1 ) C 1 (tt 0 ) ∧ C 2 (tt 0, tt 1 ) ∧... ∧ C n (tt 0, tt 1,..., tt n-1 )

21 Executability Analysis (cont) Algorithm:

22 Executability Analysis (cont) Example: (b) (a)

23 Executability Analysis (cont) Problem 2: Generating Executable Test Sequence –Basic idea: select a set of appropriate initial clock values –assume that (1) a set of linear constraints C({d ij |i, j}, {tt k |k}); (2) the value bounds of all initial clocks (d ij ∈ [0, D ij ]) are known, determine bounds of d ij (i=1,2; j=1,2,…,n i ) to make time constraints satisfied. Method: Solve a linear programming problem –evaluate the value of Max(d ij |i, j) and Min(d ij |i,j) No solution: must-inexecutable Has a solution: may-executable the value bound of d ij : [Min(d ij |i,j), Max(d ij |i, j)] Problem: –More than one local clocks –Difficult to determine initial values of each local clock

24 Executability Analysis (cont) Problem 3: Adaptive Executable Test Sequence –A heuristic method: set part of initial clock values in advance according to the test scenario –determine whether the test sequence is must-inexecutable or may- executable If must-inexecutable then prune it from reachability tree; Else evaluate value bounds of unset initial clocks. –divide the valid clock region into several sub-regions, so that only a unique test sequence tree exists in one sub-region

25 Example: set d 21 =0 Adaptive test sequence

26 Conclusion and Future work Present a formal method to generate timed interoperability test sequence for time dependent protocols Executability analysis can be reduced to a Linear Programming problem. Alleviate state space explosion problem Future work: –Analyze fault coverage of this method –Extend this method to the more common situations of interoperability testing –Apply it to the real-life protocol testing activities

Thank you! If you have any questions, please send to: