1. Testing Transition Systems with Input and Output Testers Alexandre Petrenko Nina Yevtushenko Jia Le Huo TestCom’03, May 27 th, 2003

2. 2 Outline Motivation Basic idea Testing frameworks –Queued-quiescence –Queued-suspension Conclusions

3. 3 Conformance Testing SpecImp TestsTest execution Does Imp conform to Spec? Fault model 

4. 4 Input/Output Transition Systems Non-deterministic IOTS: model of Spec and Imp Output of IOTS cannot be blocked: basic assumption Spec and Imp are input-enabled coin coffee, milk  ?coin !coffee !milk ?coin Spec

5. 5 ?coin Motivating Example ?coin !coffee !milk ?coin !coffee !milk ?coin !coffee SpecImp Quiescent trace coin coin coffee milk yields the ioco test case: fail coin coffee milk, coffee pass  milk,  fail ?coin Is the test case still sound if output of Spec cannot be blocked?

6. 6 Contradiction The ioco test case blocks coffee after coin Output of IOTS cannot be blocked: basic assumption How can I block a coffee on its way out?

7. 7 Problem Statement A tester not blocking output of Imp must be input-enabled An input-enabled tester usually –has unbounded test runs –makes choice between input and output Both features are not desirable in practice We have to define a tester that –is input-enabled –reaches verdict in finite steps –never chooses between input and output

8. 8 Basic Idea Tester Imp Output queue finite capacity for practical reasons Input test process executes a given input sequence Output test process produces pass when an expected output sequence followed by quiescence is read from the queue and fail otherwise Input-enabled, and no choice between input and output Input queue

9. 9 Queued-Quiescent Traces For a quiescent trace  of Spec,   I   O   Qqtraces(Spec) For input sequence , Qqtraces o (Spec,  ) includes all the output sequences in response to  : {  '   O*  |  '   Qqtraces(Spec)} Qqtraces(Spec) = Qqtraces(Imp) (qq-trace equivalent) ?coin !coffee !milk ?coin Spec ?coin !coffee !milk ?coin !coffee Imp !coffee !milk    Qqtraces o (Spec,  ) !coffee !milk    Qqtraces o (Imp,  )

10. 10 Queued-Quiescence Relations For non-oscillating Spec and Imp, if input sequence  exists such that : –Qqtraces o (Imp,  )  Qqtraces o (Spec,  ) =  (no pass) then Imp is qq-separable from Spec –Qqtraces o (Imp,  ) Qqtraces o (Spec,  ) (  fail,  pass) then Imp is qq-distinguishable from Spec –Qqtraces o (Imp,  )  Qqtraces o (Spec,  ) (no fail) then Imp is qq-weakly-distinguishable from Spec qq-separable  qq-distinguishable  qq-weakly- distinguishable

11. 11 Test Derivation with Fault Model  Given Spec and Imp, check input sequences of length 0, 1, 2, … until a sequence is found such that –qq-separates, or –qq-distinguishes, or –qq-weakly-distinguishes Imp from Spec, or –the input buffer capacity k is reached Repeat for all Imp   k = ?

12. 12 A Finer Tester After executing the qq-test case with input coin on Spec and Imp !coin ?coin !coffee !milk ?coin !coffee !milk ?coin !coffee SpecImp fail ?milk pass   fail ?milk, ?coffee  ?milk, ?coffee,  The resulting IOTS can be distinguished by a qq-test case: ?coin

13. 13 Queued-Suspension Testing 11 22  11  qq-test case of Spec-after-(  1,  11 ) qs traces are sequences of qq traces executed by the qq testers: (  1  1i  )(  2  2j  )...(  n  nk  ) 22  12  qq-test case of Spec-after-(  1,  12 )... 33  21  22  23   24   25  33 33 33 33 qq-test case of Spec

14. 14 Conclusions Outputs cannot be blocked: basic assumption Testing frameworks: qq and qs Comparison with the ioco framework: –information of Interleaving of inputs and ouputs are lost –so the conformance relations are coarser than ioco –but ioco testing is not applicable here Open questions: oscillating Spec, k = ?, etc

15. 15 Merci beaucoup!

16. 16 An Example ?coin !coffee !milk ?coin Spec coin Output queue (capacity  2) Input queue (capacity  2) ?coin !coffee !milk ?coin    !milk ?coin  !coffee

17. 17 An example (Cntd.) We have coin coffee milk pass  fail When the ioco test case below is applied to Spec with queues fail coin coffee milk, coffee pass  milk,  fail Is this test case still sound?