Automatic Test Generation from here until the end (of my Phd.) University of Geneva Levi Lúcio SMV & Les Diablerets

2 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 3 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 4 The Overall Picture We do Model-Based test generation :  Given a model (CO-OPN / Fondue) of an SUT…  … we want to generate meaningful tests using that model. Test Cases ModelSUT

SMV & Les Diablerets 5 The Overall Picture (2) Main challenges for a test set:  Coverage of infinite execution paths (loops)  Coverage of infinite input values My predecessors and Didier developed a theory for coping with this problem:  By stating hypotheses about the behavior of the SUT in order to reduce the exhaustive test set;  If the hypotheses hold on the SUT, the final test set will be finite and valid.

SMV & Les Diablerets 6 Test Generation Flow Does P satisfy SP? PSP Test Selection (Hypotheses H Application) Execution of P using T Oracle P satisfies, or not, T! P does not satisfy SPUndefinedP satisfies SP! P satisfies, or not, H Test Set T no inconclusiveyes noyes Correction of P Test Procedure The validity of the tests is decided by executing them against the model prototype!

SMV & Les Diablerets 7 Modeling Language Integration The idea is to integrate the test theory with the CO-OPN language; CO-OPN allows building specifications based on three different kinds of modules:  Algebraic Data Types (Adts) - types + operations  Class modules - state and coordination  Context modules - coordination but no state

SMV & Les Diablerets 8 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 9 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 10 Test Language The test language is a language for expressing constraints on the operation sequences and operation’s parameters that compose tests; It includes predicates to:  Limit the size and type of the operations’ sequences;  Choose input values from their domain: All values One value One value per equivalence class

SMV & Les Diablerets 11 Test Language Mobile Phone Example A (simple) mobile phone is composed by:  A card containing (among other things) the personal identification number (PIN) to activate the phone. The card gets blocked after 3 wrong attempts at inserting a PIN.  A phone controller holding (among other things) the state of the phone, which can be: Off Standby (waiting for the correct PIN) Idle (ready to receive or make calls) In our specification the user can perform three actions:  Turn on the phone;  Insert a PIN number,  Reset the card when it becomes blocked

SMV & Les Diablerets 12 Test Language Constraints for the Phone Spec ConstraintSet NatelBehaviors; Interface Constraints insertPins; block; reachOn; Body Constraints nWrongPins; Use Boolean; Pin; Axiom [] in nWrongPins; f in nWrongPins => f. HML( ) in nWrongPins; f in nWrongPins, depth(f) HML( ). f. HML( ) in insertPins; depth(f) HML( ). f. HML( ) in block; depth(f) HML( ). f. HML( ) in reachOn; Variables f : HML; m : method; g : gate; End NatelBehaviors; Exported constraints Locally defined constraints Imported Algebric Data Types Variables used in constraint definition Constraints’ Semantics

SMV & Les Diablerets 13 Integration and Unit Testing We always perform Black-box testing… … but it can be done at different levels of the model! Coopn Model - outer context SUT interface Class 1 Class 2 Subsystem Class 3

SMV & Les Diablerets 14 Tests are created Top-Down The engineer starts defining tests for the full system and ends defining tests for the individual classes (refinement) The SUT is tested bottom up by first applying the tests to the individual classes or to the smallest subsystems ModelSUT

SMV & Les Diablerets 15 Test Drivers and Oracles Tools are available to execute tests on Web Pages or Graphic Gui; Could we integrate with Junit (Unit Testing)? TTCN/3 is a solution for integration/system testing… UML 2.0 test profile? A homogeneous solution doesn’t seem to exist - I’m not sure I can tackle this…

SMV & Les Diablerets 16 Testing Fondue Specifications The idea is to translate Fondue into CO- OPN so that the test facilities can be used (Luis’ presentation); Can OCL be translated into CO-OPN behavior? Will the translated model be “testable” with the constraints’ language? Should we already express test intentions at the Fondue level? - are these intentions already present anyway?

SMV & Les Diablerets 17 More ideas… We would like to manage behaviors of the SUT which cannot be calculated in test generation time:  Example: an e-banking application proposes a challenge as part of the authentication mechanism. How do we calculate the reply to the challenge in order to login successfully during the test?  Solution: Introduce the possibility to have “external” functions that will perform calculations in testing time. Predefined “templates” for the constraint language. Example: reach at least one output for each operation in the module’s interface.

SMV & Les Diablerets 18 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 19 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Technical challenges; Conclusion.

SMV & Les Diablerets 20 Implementation Issues CO-OPN Spec Constraint Modules Semantic explorer: - Java - Prolog Validated tests

SMV & Les Diablerets 21 Implementation Issues (2) Unfortunately the tests cannot be fully calculated syntactically:  Easy to calculate input values for each operation…  But how to calculate the outputs?  Worse, how to calculate inputs in function of outputs? The Java prototype is one solution, but only works “forward”; Prolog works “forward” and “in reverse”…

SMV & Les Diablerets 22 Implementation Issues (3) But… representing the state of a real model in Prolog may be too heavy. Resolution may become too slow! Will it be possible to “reverse” all operations without falling in Prolog resolution traps? Possible solutions:  Implement several resolutions for each of the tackled problems;  Be less ambitious in the constraint language;  Use CLP technology to treat certain types.

SMV & Les Diablerets 23 Implementation Issues Under Thought Generation of object parameters for dynamic OO models; Mechanisms for decomposing CO- OPN’s operations in classes of behavior:  According to CO-OPN’s semantics!

SMV & Les Diablerets 24 Presentation Plan The overall picture; Ideas I would like to explore in my thesis; Implementation issues; Conclusion.

SMV & Les Diablerets 25 Conclusion There are many ideas, but how many of them are feasible in the remaining time? Development is being done now, more experimenting with Prolog and CLP is necessary… I am now focusing on the core of the test generation; Remaining time for issues to be selected.