1. An Automated Test Strategy Based on UML Diagrams
12 January, 2019

2. Objectives
Use for test planning the same UML diagrams developed (e.g., by Rational Rose) for design and analysis
Tool-supported generation and maintenance of an updated and strategic test plan, as the design evolves.
12 January, 2019

3. Cow_Suite approach Cow_Suite =Cowtest pluS UIT Environment
Cow_Suite consists of two main components:
UIT (Use Interaction Test) and
CoWTeSt (Cost Weighted Test Strategy)
Cow_Suite =Cowtest pluS UIT Environment
The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps.
The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization
[*: the internal description of the project model, given by Rational Rose in a .mdl file]
12 January, 2019

4. Cow_Suite Scheme Test Procedures list U I T Selection Strategy
Obj 1
Obj 2
Obj 3
Method1()
Method3()
Method2()
SDs selection according to the strategy chosen
Interaction with the user for deriving executable test procedures
Automatic UIT application for Test cases derivation
Test case
Test Procedures
list
U I T
Test specification
12 January, 2019

5. Nodes and Arcs identification (Activity 1)
The UML design is analyzed for deriving two different graph: The Main Graph and the Design Graph. These represent two different points typologies of test.
For deriving the two different graphs is necessary to analyze:
the Use Case View
UCs, SDs, Actors represent the nodes of the Main Graph
Their relations represent the arcs of the Main Graph
the Logical View
The UC realizations and their associated SDs represent the nodes of the Main Graph, the remaining design elements are the nodes of the Design Graph
The relations and the dependences are the arcs of the two graphs
12 January, 2019

6. Use Case View
The Use Case View: describes the set of UCs (and scenarios) that represents the significant functionalities and the interaction of the system with the external world.
The tree derived represents a detailed documentation of how features/system requirement are realized in the Use Case View.
The path from a high level Use Case to a leaf (UCs or SDs) describes the refinement of the associated system feature
The Test Cases derived from the SDs of the Use Case View are specifically designed for
System integration Test
Subsystem integration Test
12 January, 2019

7. Use Case View vs Logical View
The Logical View describes how the system functionalities are provided in terms of static structure and dynamic collaborations of the objects. This information is used for completing the graphs and in particular fro the Main Graph:
The “Use Case Realizations” link the information of the Use Case View and the Logical View. For each UC its UC realization:
Traces the evolution of the UC in the Design Model
Details the sub-flow of a UC with Sequence (Collaboration) Diagrams
Specifies the Classes and the relationship that participate in the realization of the UC (Class Diagrams)
Optionally holds the package Design Link which holds the list of the packages of the design model that effectively implement the UC
12 January, 2019

8. Trees Derivation (Activity 2)
The Main Graph and the Design Graph are automatically organized by Cow_Suite tool in different trees by using a DFS_mod algorithm
Hierarchical Trees
The case study is Course Registration System from RUP documentation
12 January, 2019

9. Choice of a tree (Activity 3)
Hierarchical Tree
The users can get a complete view of the status of the functionalities specification and a detailed documentation:
UCs and their realization
SDs associated to each specification level
The links between Use Case and Design Model
Reused nodes
Not linked model elements
12 January, 2019

10. Design Link Package
Design Link package: for every use case realization it collects the package/s that represent the implementation of the relative UC.
It is an explicit link between the Main Trees and the Design trees
12 January, 2019

11. Design Tree Analyzing this tree it is possible to:
Design Tree: is derived directly from the Design Model and represents the organization of the packages in a hierarchy.
Analyzing this tree it is possible to:
Observe the dependence relationship between packages
Know which are the reused packages and where the reuse happens
Connect to the main tree the low level SDs not explicitly linked in the .mdl file
By selecting the SDs of the Design Tree and applying to them the Cow_Suite test derivation strategies, a list of low level Test Cases can be derived
12 January, 2019

12. Weights assignment (Activity 4)
Systems are naturally composed of different parts realizing one or more functionalities, each with different importance for the overall system performance.
The effort devoted to the test of different functionalities must be planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…).
Following level by level the tree structure of the UCs and SDs, the strategy allows the people in charge of the development of one or more functionalities to simply annotate the corresponding node in the tree with a number (weight).
0.75?
0.60?
0.65?
0.65!!!
Sequence
Diagram: To ...
To Do List
12 January, 2019

13. Deduce the Critical Profile
The importance, represented by number (weight) belonging to [0,1] interval, can be assigned in two different manner:
Interacting with Cow_Suite Tool: Following level by level the tree structure of the UCs and SDs, the user annotates the corresponding node in the tree with its weights
Directly in the .mdl file (UCs only): During the development of the Use Case Model architects/designers assign a value, belonging to [1,9] interval, to each Use Case.
The value is memorized in the use case documentation tab with the following structure: WGH = value
(For SDs or when no value is expressed for UCs the default value is 5)
12 January, 2019

14. Weighted tree derivation
The tool :
reads the assigned values the .mdl file (if they exist)
sums the values of the node at the same level in the tree
normalizes the weights among the nodes at the same level so that their sum is 1.
The user can always modify the assigned weights
12 January, 2019

15. Integration Testing vs Integration Stage
To realize a complete system functionality, it is generally necessary to execute several actions realizing lower level functionalities
A system functionality is realized by the different components interaction.
Integration Testing: testing interactions between system components (processes, packages, subsystems… ) at an appropriate level of abstraction.
Integration Stage: is useful because generally the functionalities are not specified at the same level of detail and with the same number of UCs and SDs
12 January, 2019

16. Integration Stage definition
The first integration stage is represented by the main UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree).
The i-th integration stage is represented by the UCs positioned in at i-th level of the tree and every SDs, children of these nodes, situated at i+1-th level.
NOTE: the SDs at level i+1 represent the interaction between the different components that realize the functionalities described in the UCs at i-th level.
12 January, 2019

17. Integration Stage selection (Activity 5)
The Cow_Suite user selects the integration stage at which the test is conducted.
The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the selected integration stage plus all the tree leaves existing at higher levels.
12 January, 2019

18. Test Strategy selection (Activity 6)
Two different situations are considered in test cases planning:
1: Cowtest_ing with fixed number of tests
A certain budget is available, which is translated in terms of number of tests, NT, to execute.
In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived. NT
Test Case
Test Case
12 January, 2019

19. Test Strategy selection (Activity 6)
2: Cowtest_ing with fixed functional coverage
A certain percentage of functionalities must be covered.
In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to execute.
Functional Coverage = 90% 10 2 1
Minimum Number of test to execute = 16
12 January, 2019

20. Select the Test Strategy
Cowtest_ing with fixed functional coverage.
The most critical system functionalities are
selected for reaching the fixed coverage.
Test Procedures are distributed accordingly.
Cowtest_ing with fixed number of tests.
The selection of the most suitable distribution of test procedures is developed on the basis of leaves weights.
12 January, 2019

21. Weighted Tree Derivation (Activity 7)
The weights assigned to each node are used to define a relative importance factor, in terms of how risky is that node and how much effort should be put to test it.
Considering every node, from the root down to the elements belonging to the integration stage considered, its final weight is computed as the product of all the nodes weights on the complete path from the root to this node.
The final resulting weight is an element of discrimination for choosing amongst the tests to execute.
FW=0.12
FW=0.6
GEF
UML Meta-Model
Libraries
ArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.6
0.4
0.3
12 January, 2019

22. Cow_Suite Scheme U I T Selection Strategy Cow_Suite
Obj 1
Obj 2
Obj 3
Method1()
Method3()
Method2()
SDs selection according to the strategy chosen
Automatic UIT application for Test cases derivation
Test case
U I T
12 January, 2019

23. Sequence Diagrams Selection (Activity 8)
For each fixed integration stage, a weighted subtree is derived according to the choosen test criterion. Only the relative SDs are considered for Test Case generation.
SDs selection
Final weight
N. Test Procedures
12 January, 2019

24. Test Case Derivation (Activity 9)
SDs Selection
The tool uses the UIT method, Use Interaction Test, to derive the Test Cases.
The method is mainly based on the SDs (particularly objects, messages and parameters)
12 January, 2019

25. Use Interaction Test methodology
A method to systematically derive Integration Test suites from UML diagrams.
Inspired at large by the Category Partition method:
it defines classes of functional equivalence by selecting significant values (choices) of relevant categories (parameters or data)
Incremental test strategy (over the Use Case organizational structure)
12 January, 2019

26. 9.1 Highlight Test Units
Horizontal axis shows a set of objects, Test Units, that interact through messages.
Those are system units separately testable for exercising a possible use of system.
Example:
DecisionModel, Decision, GoalModel, Goal.
12 January, 2019

27. 9.2 Identify Interactions Categories
Vertical axis shows:
the exchanged messages, Interactions Categories, involved in object interactions.
We list all the messages entering in the selected Test Unit.
Message: is a communication where the sender object invokes an action, a service, belonging to the receiver object which will perform it.
12 January, 2019

28. 9.3 Identify Settings Categories
Vertical axis shows:
their parameters and relevant inputs, Settings Categories.
Example:
DecisionModel
Settings: _decisions
Interaction:
DecisionModel()
getDecisions()
Decision
Interactions:
Decision(name:String,priority:int)
GetName(),
GetPriority()
12 January, 2019

29. 9.3 Additional Information: Class Diagram
Complementary analysis of the Class Diagram description (mainly used for searching the Settings Categories).
These are attributes (or a subset of them) of a class (and the corresponding SD’s object) that affect object interactions and are represented by input parameters used in messages or data structures.
12 January, 2019

30. 9.4 Define Messages_Sequences
For each SD where the studied Test Unit is involved, observing the vertical temporal order of the messages along its lifeline the tool detects a set of: Messages_Sequences created considering each message entering in the Test Unit plus, if any, all the messages (not yet considered) belonging to its activation bounded from focus of control region.
A Messages_Sequence describes interactions between the selected Test Unit and the other objects necessary to realize a specific system functionality.
12 January, 2019

31. The implemented UIT: UIT_sd
From an industrial point of view UIT is too expensive because:
generates many Test Cases of a small granularity
needs lot of ad-hoc stubs
UIT_sd, for each selected SD
Messages_Sequence is defined considering, in temporal order, each message with no predecessor association plus, if any, all the messages belonging to the nested activation (focus of control region). A Messages_Sequence represents a behavior to be tested and describes the interactions among objects necessary to realize the corresponding functionality.
12 January, 2019

32. 9.4 Define Messages_Sequences in UIT_SD
Example:
Messages_Sequence 1
DecisionModel()
Decision (name, priority)
Messages_Sequence 2
GoalModel()
Goal (name, priority)
……………
12 January, 2019

33. 9.5 Test Cases Derivation
For each traceable Messages_Sequence a Test Case is generated. It contains the list of all Settings and Interactions Categories involved in the Messages_Sequence and their values.
Detailed Test Case Description
12 January, 2019

34. Number of Test Procedures
The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.
Number of associated Test Procedures
12 January, 2019

35. 9.5.1 Analyze possible subcases
The messages involved in a Test Case (TC) may contain some feasibility conditions
usually described in messages notes or specifications tab
formally expressed using the OCL notation
TC is divided in different subcases corresponding to the diverse choices values.
12 January, 2019

36. Test Procedures Specification (Activity 10)
Test Specification
For each identified category, we consider all its possible values and constraints (“choices”).
The user interacts with the tool for inserting Choices values.
12 January, 2019

37. 10.1 Interactions Categories Specification
Categories: objects interactions, exchanged messages.
The tool implements two possible choices for the user:
the default values,
the user values
12 January, 2019

38. State Diagram
If a State Diagram is linked to a specific Test Unit (class), further information can be found about the relevant configurations of InteractionCategories
State diagram of the “Critic” class
12 January, 2019

39. 10.2 Settings Categories Specification
parameters and inputs relevant for the Test Unit.
The tool implements two possible choices for the user:
the default values,
the user values.
12 January, 2019

40. 10.3 Constraints on Choices
Some choices combinations could result contradictory or meaningless. Following the Category Partition methodology, constraints must be added to the Setting Categories choices.
Constraint are specified using:
Properties, that can be assigned to certain choices to check the compatibility with other choices.
IF Selector, is a conjunction of properties previously assigned to other choices.
12 January, 2019

41. Example SettingsCategories: pattern size: empty [Property Empty]
single character [Property Not Empty]
many character [Property Not Empty]
quoting
pattern is quoted [Property quoted]
pattern is not quoted [if Not Empty]
pattern is improperly quoted [if Not Empty]
12 January, 2019

42. Settings Categories: Specification Constraints
Test Specification:
Decision(_name, _priority):
_name
Naming [if priority<5]
Class Selection [if priority=0]
Storage
Inheritance
Modularity…..
_priority
0 [Property priority=0] 1 2 3
7 [Property not priority<5]
12 January, 2019

43. Settings Categories: Specification Constraints
Test Procedure
DecisionModel()
Opening an existing document
Decision( _name, _priority)
_name
Naming [if priority<5]
_priority
7 [Property not priority<5]
Test Procedure
DecisionModel()
Opening an existing document
Decision( _name, _priority)
_name
Naming [if priority<5]
_priority 4 X ok
12 January, 2019

44. Test Procedures Generation (Activity 11)
A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages_Sequence.
12 January, 2019

45. Test Procedures: an example
Test Specification:
DecisionModel:
Settings:
_decisions
Naming
Storage
Relationships……..
Interactions:
getDecisions
Opening a new file
Opening a saved file
After a modification and before saving
After a modification and after saving
DecisionModel()
Class constructor
Test Procedure
getDecisions()
getName()
Opening a saved file
_decisions
Naming
Note: Such a Test Case must be repeated for all possible values of _decisions
12 January, 2019

46. Test Suite artifact (Activity 12)
The set of all generated Test Procedures is named Test Suite
12 January, 2019

47. Note
The Test Procedures, as derived by the tool, are not directly executable ( Test Scripts). The derived Test Procedures have to be executed and logged by a test driver.
CowSuite goal is:
giving automatic and systematic support to construct a meaningful Test Suite.
It is not to directly execute the tests.
Cow_Suite
MDL Analizer
Test Tree
Generator
Weights Manager
Test Case
Builder
Checker
REI
Rational Rose
.Tlb library
.mdl Files
Test Driver
12 January, 2019

48. Remarks The degree of detail of the Test Suite depends directly on:
the design detail level
granularity of the inserted values (selection of the constraints and choices of the UIT categories).
Currently: the final result is a text document containing the Test Suite.
In future: different format (e.g. XML), that can be accepted in input by the adopted test driver.
12 January, 2019

49. Questions?
12 January, 2019

50. Procedure for tree derivation
The strategy followed for the tree derivation is a modified depth-first search
DFS_Mod (G)
For each vertex vV[G]
do color[u] WHITE
[u]NIL
time0
For each vertex u Actors[G]
                                    do DEF-Visit_Mod (u)
For each vertex u V[G]\Actors[G]
do if color[u]= WHITE
then DEF-Visit_Mod (u)
The visit starts only from the actors nodes
12 January, 2019

51. Procedure for tree derivation
A new actor node is added to the graph G.
The DEF-Visit_Mod ends at this step
 DEF-Visit_Mod (u)
Color[u]  GRAY
D[u]  time time +1
For each v Adj[u]
If v Actor[G] then
v’ NewNodeGeneration(v)
             [v’]u
else
    Do if color[v] = WHITE
          then [v]u
           DFS-Visit_Mod(v)
                else
               v’ NewNodeGeneration(v)
                 [v’]u
             if vPredecessors[u,v] then
             TreeDuplication [v’,v]
 color[u]  BLACK
 f[u]  time time +1
If the node v is an already visited node, its sub_tree is duplicated under the new added node v’.
Checks if the node v belongs to the path form u to the root
12 January, 2019

52. Conclusion
Systematic and automated support for test planning and test case generation, based on the same UML diagrams developed for design and analysis OK
Tool integrated with the design tool (e.g. Rational Rose) OK
12 January, 2019