Verification and Test Automation of UML Projects Nikita Voinov, Vsevolod Kotlyarov (Saint-Petersburg State Polytechnic University) The Third Spring Young Researchers Colloquium on Software Engineering Moscow May

SYRCoSE Software Quality Assurance Unambiguity of requirements Solution: usage of formal languages and notations (UML – Unified Modeling Language) Software functionality’s correctness checking Solution: –verification –testing

SYRCoSE Issues Model of the system on some formal language shall be created for verification Manual testing Long and laborious processes Lack of technologies which allow to integrate testing and verification

SYRCoSE Proposed Solution Initial Requirements (UML) Verification (VRS) Testing (TAT) correction of requirements TTCN (Testing and Test Control Notation) automated process of formalization automated process of tests generation

SYRCoSE Basic Protocol – a simple MSC diagram, which specifies: the state of the system where the system shall perform some activity – pre-condition the activity itself (transmission of a message or performance of an action) – process part the state of the system after the activity is performed – post-condition pre-condition post-condition action message process part

SYRCoSE VRS (Verification of Requirement Specifications) combines model checking with deductive verification checking the properties of requirements represented with basic protocols detection of non-deterministic behavior, unreachability of specified states, deadlocks criterion of requirement’s satisfiability in VRS is existence of sequence of required events (actions, signals) contained in basic protocols in definite order; such sequence is called “trace”

SYRCoSE Conversion of a Fragment of UML SM Diagram into a Basic Protocol Special VRS module “uml2bp” is used for autoformalization

SYRCoSE Verification Stage Step 1. Formulation of filters and heuristics for the current project. They help to decrease number of traces by pointing the trace generator to the definite direction. Step 2. Performing an automatic trace generation cycle. Step 3. Analysis of findings with deadlocks, inconsistency and other issues. Correcting the generated basic protocols or initial requirements. Repeat steps 1-3. Step 4. Analysis of generated traces with a script to check whether the coverage criteria are satisfied. Repeat steps 1-4 if needed.

SYRCoSE Sample State Machine Diagram

SYRCoSE Graph with All States without Detailed Behavior

SYRCoSE Graph with All States with Detailed Behavior of ‘Suspended’ State Suspended Start UDI_LDI UEI_LEI UEA

SYRCoSE Trace for the Graph with All States

SYRCoSE Trace for the Graph with Detailed Behavior of ‘Suspended’ State

SYRCoSE Traces Merging ( Glue In )

SYRCoSE TAT (Test Automation Toolset) templates for generation of tests in C, Java, TTCN languages automated testing cycle based on user-defined scenarios developed in formal language MSC customizable for different platforms with different environments

SYRCoSE Overall Scheme of Automatic TTCN Tests Generation UML diagram

SYRCoSE Adding Values to MSC Input: MSC with parameters Output: MSC with values Configuration items:.xls file with parameters values; VB script for xls-> txt conversion; add_values.tcl (script) Notes: the script finds identical parameters names in MSC and.xls file and replaces them in MSC by values in.xls..xls file with parameters values : MSC with values :

SYRCoSE TTCN Test Cases Generation TTCN test case: File for test cases launch:

SYRCoSE Results of Piloting in Telecom Project autoformalization stage: about 4000 basic protocols were generated in 3 minutes (while manual efforts are estimated in basic protocols per day) verification stage: about 100 findings were discovered, 12 of them were considered defects and fixed in future versions of the product tests generation stage: manual automated

SYRCoSE Novelty of Work automated conversion of UML specifications into formal model in the language of basic protocols methodic of large-scale models analysis using VRS technology new TAT template for generation of tests in TTCN language

SYRCoSE THANK YOU