Presentation is loading. Please wait.

Presentation is loading. Please wait.

Model-Based Black Box Testing and its Standardization Layers THE SOLUTION TO BOOST YOUR TEST EFFICIENCY MaTeLo.

Similar presentations


Presentation on theme: "Model-Based Black Box Testing and its Standardization Layers THE SOLUTION TO BOOST YOUR TEST EFFICIENCY MaTeLo."— Presentation transcript:

1 Model-Based Black Box Testing and its Standardization Layers THE SOLUTION TO BOOST YOUR TEST EFFICIENCY MaTeLo

2 Since 1998 French: 45 experts Turnover 2010: 4 M Expertise offer: PROCESS IMPROVEMENT SAFETY ENGINEERING SYSTEM ENGINEERING OPTIMIZED TESTING SOFTWARE RELIABILITY EMBEDDED SYSTEMS ALL4TEC

3 OUR COMPETENCES OFFER TrainingConsultingEngineeringTooling MaTeLo EXAM Safety Architect M Elopée

4 AGENDA THE TEST PROBLEM THE EXPECTED SOLUTION THE MODEL-BASED TESTING SOLUTION A PRAGMATIC INDUSTRIAL USE-CASE A MORE STANDARDIZED TOOLS INTEGRATION

5 THE TESTING PROBLEM Test Availability System Complexity Testing Effort Time Reality Product Life Reliability Market Expectation Test Needs Usage Intensity Development Phase Customer Validation Normal Usage Test Resources Lack Test Focus Deviation

6 HOW TO SOLVE THIS ISSUE? Test the SUT usage before product release Use best in class engineering process Widely use automatic generation Use dedicated tools that decuple engineering productivity Use formalization and traceability

7 Test Environment Test Environment MODEL-DRIVEN ENGINEERING Functional Requirements Functional Requirements Usage Model Usage Model Design Model Design Model Test Cases Code Test Scripts EXE DESIGNTEST Generate DeriveCompile ImplementScenarize MaTeLo Model Based Testing

8 WHAT IS MaTeLo? Formal Test Specification Fully Measured Process Product Usage Point of View Markov Test Logic MaTeLo Test Execution Compatibility Systematic Engineering Productivity More Productive Coverage Bidirectional Requirements Traceability

9 MATELO SCREENSHOT

10 INTEGRATED TEST WORKFLOW MaTeLo Usage Model Editor Test Design into Usage Models MaTeLo Usage Model Editor Test Design into Usage Models MaTeLo Test Campaign Analysis Confidence And Coverage Reporting MaTeLo Test Campaign Analysis Confidence And Coverage Reporting MaTeLo Testor Automatic Test Cases Generation MaTeLo Testor Automatic Test Cases Generation PLAN EXAM Test Automation EXAM Test Automation DO CHECK ACT

11 MaTeLo EDITOR Usage Model Design

12 MATELO USAGE MODELS BEGINNING END STATE TRANSITION MACRO CHAIN P robability on every Alternatives A ssociation of Requirements C onfiguration of Test Operations S etup Transition as Test Step

13 Stimulations Inputs Stimulation Equivalence Classes Timing Verifications Expected Outputs Timing MODEL TRANSITION = TEST STEP RequirementsTest Operations Select_Gear(5) Accelerate(100%) Check_Speed(185) Test Oracle Outputs = f(Inputs)

14 MATELO PROFILE Profiles can be embedded to qualify the usage model Operational profiles Test profiles (+/- 25) Data distributionUsage path probability

15 MaTeLo EDITOR Requirements Management

16 REQUIREMENTS MANAGEMENT IBM Rational DoorsMaTeLo Usage Model Association (N,M) MaTeLo Requirements Library Import Requirement Update New Baseline Update Quick Model Update Requirements Change Analysis Model Impact Highlighting Available for other ReqMgt tools with import of XML or CSV files

17 MaTeLo TESTOR Test Cases Generation

18 MaTeLo TEST STRATEGY DEFINE THE TEST STRATEGY, BY CHOOSING Test Algorithm Test Profile Part of model to test Most probable approach Start Close FREQUENCY FOCUS Start Close Risk based approach Custom Test profile CRITICALITY, COMPLEXITY UPDATE FOCUS Arcs coverage approach Close Start REQUIREMENTS COVERAGE Usage Test profile Usage approach Start Close OPERATIONAL COVERAGE Generate

19 MaTeLo TESTOR: HTML TEST PLAN Input Expected Result State Requirement Test Function Verdict

20 Example: MaTeLo FOR EXAM Test Repository SUT Interface Test Operations Stimulation Measurement Administration Sub Test Sequences Automatic Test Cases Automatic call of Test Operation Automatic parameterisation Requirements association Test Case description generation Usage model respect Association Generate Requirements Name UUID Description

21 MaTeLo TCA Test Campaign Analysis

22 TEST CAMPAIGN PROCESS Target Quality Criteria 1.Coverage 2.Reliability Confidence Indicators Bugs New Package New version New Test Cases Generation Metrics Quality/Version Operational Confidence Needs Coverage Functional Coverage

23 From Pragmatic Tools Integration To Its Standard Adoption

24 THEORETICAL MBT WORKFLOW Automation Test Framework Automation Test Framework Test Scheduler Simulator System Under Test System Under Test Execution Log Test Execution Log Usage Model Usage Model Test Cases Test Cases Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Test Scripts Test Scripts

25 Automation Test Framework Automation Test Framework A 1 ST INDUSTRIAL TOOLS INTEGRATION Usage Model Usage Model Test Cases Test Cases Simulator System Under Test System Under Test Execution Log Test Execution Log Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Scheduler Test Scripts Test Scripts EXAM UML Python dSpace MicroNova Carts Vision CAN IOs Doors MaTeLo

26 Automation Test Framework Automation Test Framework A 2 nd INDUSTRIAL TOOLS INTEGRATION Usage Model Usage Model Test Cases Test Cases Simulator System Under Test System Under Test Execution Log Test Execution Log Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Scheduler Test Scripts Test Scripts dSpace MicroNova Carts Vision CAN IOs Doors TestStand Proprietary Opal-RT NI Clemessy Diagnostic Calibration Radio Reqtify MS Excel MaTeLo EXAM UML Python

27 Automation Test Framework Automation Test Framework VARIOUS TOOLS INTEGRATION Usage Model Usage Model Test Cases Test Cases Simulator System Under Test System Under Test Execution Log Test Execution Log Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Scheduler Test Scripts Test Scripts TestStand Proprietary dSpace Opal-RT MicroNova NI Clemessy Carts Vision Diagnostic Calibration CAN IOs Radio Doors Reqtify MS Excel EXAM UML Python PROVEtech Proprietary CANoe EXACT TTCN-3 VB C, C# CSV Java Simulink CANoe Flexray Temperature XML CSV HP QC MaTeLo

28 Automation Test Framework Automation Test Framework VARIOUS TOOLS INTEGRATION Usage Model Usage Model Test Cases Test Cases Simulator System Under Test System Under Test Execution Log Test Execution Log Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Scheduler Test Scripts Test Scripts TestStand Proprietary dSpace Opal-RT MicroNova NI Clemessy Carts Vision Diagnostic Calibration CAN IOs Radio Doors Reqtify MS Excel EXAM UML Python PROVEtech Proprietary CANoe EXACT TTCN-3 VB C, C# CSV Java Simulink CANoe Flexray Temperature XML CSV HP QC MaTeLo

29 Automation Test Framework Automation Test Framework MBT AND ITS STANDARDS Usage Model Usage Model Test Cases Test Cases Simulator System Under Test System Under Test Execution Log Test Execution Log Test Cases Verdicts Test Cases Verdicts Confidence & Coverage Reports Confidence & Coverage Reports Functional Requirements Functional Requirements Atomic Test Action Atomic Test Action Test Scheduler Test Scripts Test Scripts TestStand Proprietary dSpace Opal-RT MicroNova NI Clemessy Carts Vision Diagnostic Calibration CAN IOs Radio Doors Reqtify MS Excel EXAM UML Python PROVEtech Proprietary CANoe EXACT TTCN-3 VB C, C# CSV Java Simulink CANoe Flexray Temperature XML CSV HP QC RIF ATX ODS OTX HIL API ODX FIBEX XCP GDI ATX MaTeLo

30 ItemsToolsStandard Application Feeling RequirementsDoorsRIF, ReqIF3/5 Test Specification ModelMaTeLo, Word- (DSL)- Test Cases SpecificationDoors, Word, EXAMATX, OTX0/5, 2/5 Test Campaign, ManagementMaTeLo, Quality CenterATX0/5 Test Case ImplementationEXAM, ECU TEST, TestStand, vb, python…HIL API, TTCN31/5, 2/5 Test Execution--- Physical SignalNI, dSPACE, Vector…HIL API1/5 Network SignalCAN, LIN, FlexRay…FIBEX3/5 ECU parameterCANape, INCAXCP5/5 DiagnosticCANdela,ODX, UDS5/5 Simulator SignalSimulink, AMEsim…HIL API1/5 Test Device SignalAgilent, LeCroy…GDI3/5 Test Case Execution Log, TraceNI, EXAM, dSPACE, ETAS…ODS, ATML2/5, 2/5 Test Case VerdictQuality Center, MaTeLoATX0/5 IssueBugZilla, Jira, Quality CenterISSUE? Requirement CoverageDoors, Quality Center, MaTeLoATX, ReqIF0/5, 2/5 Confidence IndicatorQuality Center, MaTeLoATX0/5

31 QUESTION ? Sales contact Wiki & Forum & Documentation

32 ABSTRACTION LAYERS Usage Scenario Description Test Flow Control Device Driver Test Flow Control Device Driver Test Case Specification System Under Test Hardware In the Loop Test Bench Python Precompiled Application Python Precompiled Application UML Sequence Diagram Markov-Chain Usage Model > MaTeLo Third Party Equipment ATX HIL API ODX, FIBEX, XCP…

33 EXAM ABSTRACTION MANAGEMENT HIL Plateform dSpace NovaSim Carts ASAM HIL API … Proprietary Network CAN_1 CAN_2 LIN_X FlexRay_1 … FlexRay_2 Functions Manual Gear Auto Gear Hand Free ACC … StartStop Test Tools CANoe CANape INCA MS Excel … Diag Tool Env. Model Gasoline Diesel Turbo Hybrid … Electric Available Test Configurations Test Case 1Test Case 2Test Case n Strategic Test Cases Possible Usage Scenarios ATX HIL API ODX, FIBEX, XCP…


Download ppt "Model-Based Black Box Testing and its Standardization Layers THE SOLUTION TO BOOST YOUR TEST EFFICIENCY MaTeLo."

Similar presentations


Ads by Google