Presentation on theme: "Software Self-Testing"— Presentation transcript:
1 Software Self-Testing This work has been supported by the European Social Fund within the project «Support for Doctoral Studies at University of Latvia».Software Self-TestingEdgars DiebelisProf. Dr. sc. comp. Jānis Bičevskis
2 Self-TestingAbility to execute stored test examples to ensure that software functionality is correct.Testing approachManual testing;Automated testing with testing tools;Self-Testing.
3 Content Problem statement. Concept of Self-Testing. Implementation of Self-Testing.Comparison of the Self-Testing Concept with Conventional Testing Support Tools.Efficiency Measurements of Self-testing.Conclusions.
4 Problem Statement„Computing systems’ complexity appears to be approaching the limits of human capability”(Kephart, J., O., Chess, D., M.).IBM Autonomic Computing Manifest, 2001outlined four key features that characterise autonomic computing.Smart Technology approach, 2007identified seven types of the smart technology;self-testing is one of the type of the smart technology.
6 Self-Testing Functionality Self-testing contains two components:Test cases of system’s critical functionality to check functions, which are substantial for system using;Built-in mechanism (software component) for automated software testing (regression testing) that provides automated running of test cases and comparing test results with standard values.
7 Test storage file (XML) Test storage modeTest storage file (XML)1.2.3.
9 Test PointsTest point is a programming language command in the software text.Test point ensures that:particular actions and field values are saved when storing tests;the software execution outcome is registered when tests are executed repeatedly.By using test points, it is possible to repeat the execution of system events.
10 Test Point Types Test point types: input field test point; comparable value test point;system message test point;SQL query result test point;application event test point;test execution criteria test point;self-testing test points;etc.1010
15 Comparison of the Self-Testing Concept with Conventional Testing Support Tools
16 Testing Tools Selection Automated Testing Institute (ATI) opinion.“ATI Automation Honors ” awards.Since May 2009, the ATI has been publishing its magazine Automate Software Testing.Website has a list of 716 automated testing tools.ATI annual conference on automated testing (Verify/ATI).~8000 registred users.
17 Testing Tools TestComplete FitNesse Ranorex T-Plan Robot the best commercial automated functional testing tool in 2010FitNessethe best open source code automated functional testing tool in the NET sub- category in 2010Ranorexthe best commercial automated functional testing tool in the NET and Flash/Flex sub-categories in 2010T-Plan Robotthe best open source code automated functional testing tool in the Java sub- category in 2010Rational Functional Testerin 2009 and 2010, it was a finalist among the best commercial automated functional and performance testing toolsHP Unified Functional Testing Softwarethe best commercial automated functional testing tool in 2009Seleniumthe best open source code automated functional testing tool in 2009 and 2010
18 Criteria for Comparison Testing method (TM)Test automation approach (TAA)Test automation framework (TAF)Testing levelTest recording and playbackDesktop applications testingWeb applications testingServices testingDatabase testingTesting in production environmentSystem user can create testsSimultaneous running of several testsPerforming simultaneous actionsIdentifying the tested objectTest result analysisTest editingScreenshotsControl pointsObject validationObject browserTest logTest schedule plannerIdentification of the end of command executionPlug-ins and extensionsetc.
21 Self-Testing further development New Test automation frameworks.Test editor and log.Object validation and object browser.Load, stress and other testing levels.Web applications and services testing.Additional platforms.Plug-ins and extensions.Integration with external environment testing.
22 Self-Testing advantages White box testing.Testing in production environment.Users without in-depth IT knowledge to define and run test cases.Testing external interfaces.Perform system testing without specific preparation for running the test.
24 Efficiency Measurements Retrospective analysis of incident notifications in a real project.It is not possible to apply and compare two different concepts in the same conditions.Analysis of incident notifications (1,171 in total) in the CSAS in the period from July 2003 to 23 AugustSubjective opinion; however, the high number of incident notifications and the statistics do reflect trends.2424
25 Statistics of Incident Notifications Type of IncidentQuantity% of totalHoursDuplicate685.8123.160.47User error433.6767.461.37Unidentifiable bug17815.220.52Identifiable bug73662.8566.79Improvement1028.71241.364.89Consultation443.76293.925.96Total:11711004931.62525
26 Bugs Unidentifiable by the Self-testing Bug typeQuantity% of totalExternal interface bug52.81Computer configuration bug126.74Data type bug73.93User interface bug2514.04Simultaneous actions by usersRequirement interpretation bug4123.03Specific event8346.63Total:1781002626
27 Bugs Identifiable by the Self-testing Test pointQuantity% of totalHoursFile result test point598.02150.034.56Input field test point14619.84827.1425.11Application event test point10514.27364.2411.06Comparable value test point283.893.532.84System message test point111.4958.841.79SQL query result test point38752.5854.65Kopā:7361002727
29 Conclusionself-testing approach offers not only options equal to those offered by other globally recognized testing support tools; moreover, self-testing additionally offers options that other testing tools do not possess:testing external interfaces;testing in production environment;testing with the white-box method;possibility for users without IT knowledge to capture tests.Testing support is part of systems developed and it is available throughout the entire life cycle of software.
30 Conclusion IISelf-testing changes the testing process by considerably broaden the role of the developer in software testing.Self-testing requires additional work to include the self-testing functionality in the software and to develop critical functionality tests.Self-testing saves time by repeated (regression) testing of the existing functionality.Implementation of the self-testing functionality is useful in incremental development models, in particular in systems that are gradually improved and maintained for many years.