1. Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology

2. Motivation Poor testability Ineffective testing Severe penalties

3. What is Software Testability? degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context.

4. Software Testability …... is not an intrinsic property of a software artifact …can not be measured directly (such as software size). … is an extrinsic property which results from interdependency of the software to be tested and the test goals, test methods used, and test resources (i.e., the test context)

5. Lack of testability is bad. Why? A lower degree of testability results in increased test effort. In extreme cases a lack of testability may hinder testing parts of the software or software requirements at all.

6. Background properties of the software requirements properties of the software itself (such as size, complexity and testability) properties of the test methods used properties of the development- and testing processes qualification and motivation of the persons involved in the test process

7. Testability of Software Components Controllability Observability Isolateability Separation of concerns Understandability Automatability Heterogeneity

8. Controllability The degree to which it is possible to control the state of the component under test (CUT) as required for testing.

9. Observability The degree to which it is possible to observe (intermediate and final) test results.

10. Isolateability The degree to which the component under test (CUT) can be tested in isolation.

11. Separation of concerns The degree to which the component under test has a single, well defined responsibility.

12. Understandability The degree to which the component under test is documented or self-explaining.

13. Automatability The degree to which it is possible to automate testing of the component under test.

14. Heterogeneity The degree to which the use of diverse technologies requires to use diverse test methods and tools in parallel.

15. Testability of Requirements Requirements need to fulfill the following criteria in order to be testable: –consistent –complete –unambiguous –quantitative –verifiable in practice

16. Improving software testability Test-driven development design for testability (similar to design for test in the hardware domain)

17. So which system is “testable system”? Roy Osherove: “For each logical part of the system, a unit test can be written relatively easily and quickly that satisfies all the following PC-COF rules at the same time: –Partial runs are possible –Consistent results on every test run –Configuration is unneeded before run –Order of tests does not matter –Fast run time”

18. Consistent results on every test run The test always has to fail or pass, but never switches between modes until a bug is fixed.

19. What’s the problem with this code? public bool CanPurchase(Person p){ if (!(PersonValidator.IsValid(p))){ return false; } if (p.SSID != null && p.SubscriptionType != "Canceled" && p.CreditOnFile > 0){ return true; } return false; }

20. Interfaces make code more testable IValidator m_validator; public void SetValidator(IValidator validator) { m_validator = validator; } public bool CanPurchase(Person p) { if (!(m_validator.IsValid(p))) { return false; } if (p.SSID != null && p.SubscriptionType != "Canceled" && p.CreditOnFile > 0) { return true; } return false; } MyFakeValidator val = new MyFakeValidator(); val.whatToReturn = true; PersonLogic logic = new PersonLogic(); logic.SetValidator(val); Person p = new Person(); bool result = logic.CanPurchase(p); Assert.IsFalse(result);

21. Configuration is not needed, because… … ability to configure a class at runtime is an important one for unit tests. … if code requires external configuration before it is tested, it will take more time to create tests for that code. … it can also make the tests less manageable and not as easy to write.

22. What’s the problem here? public bool IsConnectionStringValid() { string connString = ConfigurationSettings.AppSettings["conString "].ToString(); //do some stuff return true; }

23. Use virtual method Class ConnectionHelper{ public bool IsConnectionStringValid() { string connString = getConnectionString(); //do some stuff //... return true; } protected virtual string getConnectionString(){ return ConfigurationSettings.AppSettings["conString"].ToString(); }

24. Derive and override it in testable class TestableConfigBasedClass myClass = new TestableConfigBasedClass(); myClass.mConnectString = "bad string"; Assert.AreEqual(false,myClass.IsConnectionStringValid()) public class TestableConfigBasedClass:ConnectionHelper { public string mConnectString; protected override string getConnectionString(){ return mConnectString; }

25. Fast run time In a non-testable system you would find it really hard, or really time consuming to write some of the tests against objects and make them run fast. That’s because you’d most likely have objects that do some sort of time consuming activity using their external dependencies.

26. What if the IsValid method… if (!(PersonValidator.IsValid(p))){ return false; } Calls a web service to do its bidding? reads rules from the database? To make it testable : Refactor the code to use interfaces, OR use a virtual method to check the validation and return a true/false result, then override it in your “testable” class.

27. Order of tests does not matter (Data layer problem) bool Insert(Person p) { //insert person to database } bool Delete(Person p) { //Delete person from database } most unit test frameworks cannot guarantee the order in which unit tests will be executed

28. Partial Runs of tests are possible When you have external state you’re likely to have a problem if you don’t run all the tests in your suite, since the external state between specific tests may need to be adjusted for the next test. This problem is solvable, but it’s not as easy as writing simple unit tests without external state The whole problem of rolling back external state is that of controlling the before and after states and making sure they are the same for each test run. If you can’t do that easily you’re either not using a testable system design, or you’re not writing a unit test.

29. References http://www.sce.carleton.ca/faculty/yee/Improving_Testabi lity.pdf [žiūrėta 2011.04.26]http://www.sce.carleton.ca/faculty/yee/Improving_Testabi lity.pdf http://en.wikipedia.org/wiki/Software_testability [žiūrėta 2011.04.26]http://en.wikipedia.org/wiki/Software_testability http://replay.web.archive.org/20060424002143/http://web logs.asp.net/rosherove/articles/Design4Tesatbility1.aspx [žiūrėta 2011.04.26]http://replay.web.archive.org/20060424002143/http://web logs.asp.net/rosherove/articles/Design4Tesatbility1.aspx http://www.etestinghub.com/testing_requirements.php [žiūrėta 2011.04.26]http://www.etestinghub.com/testing_requirements.php