1. Alcatel-Lucent CDC Workshop, Coaching & Knowledge Transfer Unit Testing

2. Introduction  Let’s find out WHAT a unit test is?  Convince you WHY unit testing is so important.  Showing you HOW you should create unit tests.  Best practices.

3. What?

4.  Let’s find out first what a “unit test” is.  Let’s define a clear definition, so that we all are talking about the same thing.  A gentle introduction to unit testing…

5. What is a unit?  Let’s first define what a “unit” is.  A narrow definition of “unit”: A “unit” is the “smallest testable part of an application”.  In Object Oriented design, the smallest unit is a “Class” (at first approximation).

6. What is a test?  Let’s also define what a “test” is.  A definition of “test”: A “test” is a method to verify or falsify an expectation with an observation.  A definition by wikipedia: A unit test = “a procedure used to validate that individual modules or units of source code are working properly”

7. What is a unit test?  In human language:  A unit test = “a piece of software that tests another piece of software and tells you whether the tested piece of software is working correctly”.  Let’s make sure our code is doing what it is supposed to do… …by testing it

8. What is a unit test?  In human language:  A unit test = “a piece of software that tests another piece of software and tells you whether the tested piece of software is working correctly”.  Let’s make sure our code is doing what it is supposed to do… …by testing it

9. Let’s ask Martin Fowler?  http://www.artima.com/intv/testdriven4.html “A conversation with Martin Fowler” http://www.artima.com/intv/testdriven4.html  Bill Venners: What is the unit in unit test? Define unit test.  Martin Fowler: That's very difficult. To a first approximation, it's a class. But as you work with unit tests more, you begin to realize you're testing little areas of responsibility, and that could be a part of a class or it could be several classes together. I don't get too hung up about it, but I'd say if you're just starting out, think of unit tests as just writing a test case per class.

10. A (very) simple example  1 + 1 = 2 ?

11. Parts of a unit test  SetUp  Preparation phase, where state can be initialized, data made ready,...  Test  Execution of check logic to compare expected and received results  Teardown  Restoration of the state to the previous state before the test was run

12. Assertion  Assert = Verify that the behavior is what you expect.

13. Assertion  Assert class supplies methods to aid you in testing  Validates values  Compares values  Reacts upon exceptions

14. Assertion  Assert.AreEqual(expected, actual)  Assert.AreNotEqual(expected, actual)  Assert.Fail()  Assert.Greater(value, value)  Assert.Less(value, value)  Assert.IsNull(value)  Assert.IsNotNull(value)  [ExpectedException(typeof(Exception))]

15. What kind of unit tests?  Strictly, a “unit test” tests a “unit”, which is – as seen in the definition – the “smallest testable part of an application”.  However, you would probably like to test a functionality that uses more than one class…

16. What kind of unit tests?  Typically, you have following kind of unit tests:  1. (Strict) Unit tests  2. Integration tests  3. Acceptance tests

17. What kind of unit tests?  (Strict) unit tests:  Tests “smallest testable part of an application”.

18. What kind of unit tests?  Integration tests:  Tests more than 1 thing  E.g.:  Tests which uses multiple classes.  Tests that uses a database.  …

19. What kind of unit tests?  Acceptance unit tests:  Necessary to prove that a certain required business rule is working correctly.  Functional nature.  Probably uses more “units”.  Not necessarily written up-front…  According to the analysis document:  Pre conditions  Post conditions  Normal flow  Alternate flows

20. Questions  Any questions about the “WHAT”?

21. Unit Testing in.NET

22. xUnit frameworks  Original framework was created for SmallTalk  Ported to various languages where ‘X’ stands for the language  JUnit (java), Nunit (.net), CppUnit (c++), VBUnit (visual basic), RUnit (ruby), PyUnit (python),...  Standard test architecture  NUnit: standard for.NET

23. Unit Testing in.NET  With NUnit / TestDriven.NET  With Team System  Whatever unit testing framework you are using does not matter, the principles are the same...

24. NUnit  Framework for running.NET unit tests  Uses attributes to mark classes / methods:  [TestFixture]  [Test]  [SetUp]  [TearDown]  Has GUI application to run tests

25. NUnit

26. TestDriven.NET  AddIn for Visual Studio to run Nunit tests from within the IDE  Support for code coverage  Support for debugging tests

27. With Team System  WHAT is Team System  HOW does it work  Unit Testing in Team System

28. What

30. Unit Testing in Team System  Using the Test Manager  Select tests to be run  Results are displayed in a graphical way

31. Unit Testing in Team System

32. Code Coverage in Team System  Using the Test Manager  Select tests to be run  Results are displayed in a graphical way

33. Code Coverage in Team System

34. A simple exercise  Visual Studio Solution: ItemSolutions.Calculator  The calculator can sum 2 numbers  There should be a test to prove this functionality is working.  Make a test that proves that 2 + 2 = 4

35. Questions  Any questions about “Unit testing in.NET”?

36. Why?

37. The problem  Unit tests don’t get written, because:  Most developers know that they “should” write unit tests, but they think it is cumbersome and it takes more time. So, they don’t...  Most unit tests are written after the business logic is written. Writing a test afterwards is difficult. Therefore, you can imagine that most tests don’t get written at all.  Testing of code is usually done during user testing.  Result: unit tests don’t get written at all.

38. The problem  Let’s convince you first WHY you have to write unit tests…

39. Why is unit testing so important?  Prove that your code is doing what it is supposed to do.

40. Why is unit testing so important?  Make sure you do not break other code when you develop a new functionality.

41. Why is unit testing so important?  Make sure your code integrates with the code of your colleagues.

42. Why is unit testing so important?  Make high quality code.

43. Why is unit testing so important?  Reduce cost of bug fixing.

44. Why is unit testing so important?  Sleep well at night, because you know your code is doing what it is supposed to do...

45. How? Test driven development

46.  Let’s first compare traditional development with test driven development.

47. Test driven development  Traditional development:  Write the code (“implementation”)  Test the code “manually”  Bug fixing

48. Test driven development  Test driven development (TDD):  1. Write a test first  2. Write the least code you need in order to compile  3. Run the test: it will fail (otherwise, the test is not good)  4. Implement the least code to pass the test  5. Run the test again:  If it fails, go back to 4  If it passes, start again with step 1

49. The Ugly Duckling  Following statements are mainly based on the document “The Ugly Duckling” from Martin Fowler.  An article written in 1998 (!), which explains “testing methods” and notices that it is considered as “the ugly duckling”.  http://www.martinfowler.com/distributedComputing/duck ling.pdf http://www.martinfowler.com/distributedComputing/duck ling.pdf

50. Testing methods in practice  Self-testing code  Write code that tests your code.  Testing is not something to start after you have finished coding.  Test code is as important a deliverable as production code and should be given the same emphasis.

51. Testing methods in practice  Incremental development  Every time I try to add new features to software, I stop and ask myself: “What is the smallest piece of new function I can add?”  I then focus on adding that feature alone, and I do not move to a new feature until the feature is complete - including the self-testing code for that feature.  Development then proceeds by small steps, with the testing code and the production code proceeding in tandem.

52. Testing methods in practice  Write test code before production code  It helps to focus on exactly what this incremental step involves.  Helps you concentrate on the interface for the new feature rather than the implementation.  You are asking yourself: “How will a client use this new feature?”

53. Testing methods in practice  Simple indication of test results  The tests must give a simple indication of whether they pass or fail.  “OK”  “Here is a list of failures”

54. Testing methods in practice  Quick feedback  If in the future you do something that breaks some other part of the system, the tests will quickly tell you.  You’ll know it must be something you just did.  Quickly find the bug vs. hours of bug chasing.

55. Testing methods in practice  Continuous integration  “It amazes me when a programmer checks in code after days of work and just assumes that it will work with whatever anyone else has checked in.”  Integrate a lot: check in a lot and make regular builds.  Continuous integration makes bugs show up early.

56. Testing methods in practice  “Well this sounds reasonable, but I’ve got deadlines to make”  The crucial realization is that self-testing code actually speeds up writing code. It does this because it makes debugging much shorter.  In traditional development: much more time and effort is spent in removing bugs than writing code.  Less time debugging, thus developing faster.

57. Testing methods in practice  A unit test runs without intervention  No intervention should be required to run a unit test.  Anyone should be able to run it.  A unit test must be able to run in an automated process (build).

58. A simple exercise  Visual Studio Solution: ItemSolutions.Calculator  Let’s go back to the calculator again and implement a multiplier…  Try to prove that 2 x 2 = 4  Write tests first!!!

59. Questions  Any questions about the “HOW”?

60. Quality

61. Quality of tests  Quality of code is crucial  Quality of unit tests is even more important  It’s better to have no tests, than bad tests

62. It’s all about quality…  Good unit tests allow us to produce high quality software…

63. It’s all about quality…  …but how do we follow-up?  Automated build system  Test reports  Code coverage  …

64. Automated build system  Integrate often: check in frequently  An automated build system should make a build as frequent as possible (at least once a day, “daily build” or “nightly build”).  All unit tests should be run on the production code.  Automated feedback on the unit tests should be given.

65. Test report  Used in an automated build system  Feedback on unit tests  Results are displayed in a graphical, easy to understand report  Also valuable for management

66. Test report

68. Code Coverage  Calculates coverage of unit testing  Does not monitor quality of unit tests  Does monitor what code is tested / untested  Results can be displayed in a graphical, easy to understand report  Also valuable for management  You should set a minimum code coverage percentage per project.  Be careful with this figure, it can be a misleading indicator...

69. Code Coverage

70. Quality control  How can we check that our tests are good?  Replace some functionality, by some other but broken functionality  Run the tests again  If all tests succeed then:  There might be a test missing  Or: There might be a bad test (should fail with the broken functionality)

71. Automatic quality control  Use a code mutation framework  Mutates some code and run the tests  Both for java and.NET  Jester: JUnit test tester  Nester: NUnit test tester

72. A simple exercise  Visual Studio Solution: ItemSolutions.Calculator  Let’s check the quality of our tests  Replace the sum method by the multiply method and vice versa  If any tests succeeds, the quality of the tests is poor

73. A simple exercise  Change the tests so quality can be guaranteed

74. Questions  Any questions about “QUALITY”?

75. Applied on the work floor

76.  Let’s take a look how test driven development can be applied in a real environment...

77. How  Automated build system: continuous integration cycle  Agile device: lavalamp  Project dashboards for each project  Reports  Unit test report  Code coverage report  Functional coverage report

78. How

79. Continuous integration  Automated build triggered by changes in code repository  Fast feedback  Using cruise control  Standard ANT / NANT scripts Get latest code Build itTest it Give feedback

80. Agile device: lavalamp  Shows status on all projects  Build failure  Test results (success / failure / ignored)  Coverage results (percentage)

81. Project dashboards  Graphical approach for all projects  Project information is accessible to everybody, from anywhere and at anytime  Professional look on projects for customers

82. Project dashboards

84. Questions  Any questions about “Applied on the work floor”?

85. Best practices

86.  Purpose  WHAT are the best practices  HOW can we implement them  What is GOOD and what is BAD  Does this impact on the DESIGN  EXAMPLE driven

87. Purpose of best practices  Create general conventions  Ensure quality of code but also quality of tests  Its better to have no tests than bad tests  Tests should be maintainable  Tests should be clear to everybody, not a pain in the ass

88. Best practices PC-COF

89.  Tests should comply to the PC-COF* rules:  Partial runs are possible  Consistent pass / fail result  Configuration is not needed  Order does not matter  Fast * Roy Osherove: www.iserializable.com

90. Partial runs are possible  Tests should be executable without depending on other services,...

91. Consistent pass / fail  Tests should always yield the same results  Tests should pass and fail in a predictable manner

92. Configuration is not needed  Tests should run without configuration  This is very hard to accomplish:  Database connection settings  Authentication / Authorization  Assembly dependency ...

93. Order does not matter  The order in which tests are executed should not matter for consistent results

94. Order does not matter: sample  Problem:  Insert person in a database  Delete that person from the database  Should the delete come before the insert or the insert come before the delete?  Solution:  Rollback database state after each test.  Each test should be self reliant and should not use data from other tests

95. Fast  Tests should run fast to allow people to run them often  If tests are slow, they will never be run and results will be poor

96. PC-COF: practical  Problem:  When a client opens a new bank account, its social security number needs to be validated against an authorized authority.  Link this problem to the PC-COF rule and explain  How can we solve the problem?

97. PC-COF: practical  Solution:  Extract an interface of the service (IValidator)  Create a dummy validator class implementing the interface, a stub  Add a constructor to control the result (true / false)  Replace the authorized authority by the dummy validator

98. PC-COF: practical  Can you think of any other functionality which might be hard to test?

99. PC-COF: practical  Problems:  Communication system  Workflow system  Database application

100. Best Practices What should you test

101. What should be tested  All code that performs some logic  All code that is susceptible to change  All code that may be susceptible to bugs  All code that is difficult  All code that will need refactoring  All code that will be the core of the system

102. What shouldn’t be tested  Typical entity methods: constructors, getters, setters that do not contain any logic  Temporary helpers only used for tests  Real interfaces, web services, external systems (use stubs / interfaces)

103. Test one thing at a time  Test only things related to the test you are writing  Small problems are easier to test  Large problems are easy to get stuck upon  Large wholes make tests hard to read and understand, slow and unlikely to be changed when failed

104. Test one thing at a time: example  Create new person and save it to the database  Later changes (validation / authorization) shouldn’t affect this test  Validate person before saving  Authorize user before saving

105. Acceptance tests  This kind of tests is used to test the complete flow of one part of the application  Create new person and save it to the database while validating the person and authorizing the user

106. Best Practices Design

107.  Test Driven Development may have implications on the Design of an architecture  Design should be testable, so architecture may change to allow to be tested  A testable design:  Avoid singletons  Use interfaces  Avoid GOD methods  Use factory pattern  Single responsibility for classes and methods

108. Best Practices Naming

109. Naming conventions  Names of tests should be clear at first sight  Everybody should understand the purpose without reading the test  Behaviour of tests should be clear  Test name should express a specific requirement  Test name should include the expected input or state and the expected result MethodName_Behaviour_ExpectedResult

110. Naming conventions: problem  Public int Sum(params int[] values)  Numbers larger than 1000 that are passed in should not be considered (becomes 0)

111. Naming conventions: solution TestSum_NumberBiggerThan1000TestSum_NumberIsIgnoredTestSum_NumberIgnoredIfBiggerThan1000TestSum_NumberIgnoredIfBiggerThan1000_NumberIsIgnored

112. Naming conventions: example  When a user is created without a username (null), an exception should be thrown TestCreateUserWithNullUserNameTestCreateUser_UsernameIsNull_ThrowsInvalidOperationException

113. Naming conventions @ KBC  The naming conventions are used in a slightly different way at KBC  Unit Tests: Class_Method_Behavior_ExpectedResult  Scenario Tests: Class_Step_Action_Behavior_ExpectedResult

114. Best Practices Bugfixing

115.  Write a test when a bug is found  Cover the bug in a test so it is reproducable  Fix the bug  Run the tests to make sure the bug is fixed and it cannot re-enter the system again  Result:  Certainty that you didn’t break other code you didn’t think of or from another developer  Quality and bugfree software  No more sleepless nights

116. Bugfixing: practical  Visual Studio Solution: ItemSolutions.BugFixing  Code to manage bar expenses / income  Price of drinks should be fixed  Between 16h and 18h it’s happy hour: buy 2 drinks, pay only 1  The functionality is only working from 16h till 17h  Write tests to prove this bug and to fix this bug

117. Best Practices Order of statements

118.  Order of statements and Asserts can decide upon the quality of your tests  You have to maintain a consistent state while running tests  Data can remain in the database when statements are executed in the wrong order  Asserts throw exceptions !!! when they fail  The AutoRollbackTest covers this problem

119. Order of statements: problem Delete new record Assert on new values Insert new record

120. Order of statements: solution Assert on new values Delete new record Insert new record

121. Best Practices Database testing

122.  According to the PC-COF, this is bad  External system  Susceptible to failure  Inevitable and needed  Use transaction base testing by using Auto Rollback tests  Open connection and transaction in SetUp  Rollback transaction and close connection in TearDown  Data will never remain in the database!

123. Best Practices Dates

124.  Fixed dates  When working with pre-defined dates, fixed dates can be used in tests  Calculations for fixed holidays, leap-years,...  Use new DateTime to create the fixed date  new DateTime(2007, 05, 07)

125. Dates  Variable dates (timespans)  When working with variable dates, DateTime.Now should be used  Age calculations / assertions, timespan calculations, calculations against the current system date  Without using DateTime.Now, consistent results cannot be ensured  Use current system date and add a number of days / years /... instead  DateTime.Now.AddDays(5)  DateTime.Now.AddYears(-5)

126. Dates: example  NumberOfDaysLeft = Current Date – Due Date (= calculating a timespan)  Use fixed date in test: test result will not be consistent and will change throughout time  Hard to make some consistent assertions  Use DateTime.Now in test: test result will be consistent

127. Best Practices Test data factories

128.  Some tests have to prepare lots of data in the SetUp  To avoid code duplication, a separate TestFactory class can be used  This factory can prepare the data so it is reusable

129. Test data factories: example  To test if holidays can be assigned to an employee:  An employee should be created  Days should be assigned to this employee  Holidays should be assigned  If we put this in the SetUp:  Code cannot be reused  SetUp is large, hard to maintain and read

130. Questions  Any questions about the “Best Practices”?

131. Contact Item Solutions Plantin en Moretuslei 155 2140 Borgerhout  +32 3 236 64 03  info@itemsolutions.com Speaker Kristof Rennen  kristof.rennen@itemsolutions.com