2. xUnit Test Patterns

3. (Some) xUnit Test Patterns (in practice) by Adam Czepil

4. Who needs these? What kind of problems xUnit test patterns may help to solve? Who will benefit the most?

5. Goals of Test Automation Code Smells Principles of Test Automation Behavior Smells Projects Smells Test Patterns

6. Is it a fish?

7. Or is it a mammal?

8. Revenons a nos moutons!

9. What is Defect Localization and why developers in good projects (almost) dont use debugger? Debugging is an art

10. The problem

11. We run the test (possibly manual test)

12. The problem We run the test (possibly manual test) The test fails

13. The problem We run the test (possibly manual test) The test fails We have to figure out why it fails?

14. The problem We run the test (possibly manual test) The test fails We have to figure out why it fails? The problem is that this can be very time consuming...

15. Fortunatelly we have extra-skilled developers...... who can find the problem very quickly ;-)

16. What we expect from very good tests is that

17. if the tests fail they provide information

18. What we expect from very good tests is that if the tests fail they provide information detailed enough so we can find the source of the failure quickly.

19. Defect Localization ability to very quickly find the actual cause of test failure

20. Defect Localization is

21. one of the major goals of test automation!

22. Defect Localization is

23. one of the major advantages of unit tests over customer (functional) tests!

24. What to do to have good Defect Localization?

25. Frequent Debugging

26. When we deal with Frequent Debugging?

27. When manual debugging is required to determine the cause of most test failures.

28. When we deal with Frequent Debugging? When manual debugging is required to determine the cause of most test failures. This is also true when instead of debugging we use extensive logging during testing.

29. Frequent Debugging...

30. reduces productivity!

31. Frequent Debugging... reduces productivity! and makes development schedules much less predictable!

32. Frequent Debugging... reduces productivity! and makes development schedules much less predictable! because a single manual debugging session could extend the time required to develop the software by half a day or more.

33. What are the causes of Frequent Debugging?

34. Frequent Debugging is caused by

35. What are the causes of Frequent Debugging? Frequent Debugging is caused by missing tests or

36. What are the causes of Frequent Debugging? Frequent Debugging is caused by missing tests or lack of Defect Localization in our suite of automated tests.

37. We should be able to quickly determine the root cause of test failure based on

38. We should be able to quickly determine the root cause of test failure based on Assertion Message

39. We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures

40. We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures and knowledge about changes being made.

41. What are the main causes for no Defect Localization?

42. not enough detailed unit tests compared to functional tests,

43. What are the main causes for no Defect Localization? not enough detailed unit tests compared to functional tests, missing component tests (for a cluster of classes) that would point out an integration error between classes.

44. What are the main causes for no Defect Localization? not enough detailed unit tests compared to functional tests, missing component tests (for a cluster of classes) that would point out an integration error between classes, Infrequently Run Tests, because it causes that it is difficult to point out the change that caused the tests to fail.

45. Very good solution for Frequent Debugging problem is...

46. TDD – Test Driven Development

47. Very good solution for Frequent Debugging problem is... TDD – Test Driven Development (or at least Test First)

48. What I want you to remember?

49. Defect Localization is about finding quickly bug causing the tests to fail.

50. What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests.

51. What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests. Frequent Debugging usually means that project is either missing tests or has poor Defect Localization.

52. What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests. Frequent Debugging usually means that project is either missing tests or has poor Defect Localization. Use TDD as often as possible

53. WARNING! Too much debugging may broke your eyes! ;-)

54. Questions?

55. How NOT to automate manual tests? What are some differences between manual and automated test-case? How to create automated test comparing to manual test?

56. Manual vs. Automated Tests

57. Lets compare some aspects...

58. Manual vs. Automated Tests Lets compare some aspects... -time

59. Manual vs. Automated Tests Lets compare some aspects... -time, -fragility.

60. Time Executing manual tests requires A LOT OF time from HUMAN

61. Time Executing automated tests requires almost NO time from HUMAN

62. Since HUMAN time is precious and COMPUTER time is cheap we should care much about the first and dont have to worry too much about the latter.

63. Conclusions Opposite to manual tests we can create unlimited number of regression test cases.

64. Fragility When the test fails/stops in some non-final point HUMAN usually finds a way (work around) to continue its execution.

65. Fragility When the test fails in some non-final point test execution STOPS.

66. HUMAN, although much slower than COMPUTER, is smarter test executor, especially when the test fails before all test conditions have been checked.

67. Conclusions Opposite to manual tests we have to care about assertions order and find a way of handling not executed assertions.

68. Verify One Condition per Test

69. Verify One Condition per Test principle is not the same as One Assertion Per Test principle

70. Lets see an example... exampleexample refactored

71. When (not) to use setup (@Before) and teardown (@After) in xUnit?

72. Verify One Condition per Test helps to give us good Defect Localization to pinpoint the source of the failure.

73. Another reason we Verify One Condition per Test is to Minimize Test Overlap

74. Another reason we Verify One Condition per Test is to Minimize Test Overlap, so that we have fewer tests to modify if we later modify the behavior of the SUT.

75. So? How NOT to automate manual tests?

76. Dont write long, complex test scenarios

77. So? How NOT to automate manual tests? Dont write long, complex test scenarios, instead Verify One Condition per Test

78. So? How NOT to automate manual tests? Dont write long, complex test scenarios, instead Verify One Condition per Test, but not necesserily one assertion per test.

79. Questions?

80. Hey! Test code is code too! Is test code different than regular code? How to organize test cases into classes/files/directories? What about test utils/helpers? Where to keep them?

81. Questions?

82. There is a lot of patterns described in the book. Here are maps of them...

83. The Maps

87. Why its worthwile to study xUnit patterns?

88. Questions?

89. Thank you! Go there: http://xunitpatterns.com/http://xunitpatterns.com/ Really! You dont have to read everything there...