1. Agile Developers Go Testing Part II Darcy Casselman @flying_squirrel Alexei Zheglov @az1 November 2010

2. What are we talking about today? A lot of things… They may seem disconnected, but they all: –improve software quality –involve testing –done by developers None of them can do the job alone We cannot: –Talk about all of them –Give them equal time

3. Unit Testing We showed you a very simple unit test example earlier We’ll show you a more difficult example later Let’s step back first

4. Focus on Quality quotes "...in 2000, during the dot-com bubble,... software quality for North American teams... ranged from 6 defects per function point to less than 3 per 100 function point, a range of 200 to 1. The midpoint is approximately 1 defect per 0.6 to 1.0 function points. This implies that it is common for teams to spend more than 90 percent of their effort fixing defects.” http://www.amazon.com/Kanban-David-J-Anderson/dp/0984521402/ http://www.amazon.com/Software-Assessments-Benchmarks-Best-Practices/dp/0201485427/

5. Logical Connection? “The midpoint is approximately 1 defect per 0.6 to 1.0 function points. This implies that it is common for teams to spend more than 90 percent of their effort fixing defects.” How does he know what they are spending their time on?

6. Capacity Allocation To Fixing Defects

7. = Defect Injection Rate

8. Bugs, Features http://www.flickr.com/photos/dratz/1045336659/

9. Another View Root-cause analysis of established practices E.g. unit testing Five-whys applied to the solution, not the problem The link between the third and fourth whys? Source: Karl Scotland http://availagility.co.uk/2010/10/04/a-root-cause-analysis-of-agile-practices/

10. Compare Two Teams Team A Just finished 40 “features” (units of work) Introduced 10 “bugs” (units of rework) Team B 10 features 8 bugs

11. Team A Progression Terms beyond the first three are negligible Most defects eliminated in the next 2 releases The 2 nd maintenance release scope is very small Illusion that any team can do it Time spent fixing defects: 25%

12. Team B Progression The sum of the first 3 terms: only 24.4 (less than half of the total!) The sum of the first 10: approx. 45 Time spent fixing defects: 80%

13. We want to be more like Team A and certainly not like Team B (or better)

14. Testing Is what we are talking about today Leaving out other technical practices Remembering that testing itself does not provide quality

16. Test Matrix Why? UnitDid we build the code right? Does it work as specified?

17. Test Matrix Why? UnitDid we build the code right? Does it work as specified? AcceptanceDid we build the right code? Does it do what the user wants?

18. Test Matrix Why? UnitDid we build the code right? Does it work as specified? AcceptanceDid we build the right code? Does it do what the user wants? IntegrationDo system components work together? What about performance?

19. Test Matrix Why? UnitDid we build the code right? Does it work as specified? AcceptanceDid we build the right code? Does it do what the user wants? IntegrationDo system components work together? What about performance? SystemIs the system installed and configured correctly?

20. Test Matrix Why? UnitDid we build the code right? Does it work as specified? AcceptanceDid we build the right code? Does it do what the user wants? IntegrationDo system components work together? What about performance? SystemIs the system installed and configured correctly? ExploratoryWhat can we find out about the system that is not specified by these tests?

21. Test Matrix Coverage % Auto ? ToolsDevelopers’ Role Unit Acceptance Integration System Exploratory

22. Test Matrix Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/http://regulargeek.com/2009/06/12/how-much-testing-is-enough/ Coverage % Auto ? ToolsDevelopers’ Role UnitYes  NUnit/JUnit and similar tools  isolation (mocking) frameworks AcceptanceYes  acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)  domain-specific languages (DSLs) IntegrationYesUI testing frameworks (e.g. Selenium) SystemYessame ExploratoryNo-

23. Test Matrix Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/http://regulargeek.com/2009/06/12/how-much-testing-is-enough/ Coverage % Auto ? ToolsDevelopers’ Role Unit90%Yes  NUnit/JUnit and similar tools  isolation (mocking) frameworks Acceptance50%Yes  acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)  domain-specific languages (DSLs) Integration20%YesUI testing frameworks (e.g. Selenium) System5%Yessame Exploratory-No-

24. Test Matrix Sources: Robert C. Martin, Rob Diana http://regulargeek.com/2009/06/12/how-much-testing-is-enough/http://regulargeek.com/2009/06/12/how-much-testing-is-enough/ Coverage % Auto ? ToolsDevelopers’ Role Unit90%Yes  NUnit/JUnit and similar tools  isolation (mocking) frameworks take personal responsibility Acceptance50%Yes  acceptance testing frameworks (Cucumber, FitNesse, Robot etc.)  domain-specific languages (DSLs) collaborate Integration20%YesUI testing frameworks (e.g. Selenium) System5%Yessame Exploratory-No--

25. A Very Short Introduction to Acceptance Testing

26. Consider This Test Script A new customer registers The customer places three books in the shopping cart The customer goes to check-out The customer fills in their delivery address (province: Ontario) The system offers free delivery to the customer Source: Gojko Adzic. Bridging the Communication Gap, pp. 78-79 http://www.amazon.com/gp/product/0955683610

27. Compare the Test Script to This The system offers free delivery to Ontario customers when they place their first order and their order contains at least three books

28. Convert to the Tabular Form number of booksorder numbership tofree delivery? 31OntarioYes

29. number of booksorder numbership tofree delivery? 31OntarioYes 21OntarioNo

30. number of booksorder numbership tofree delivery? 31OntarioYes 21OntarioNo 32OntarioNo

31. number of booksorder numbership tofree delivery? 31OntarioYes 21OntarioNo 32OntarioNo 31AlbertaNo

32. How To Do It Collaborate: –developers –business analysts –testers –product owner Which tool to use is not as important Don’t look up terminology in Wikipedia

33. Books http://www.amazon.com/gp/product/0955683610 http://manning.com/adzic/

34. Unit-Testing When System Under Test (SUT) Has Dependencies Break the dependencies

35. What If We Don’t? Configuration Database data Remote services Permissions Time to run Infrequent errors Dependencies in development

36. Can You Run Unit Tests Like This? http://www.flickr.com/photos/theredproject/2878006012/

37. Code Example NotificationService StudentRepositoryCaller GradingEngine

38. Code Example Mock StubUnit Test System Under Test

39. (Open IDE)

40. Source http://www.osherove.com/blog/2010/9/10/2010-poll-which-isolation-framework-do-you-use-in-net.html September 2010http://www.osherove.com/blog/2010/9/10/2010-poll-which-isolation-framework-do-you-use-in-net.html

41. Two Camps Design for Testability: Moq, RhinoMocks, etc. Dynamically fake objects implementing interfaces/base classes in their own way Require software under test to be designed to be testable Programmers to follow SOLID principles (esp. Dependency Inversion Principle) Mock Anything, e.g. Typemock Isolator, JustMock Rely on Profiler API Can intercept anything and run something else instead Can fake statics, sealed classes, etc. Don’t force design for testability

42. Popular Frameworks (Java) “Conventional”: –Mockito http://code.google.com/p/mockito/http://code.google.com/p/mockito/ –EasyMock http://easymock.org/http://easymock.org/ –JMock http://www.jmock.org/http://www.jmock.org/ Alternative: –JMockit http://code.google.com/p/jmockit/http://code.google.com/p/jmockit/

43. Roy Osherove The Chief Architect at TypeMock Wrote: “A customer switches to Moq – and I’m happy” And: “This is pure awesome. This is why I come to work every day.” How is it possible? Read it: http://www.osherove.com/blog/2010/9/29/a- customer-switches-to-moq-and-im-happy.html http://www.osherove.com/blog/2010/9/29/a- customer-switches-to-moq-and-im-happy.html

44. Back to Our Code Example Grading engine operates on Student and Grade But Student has a Department And Grade has a Course and Course has a Semester We start mocking Soon, we have a lot of mocks

45. “Every time a mock returns another mock a fairy dies” Who said it? Not the guy whose tweet comes up first in Google search results I am not 100% sure What the heck does it mean anyway?

46. “Every time a mock returns another mock a fairy dies” Who said it? Not the guy whose tweet comes up first in Google search results I am not 100% sure What the heck does it mean anyway? We don’t have time for it in this presentation

47. http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627/

48. Creating Objects Constructor injection (as in our code example) Factory What if dependencies have dependencies? What if dependencies of dependencies have dependencies?

49. Pretty Soon, Things Begin To Look Like This Source: http://ayende.com/Blog/archive/2010/01/22/rejecting-dependency-injection-inversion.aspxhttp://ayende.com/Blog/archive/2010/01/22/rejecting-dependency-injection-inversion.aspx (different box color = different life span) (and this is a simple example)

50. These Tools Can Help StructureMap Castle Windsor Unity NInject autofac

51. Conclusions

52. Quality: More Important Than This http://www.flickr.com/photos/markhillary/2681349855/

53. More Conclusions You have to test No single testing strategy is nearly enough Employ multiple strategies Each of them is difficult to master We barely scratched the surface today Agile brings testing into focus Combine hard and soft skills Collaborate

54. Q&A

55. http://creativecommons.org/licenses/by-nd/3.0/