1. Introduction Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Testing

2. Introduction Nobody enjoys testing

3. TestingIntroduction Nobody enjoys testing So you can skip this lecture if

4. TestingIntroduction Nobody enjoys testing So you can skip this lecture if –your programs always work correctly, or

5. TestingIntroduction Nobody enjoys testing So you can skip this lecture if –your programs always work correctly, or –you don't care if they're correct or not, so long as their output looks plausible, and

6. TestingIntroduction Nobody enjoys testing So you can skip this lecture if –your programs always work correctly, or –you don't care if they're correct or not, so long as their output looks plausible, and –you like being inefficient

7. TestingIntroduction Nobody enjoys testing So you can skip this lecture if –your programs always work correctly, or –you don't care if they're correct or not, so long as their output looks plausible, and –you like being inefficient The more you invest in quality, the less total time it takes to build working software

8. TestingIntroduction Testing tells you:

9. TestingIntroduction Testing tells you: –if the program is doing what it's supposed to

10. TestingIntroduction Testing tells you: –if the program is doing what it's supposed to –what the program actually is supposed to do

11. TestingIntroduction Testing tells you: –if the program is doing what it's supposed to –what the program actually is supposed to do Tests are runnable specifications

12. TestingIntroduction Testing tells you: –if the program is doing what it's supposed to –what the program actually is supposed to do Tests are runnable specifications –Less likely to fall out of sync with the program than documentation

13. TestingIntroduction Quality is not just testing

14. TestingIntroduction Quality is not just testing Trying to improve the quality of software by doing more testing is like trying to lose weight by weighing yourself more often. – Steve McConnell

15. TestingIntroduction Quality is not just testing Trying to improve the quality of software by doing more testing is like trying to lose weight by weighing yourself more often. – Steve McConnell Good tests localize problems to speed up debugging

16. TestingIntroduction Testing comparison of 7-digit phone numbers

17. TestingIntroduction Testing comparison of 7-digit phone numbers 10 7 possible numbers

18. TestingIntroduction Testing comparison of 7-digit phone numbers 10 7 possible numbers (10 7 ) 2 possible pairs of numbers

19. TestingIntroduction Testing comparison of 7-digit phone numbers 10 7 possible numbers (10 7 ) 2 possible pairs of numbers At 10 6 million tests/sec, that's 155 days

20. TestingIntroduction Testing comparison of 7-digit phone numbers 10 7 possible numbers (10 7 ) 2 possible pairs of numbers At 10 6 million tests/sec, that's 155 days...and then you start testing the next function

21. TestingIntroduction How do you know that your tests are correct?

22. TestingIntroduction "All" testing can do is show that there might be a problem

23. TestingIntroduction "It might work in practice, but it'll never work in theory."

24. TestingIntroduction If testing isn't easy, people won't do it

25. TestingIntroduction If testing isn't easy, people won't do it Must be easy to:

26. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests

27. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests –understand existing tests

28. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests –understand existing tests –run tests

29. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests –understand existing tests –run tests –understand test results

30. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests –understand existing tests –run tests –understand test results And test results must be reliable

31. TestingIntroduction If testing isn't easy, people won't do it Must be easy to: –add or change tests –understand existing tests –run tests –understand test results And test results must be reliable –No false positives or false negatives

32. TestingIntroduction A unit test tests one component in a program

33. TestingIntroduction fixture A unit test tests one component in a program

34. TestingIntroduction fixture A unit test tests one component in a program What the test is run on

35. TestingIntroduction fixture action A unit test tests one component in a program

36. TestingIntroduction fixture action A unit test tests one component in a program What's done to the fixture

37. TestingIntroduction fixture action A unit test tests one component in a program expected result

38. TestingIntroduction fixture action A unit test tests one component in a program expected result What should happen

39. TestingIntroduction fixture action A unit test tests one component in a program expected result actual result

40. TestingIntroduction fixture action A unit test tests one component in a program expected result actual result What actually happened

41. TestingIntroduction fixture action A unit test tests one component in a program expected result actual result report

42. TestingIntroduction fixture action A unit test tests one component in a program expected result actual result report Summary

43. TestingIntroduction Test dna_starts_with

44. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first

45. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first False otherwise

46. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first False otherwise dna_starts_with('actggt', 'act') => True

47. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first False otherwise dna_starts_with('actggt', 'act') => True dna_starts_with('actggt', 'ctg') => False

48. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first False otherwise dna_starts_with('actggt', 'act') => True dna_starts_with('actggt', 'agt') => False Do this one from scratch to show ideas

49. TestingIntroduction Test dna_starts_with True if second argument is a prefix of the first False otherwise dna_starts_with('actggt', 'act') => True dna_starts_with('actggt', 'agt') => False Do this one from scratch to show ideas Then introduce a library that can take care of the repetitive bits

50. TestingIntroduction # Test directly assert dna_starts_with('a', 'a') assert dna_starts_with('at', 'a') assert dna_starts_with('at', 'at') assert not dna_starts_with('at', 't')

51. TestingIntroduction # Test directly assert dna_starts_with('a', 'a') assert dna_starts_with('at', 'a') assert dna_starts_with('at', 'at') assert not dna_starts_with('at', 't') This works...

52. TestingIntroduction This works......but there's a lot of repeated code... # Test directly assert dna_starts_with('a', 'a') assert dna_starts_with('at', 'a') assert dna_starts_with('at', 'at') assert not dna_starts_with('at', 't')

53. TestingIntroduction This works......but there's a lot of repeated code......and it's easy to overlook that not... # Test directly assert dna_starts_with('a', 'a') assert dna_starts_with('at', 'a') assert dna_starts_with('at', 'at') assert not dna_starts_with('at', 't')

54. TestingIntroduction This works......but there's a lot of repeated code......and it's easy to overlook that not......and it only tests up to the first failure # Test directly assert dna_starts_with('a', 'a') assert dna_starts_with('at', 'a') assert dna_starts_with('at', 'at') assert not dna_starts_with('at', 't')

55. TestingIntroduction # Tests in table # Sequence Prefix Expected Tests = [ ['a', 'a', True], ['at', 'a', True], ['at', 'at', True], ['at, 't', False] ]

56. TestingIntroduction Easy to read # Tests in table # Sequence Prefix Expected Tests = [ ['a', 'a', True], ['at', 'a', True], ['at', 'at', True], ['at, 't', False] ]

57. TestingIntroduction Easy to read Easy to add new tests # Tests in table # Sequence Prefix Expected Tests = [ ['a', 'a', True], ['at', 'a', True], ['at', 'at', True], ['at, 't', False] ]

58. TestingIntroduction # Run and report passes = 0 for (seq, prefix, expected) in Tests: if dna_starts_with(seq, prefix) == expected: passes += 1 print '%d/%d tests passed' % (passes, len(Tests))

59. TestingIntroduction No runnable code is copied when adding tests # Run and report passes = 0 for (seq, prefix, expected) in Tests: if dna_starts_with(seq, prefix) == expected: passes += 1 print '%d/%d tests passed' % (passes, len(Tests))

60. TestingIntroduction No runnable code is copied when adding tests But when tests fail, we don't know which ones # Run and report passes = 0 for (seq, prefix, expected) in Tests: if dna_starts_with(seq, prefix) == expected: passes += 1 print '%d/%d tests passed' % (passes, len(Tests))

61. TestingIntroduction # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

62. TestingIntroduction Produces (index, element) for each element of list # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

63. TestingIntroduction Decompose into variables by matching structure # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

64. TestingIntroduction Test passes as before # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

65. TestingIntroduction Summarize results that don't need attention # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

66. TestingIntroduction Report each result that needs attention separately # Run and report passes = 0 for (i, (seq, prefix, expected)) in enumerate(Tests): if dna_starts_with(seq, prefix) == expected: passes += 1 else: print 'test %d failed' % i print '%d/%d tests passed' % (passes, len(Tests))

67. TestingIntroduction This pattern is used for testing over and over

68. TestingIntroduction This pattern is used for testing over and over Many libraries to support it in many languages

69. TestingIntroduction This pattern is used for testing over and over Many libraries to support it in many languages We'll look at one that comes with Python

70. TestingIntroduction This pattern is used for testing over and over Many libraries to support it in many languages We'll look at one that comes with Python But first, we'll look at how to handle errors in programs systematically

71. July 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.