CSC 213 – Large Scale Programming

Today’s Goal  Understand why testing code is important  Result of poor or no testing & embarrassment caused  Learn basics of writing good suite of test cases  What test cases should do (& should not do)  Managing test cases to find as many bugs as possible  Make debugging easier with clear, focused tests  Analyzing tests to not waste time on useless ones  Learn types of test cases that most often used  Useful and helpful ideas from coders in real-world

Why Does Testing Matter?

Does It Work Correctly?  You have two options with all of this testing: 1. Show code works by writing working tests 2. Stop complaining & accept your punishment

What Is “Correct”?  Could check code against design being used  Verification is name for these checks  These tests are easy to create  Verification still assumes design is correct  Validation checks code does what it should

What Is “Correct”?  All roads lead back to requirements

What Is The Error?  Make sure each test method looks for 1 error  Easier debugging when you also use good names  Can you figure out when each of these tests fail? testAverageOnNullArray() testInsertFirstWhenEmpty() testPopOnEmptyStack() testOnTimeUsingTim() testDeposit2()

Critical Property of Test

Could We Find an Error?  Write tests with known, constant answers  Use constants in your assert* statements  Relying on other methods’ results propagate errors  Which would you want, if your money’s at stake? assertEquals(cut.addTwo(10), cut.addOne(11)); assertEquals(cut.addTwo(10), 12); assertEquals(cut.winner(), cut.runners[0]); assertEquals(cut.winner(), cut.getRunner(0)); assertEquals(cut.winner(), “Cheetah”);

Are We Sure There Is No Error?  Test all aspects of results to look for any errors  Should begin by looking that data returned correct  Check any fields and see if they are what is expected  Add assertions that array entries & other data okay  Problems caused when assume without full data

Are We Sure There Is No Error?  Test all aspects of results to look for any errors  Should begin by looking that data returned correct  Check any fields and see if they are what is expected  Add assertions that array entries & other data okay  Problems caused when assume without full data

Where Is The Error?  Check methods from simplest to most complex  Check easiest methods first and get them working  Once this done, test methods calling the easiest ones  Most complex methods tested last after all else works  Fixes method with bug & prevents wasting time  ONLY call method being tested to also help with this

Why Is This An Error?

How To Write Tests  Cannot prove there are no bugs on those tests  Only ever show no bugs exist on those tests  Unlikely & boundary scenarios are vital to test

How To Write Tests  Cannot prove there are no bugs on those tests  Only ever show no bugs exist on those tests  Unlikely & boundary scenarios are vital to test  Keep in mind why Willie Sutton robbed banks That’s where the money is

How To Test Boundaries  Each test should provide additional information  Why write two tests which expose same bug?  Before writing test ask: what error will this expose?  Must Include boundary checks for objects – null  Test each parameter unless pre-condition says no  Good set of tests should also check all fields used  Boundaries for arrays & Strings also very simple  Unless forbidden, should also check if these null  Lengths of 0, 1, & n also important to try

Interesting Boundary Cases  Number are much harder; no simple rules exist  To find boundaries, must look at variables used  Try negative, 0, & 1 when used in arithmetic  Check values around limit in if(…) statements public int gcd(int ap, int a, int b) { if (b == 7) { return ap; } ap = a; a = b; b = ap % a; return gcd(ap, a, b); }

Types of Loops Simple Loop Concatenated Loops Unstructured Loops Nested Loops

Types of Loops Simple Loop Concatenated Loops Unstructured Loops Nested Loops

Simple Loop  For all simple loops, try inputs that:  Skip loop entirely  Make 1 pass through the loop  Make 2 passes through the loop  Make m passes through the loop, where ( m > 2)

Simple Loop  For all simple loops, try inputs that:  Skip loop entirely  Make 1 pass through the loop  Make 2 passes through the loop  Make m passes through the loop, where ( m > 2) public int factorial(int i) { int retVal = 1; for (int j = 2; j < i; j++) { retVal *= j; } return retVal; }

Simple Loop  If loop executed at most n times, try inputs that:  Make n -1, n, & ( n +1) passes through the loop

Simple Loop  If loop executed at most n times, try inputs that:  Make n -1, n, & ( n +1) passes through the loop public boolean isPrime(int i) { for (int j = 2; j < i; j++) { if (i % j == 0) { return false; } } return true; }

Simple Loop  If loop executed at most n times, try inputs that:  Make n -1, n, & ( n +1) passes through the loop public int factorial(int i) { int retVal = 1; for (int j = 2; j < i; j++) { retVal *= j; } return retVal; }

Types of Loops Simple Loop Concatenated Loops Unstructured Loops Nested Loops

Nested Loops public sumMatrix(int[][] matrix) { int retVal = 0; for (int i = 0; i < matrix.length; i++) { for (int j = 0; j < matrix[i].length; j++){ retVal += matrix[i][j]; } } return retVal; }

Nested Loops  First test set runs all outer loops exactly once  Inner loop runs ( min +1), average, ( max -1) & max times  Then run all but two innermost loops exactly once  Inner loops run ( min +1), average, ( max -1) & max times  Tests should continue growing loop-by-loop

Nested Loops public sumMatrix(int[][] matrix) { int retVal = 0; for (int i = 0; i < matrix.length; i++) { for (int j = 0; j < matrix[i].length; j++){ retVal += matrix[i][j]; } } return retVal; }

Types of Loops Simple Loop Concatenated Loops Unstructured Loops Nested Loops

Concatenated Loops  If loops are entirely independent  No conditions, variables, or values in common  Woo-hoo! Just perform single loop tests on each  Otherwise treat as nested loops & make life easier  Work as if the first loop is outermost loops

Types of Loops

Unstructured Loops

For Next Lecture  Next weekly assignment available on Angel  Due as usual on Tuesday at 5PM  With good notes on reading can delay this due date  Reading on advanced testing techniques  Does un-testable code exist?  What do we do with this flaky, scary code?  What tricks are there to help us work with it?