1. Inpainting Assigment – Tips and Hints Outline how to design a good test plan selection of dimensions to test along selection of values for each dimension justification of each decision made interpretation of the test results mix between white box and black box testing How to design an efficient test plan determine the minimal number of test cases needed argue why this is sufficient replace black box by white box testing where possible

2. End User Requirements E1: Scalability What is the maximal dimension (X or Y, in pixels) of the image on which the software runs as expected? first, identify independent dimensions X (width) and Y (height), or X*Y (image area) brute-force approach: consider X,Y independent black-box test combinations of X i,Y i, with X i  {x 0, x 1, …, x n } (same for Y) use boundary-value analysis to determine x 0, x n what is the smallest possible x (i.e. x 0 ) white-box: read the article in detail black box: just try 0, 1, 2… until success what is the largest possible x (i.e. x n ) white-box: read requirements + README + assignment black-box: try sizes close/above the sample images (500,1000,…)

3. E1: Scalability (cont.) What is the maximal dimension (X or Y, in pixels) of the image on which the software runs as expected? white-box approach: are X,Y treated differently? code review: X and Y are treated identically, i.e. are present in identical computations these computations are on the same control-paths hence we have one variable only (X==Y) for this variable do black box range testing (as before) refine white-box analysis code review: search for array/buffer bounds End User Requirements

4. E1: Scalability What is the maximal image size on which the tool runs in under 5 seconds on your system? reuse the results from 1st assessment Do not test beyond the maximally accepted image size refine the question brute-force black-box approach (done by most of you) pick several image sizes run the tool, time the results two main problems with this method assumes speed is monotonic in the image size assumes the image size is the main speed parameter how do we know this is true??

5. End User Requirements E1: Scalability (cont.) What is the maximal image size on which the tool runs in under 5 seconds on your system? white-box analysis read the paper and review the computational parts of the code determine the algorithm complexity is it just O(f(N)), where N = #pixels, or O(f(N,a,b,...))? is f() a monotonic function? hints (read the paper/code): inpainting is O(N log N) where N = #scratch pixels hence speed depends on scratch area only is monotonic in the scratch area so, the optimal testing is time the tool for some reasonably large N (scratch area) compute N max for which t = 5 seconds (knowing that t = k N log N) verify/refine above N max using black box testing

6. End User Requirements E2: Input variability The software should run on BMP and PNG images identify dimensions Image format (BMP or PNG) color depth (1, 8, 16, 24, 32 bits/pixel) test the requirement black-box: OK since we don’t have many input combinations white-box (bit faster than black-box) identify image I/O code (easy) eliminate formats/depths not handled black-box test the remaining combinations

7. End User Requirements E2: Input variability (cont.) The software should run correctly for a wide range of scratch configurations identify dimensions scratch total area (?) some of you tested on that not a relevant dimension: paper/code shows clearly that the algorithm is local, so the total scratch area is irrelevant scratch local diameter – thickness? yes – it is mentioned in the assignment as a constraint bounds given: 2%..5% of the image size Scratch direction? yes – the paper clearly mentions gradient computations (and those are obviously direction-sensitive)

8. End User Requirements E2: Input variability (cont.) The software should run correctly for a wide range of scratch configurations identify dimensions (cont.) scratch position in image? yes – the paper clearly mentions neighborhood computations yes – see white-box ‘ordinary algorithm’ code reviews for-loop bounds coincide with image bounds image coordinates often involved in i+1..i-1 type of computations so we have three scratch variables local thickness orientation Position in image

9. End User Requirements E2: Input variability (cont.) Testing for the three scratch variables how many test cases (images) should I generate? generate several images, one per parameter-combination OK, but lots of work ideal for linking defects to input variables generate a few (in the limit, one) image containing a complex scratch this is OK because (recall) the inpainting is local! (so every scratch-fragment on the image acts as a small test-case..)

10. End User Requirements E3: Robustness The software should not crash or have large delays first, catalogue/eliminate the results from the previous tests second, refine the inputs/variables close to already identified crashes/delays Example (crash) Some of you found a crash when a scratch touches the lower image border black-box refinement Vary the position/angle/thickness of the scratch...so to better pinpoint the crash situation white-box refinement (code review for common coding errors) what is the crash’s cause? Out-of-bounds array indexing when does that happen? study the FIELD class...specifically the FIELD ::value(int i,int j) method

11. End User Requirements E3: Robustness (cont.) Example (long computations) white-box analysis recall the complexity O(N log N) for N = # scratch pixels white-box study (see FastMarchingMethod class and/or paper) critical operation: insertion/deletion from a sorted map map max size = scratch boundary length Insertion/deletion = O(log S), for a map with S elements hence, black-box test for very long scratches having...a relatively small area

12. End User Requirements E4: Tint preservation The inpainting should preserve the tints of the original image determine variables white-box analysis (code + paper) all images are treated as RGB triplets all computations for R, G, B are identical done on the same control paths hence, the tint variables are R, G, B note: some imaging tools use other spaces e.g. HSV, CIELab,...

13. End User Requirements E4: Tint preservation (cont.) The inpainting should preserve the tints of the original image design test cases just as for the scratch test cases can design one image per test-case can assemble several test-cases (tint-areas) in one big image recall, inpainting is local! how many test cases do we really need (how many values?) for each dimension, you have a saturation/luminance range can easily capture these in separate images, e.g....and one for green, too

14. End User Requirements E4: Tint preservation (cont.) Why don’t we need to test other tints than R, G, B? any tint is a linear combination of R, G, B if all 3 primary tints are preserved by inpainting, so is their linear combination Quantitative measuring more refined evaluation do inpainting use an image-processing tool to subtract result from original see whether tints are preserved (examine difference)

15. End User Requirements E5: Installation The software should be easily installable and run out-of-the-box on at least three platforms (e.g. Windows or Linux OS versions) identify variables trivial: platform = variable, has exactly 3 samples black-box testing install + run the software on these specific platforms use image+scratch on which software is guaranteed to run (e.g. from the sample set provided) white-box testing check the build dependencies 3rd party libraries check the code for platform-specific elements #ifdef...#endif constructs (e.g. #ifdef __WIN32)

16. End User Requirements E5: Installation (cont.) Question Is black-box testing on Windows (or Linux) 64-bit relevant? some of you used this instead of 32-bit systems however, see D1: Portability the software should compile on 32-bit OSes hence we can test on 64-bit OSes and if all runs well, this can subsume 32-bit testing if some tests fail we must test on 32-bit...or do white-box testing to further understand why