Introduction to Software Testing (Paul deGrandis) [Reading assignment: Chapter 15, pp and notes by Paul deGrandis]

Overview Terms and Definitions Tool Overview Static Checkers Unit Testing Coverage and Profiling Memory Analysis Bug Reporting and Tracking Continuous Integration

NOTE All the material and examples from here on are generated using a single makefile. Log into tux, then: wget mv vandvmakefile Makefilehttp:// Export your classpath: export CLASSPATH=`pwd`/junit/junit-4.4.jar:`pwd`/emma/emma.jar:. To make and download all tools and examples: make

Tool Overview Verification Static Checking Unit Tests for code correctness Memory and performance specifications Validation Unit Tests for behavior and requirements

Static Checkers Does not execute the code Finds suspicious code or security vulnerabilities Tools C - SPLINT, GCC Warnings Java - FindBugs, PMD Cyclone - a safe C GNU Compiler Collection FindBugs - Find Bugs in Java

Static Checkers - C Screenshots

Static Checkers - C Demo Installation of SPLINT Compiling using GCC Warning Running Splint

Static Checkers - C Demo Installation of SPLINT Download wget wget Untar tar -xzvf splint src.tgz Make (during make, splint checks itself) cd splint-3.1.2/./configure make

Static Checkers - C Demo Compiling using GCC Warnings see the screenshot three slides back

Static Checkers - C Demo Running SPLINT Simple and Basic splint -preproc +weak +show-summary +stats../src/*.c Dependable Software splint -preproc +checks +show-summary +stats../src/*.c

Static Checkers - Java Screenshots

Static Checkers - Java Demo Installation of FindBugs Running FindBugs

Static Checkers - Java Demo Installation of FindBugs Download wget Untar tar -xvzf findbugs tar.gz cd findbugs-1.2.1

Static Checkers - Java Demo Running FindBugs Run It./findbugs

Unit Tests Validate a unit of code; smallest testable part Executes the code in a sandboxed environment Testing mostly for functional requirements can also test some non-functional requirements Many approaches and schools of thought History Based, Risk Based, Data Path, DOE

Unit Tests Frameworks-C Screenshots Check Unit Test Framework

Demo Unit Tests Frameworks- C Installation of Check Writing a Check unit test Compiling your test(s) Running your test(s)

Demo Unit Tests Frameworks- C Installation of Check Download wget Untar tar -xvzf check tar.gz Make (notice the GCC warnings passed) cd check /configure make

Demo Unit Tests Frameworks- C Writing a Check unit test Write your test fixtures Write your test(s) Write your test suite Write your main method (execute your suite)

Demo Unit Tests Frameworks- C Writing a Check unit test Write your test fixtures

Demo Unit Tests Frameworks- C Writing a Check unit test Write your test(s)

Demo Unit Tests Frameworks- C Writing a Check unit test Write your test suite

Demo Unit Tests Frameworks- C Writing a Check unit test Write your main method (execute your suite)

Demo Unit Tests Frameworks- C Compiling your test(s) See README in check/check- example/square

Demo Unit Tests Frameworks- C Running your test(s) See README in check/check- example/square

Unit Tests Frameworks-Java Screenshots

Demo Unit Tests Frameworks- Java Installation of Junit Writing a Junit unit test Compiling your test(s) Just use javac Running your test(s) just use java (look at the cookbook)

Demo Unit Tests Frameworks- Java Installation of Junit Download wget Add the jar to your classpath export CLASSPATH=$CLASSPATH:`pwd`/junit-4.4.jar:.

Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test fixtures Write your test(s) Write your test suite Write your main method (execute your suite)

Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test fixtures

Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test(s)

Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test suite(s)

Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your main method

Code Coverage Degree to how much code was tested (E.g., How confident are we in our tests and code) Criteria: Statement - Has each line been executed Condition - Every evaluation (if/else, try/catch, switch, loop) Path - Every route of code

Code Coverage - C Screenshots GCC - GCOV

Demo Code Coverage - C See the Screenshot one slide back

Code Coverage - Java Screenshots EMMA

Demo Code Coverage - Java Installation of Emma Download wget Unzip unzip emma lib.zip Instrument Your Code java -cp emma.jar:./sample:../junit/junit-4.4.jar emma instr -outdir./sampleout -cp./sample/ SimpleTest Run Your Code and Generate a report java SimpleTest java -cp emma.jar emma report -r html -in sampleout/coverage.em,sampleout/coverage.ec

Memory Analysis Sandboxes an application in a VM Uses: Memory Leak - allocate memory, but don’t free Buffer Overflow - access memory outside of a fixed buffer Profiling - performance analysis

Memory Analysis - C Screenshots Valgrind

Demo Memory Analysis - C Installation of Valgrind Download wget Untar tar -xvjf valgrind tar.bz2 Make (notice the GCC warnings passed) cd valgrind /configure make

Demo Memory Analysis - C Running of Valgrind Compile with debugging enabled gcc -g myprog.c Run Valgrind valgrind --leak-check=yes myprog arg1 arg2 Full tutorial and infoformation

Memory Analysis - Java Screenshots JConsole

Issue and Bug Tracking Organizes and tracks issues, enhancements, features, and bugs in a software system Usually contains a database backend and a web frontend Generate report of project and team metrics and productivity

Issue and Bug Tracking Trac ( ) Roundup ( ) Tracker ( )

Continuous Integration Committing changes early and often Building and testing at least nightly If code breaks, revert Integration issues discovered early as developers work with the latest code

You now know … … static checkers … unit testing tools … coverage and profiling tools … memory analysis tools … Bug reporting and tracking tools