1 ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance Case Studies Instructor Paulo Alencar

2 Overview  Investigate the relationship between pair programming and program design tasks  Isolating and measuring productivity gains of pairs in design-related tasks  The controlled experiments involved professional programmers who performed programming aptitude tasks related to problem solving and algorithm design  In two case studies pairs significantly outperformed individuals, providing evidence of the value of pairs in design-related tasks  Man, K., Chan, K., Nosek, J., The Effect of Pairs in Program Design Tasks, IEEE Transactions on Software Engineering, vol. 34, no. 2, pp , March/April 2008.

3 Main Results Research question: - Do pairs outperform individuals in program design-related tasks? In previous experiments design-related tasks were intermingled with code: - Nosek – PP is productive Full-time programmers Controlled experiment - Nawrocki and Wojciechowski – PP is not productive Fouth-year computer science students Controlled experiment

4 Main Results Measuring productivity Time productivity can be measured in two important ways: - the elapsed time to complete the task - the total effort/time of the programmer completing the task A single measure was proposed: - Relative Effort Afforded by Pairs (REAP): REAP = (finish_time_of_pair) * 2 – (finish-time_of_individual) (finish_time_of_individual) * 100 % REAP < 0 – total time of pair is less REAP = 0 – total time of pair is the same 0< REAP < 100 – pairs require more time to complete task REAP = 100 – pairs require double the time REAP > 100 – pairs require more time

5 Main Results Experiment 1 – Procedural solution tasks – -A procedural algorithm task (15 multiple-choice questions on five procedural algorithmic tasks – flowcharts) Research question: do pairs outperform individuals in procedural solution tasks? Hypothesis: - H1: There will be no statistically significant difference in performance, measured in the elapsed time required to obtain all correct answers, between pairs and individuals H2: There will be no statistically significant difference in performance, measured by the number of resubmissions required to obtain all correct answers

6 Main Results Visualization of process of Experiment 1:

7 Main Results Feedback Mechanism:

8 Main Results Results of Experiment 1

9 Main Results Time of pairs and individuals per resubmission:

10 Main Results Experiment 2 – Deductive problem solving – -Sequences of three letters or squares with numbers used as problem from which an algorithm can be devised Research question: do pairs outperform individuals in deductive problem solving? Hypothesis: - H1: There will be no statistically significant difference in performance, measured in the elapsed time between subjects divided in pairs to solve a set of deductive problems - H2: There will be no statistically significant difference in performance, measured by the number of resubmissions to solve a set of deductive problems

11 Main Results Visualization of the process of Experiment 2:

12 Main Results Results of Experiment 2

13 Main Results Time of the pairs and the individuals per resubmission: