28/06/2015Dr Andy Brooks1 MSc Software Maintenance MS Viðhald hugbúnaðar Fyrirlestrar 25 & 26 The Importance of Typographic Style “Hence, the book format is a more appropriate form for representing program source code.” typographic/prentlegur

28/06/2015Dr Andy Brooks2 Case Study Dæmisaga Reference Typographic Style is More than Cosmetic, Paul W. Oman and Curtis R. Cook, Communications of the ACM, Volume 33, Number 5, pp , © ACM “... about one half of a maintenance programmer´s time is spent studying source code and related documentation.” “Unfortunately, in many instances the only reliable description of a program is the source code itself.”

Macro-typographic principles Code areas for global definitions, the main program, support routines, and included code segments should be easily identified. Module separation should be obvious. –module separation conventions The purpose and use of each component should be identified. –intermodule commenting The execution control and information flow between components should be clear. The program should be readable and different access paths available when browsing. 28/06/2015Dr Andy Brooks3

Micro-typographic principles Code areas for constants, data declarations, and code body should be easily identified within a module. The purpose and use of each section within a module should be identified. –intramodule commenting The execution control and information flow within a module should be clear. Statements should be readable and easily scanned. –statements written as sentences White space to separate groups of statements. –use of blank lines 28/06/2015Dr Andy Brooks4

Typographic principles and factors The principles can be implemented with regard to several typographic factors: –commenting –naming –blank lines –type and font usage –CharacterCase etc. 28/06/2015Dr Andy Brooks5

Book Format Paradigm Preface Table of contents Indices Pagination Chapters Sections Paragraphs Sentences Punctuation Type style Character case 28/06/2015Dr Andy Brooks6

28/06/2015Dr Andy Brooks7 Figure 1. Pages from a Book Format Listing ©ACM Preface access paths

28/06/2015Dr Andy Brooks8 Figure 1. Pages from a Book Format Listing ©ACM Index Called from: Calls to: access paths

28/06/2015Dr Andy Brooks9 Figure 1. Pages from a Book Format Listing ©ACM horizontal sentences comment indicating end of code section italics indentation CharacterCase

Experiment 1: Testing Macro-Typographic Principles Two versions of a line-oriented text editor written in Pascal were created: –traditional listing –book format listing table of contents, chapters, and module index The Skip_Blanks procedure and five calls to it were removed from the original 1,532-line program. Procedures unrelated to command parsing were also removed to reduce the original program to 1,011 lines. –“a size that could be managed by student programmers in one hour”. 28/06/2015Dr Andy Brooks10

Subjects/Þátttakendur 53 senior and graduate level computer science students. Subjects randomly assigned into two groups. –28 subjects received program version 1 –25 subjects received program version 2 (book format) The same instructions were given to everybody. No special instructions or explanations for the book format listing. –“this was deliberately done to test whether subjects could naturally use the book format listing” 28/06/2015Dr Andy Brooks11 Experiment 1

Maintenance Task “Write a Skip_Blanks procedure that would enable free-form command inputs, and indicate where (on the listing) the procedure would be called.” Subjects needed to understand the command line structure and the execution flow of the routines that manipulated the command line. Subjects were given 55 minutes. 28/06/2015Dr Andy Brooks12 Experiment 1

Results ©ACM Regarding a correct Skip_Blanks routine, book format listing outperforms traditional listing by almost 2:1 (13:7). Twice as many traditional listing subjects gave up or did not finish (11:4). 28/06/2015Dr Andy Brooks13 Experiment 1

Chi-square tests/Kí-kvaðrat próf Pearson chi-square = p < 0.025, d.f. = 3 (degrees of freedom, frígráður) Pearson chi-square = 4.09 p < 0.05, d.f. = 1 28/06/2015Dr Andy Brooks14 Experiment 1 Trad Book9484 Trad721 Book1312 p < something is not purely random reject the null hypothesis

Data only for subjects who had a correct Skip_Blanks procedure. Percentage accuracy is greater for book format (47.6% > 34.2%). –but no statistical test reported 28/06/2015Dr Andy Brooks15 Experiment 1 Results ©ACM 5 calls

Results & Discussion No difference found in average times. –Version 1, 53.5 minutes –Version 2, 52.2 minutes Macro-typographic style improved programmer performance in terms of accuracy but not time. –2:1 ratio for a correct Skip_Blanks procedure “... book format listing performed better without any explanation, description, or justification of the book format listing.” 28/06/2015Dr Andy Brooks16 Experiment 1

Experiment 2: Testing Micro-Typographic Principles Two Pascal procedures (94 LOC) from a toolbox: –traditional listing in the manner of Lightspeed Pascal –book format listing “section headings were highlighted, sections and control constructs were separated by blank lines, statements were written as sentences when possible, procedure calls were highlighted, and related clauses were aligned and/or chunked together” 28/06/2015Dr Andy Brooks17

Figure 2. Experiment 2: Pascal Code Excerpts ©ACM 28/06/2015Dr Andy Brooks18

Subjects/Þátttakendur 36 intermediate computer science students. Subjects randomly assigned into two groups. –18 subjects received program version 1 –18 subjects received program version 2 (book format) The same instructions were given to everybody. 28/06/2015Dr Andy Brooks19 Experiment 2

Short Comprehension Test Ten multiple choice and short answer questions – altogether 14 answers. Subjects were given ten minutes to answer the questions. Subjects were also asked to rate the readability of the code. –1 very poor –5 very good 28/06/2015Dr Andy Brooks20 Experiment 2

Results ©ACM score/time helps deal with the time/accuracy tradeoff 28/06/2015Dr Andy Brooks21 Experiment 2 maximum test score 14

Results & Discussion No statistically significant difference found in average times. –Version 1, 9.31 minutes –Version 2, 8.90 minutes Micro-typographic style improved programmer comprehension in terms of accuracy but not time. –Using the book format listing resulted in 3 more correct answers on average. Subjects rated higher the readability of the book format listing. 28/06/2015Dr Andy Brooks22 Experiment 2 Andy asks: What were the questions?

Experiment 3: Testing Micro-Typographic Principles in C A reverse-Polish desk calculator program written in C. –traditional listing taken from Kernighan and Ritchie´s book “The C Programming Language”. –book format listing “section headings were highlighted, sections and control constructs were separated by blank lines, statements were written as sentences when possible, procedure calls were highlighted, and related clauses were aligned and/or chunked together” 28/06/2015Dr Andy Brooks23 testing language independence

Figure 4. Experiment 3: C Code Excerpts ©ACM 28/06/2015Dr Andy Brooks24

Subjects/Þátttakendur 44 advanced computer science students. Subjects randomly assigned into two groups. –22 subjects received program version 1 –22 subjects received program version 2 (book format) The same instructions were given to everybody. 28/06/2015Dr Andy Brooks25 Experiment 3 Short Comprehension Test Nine questions, one with a two-part answer, resulting in a total of 10 answers.

Results ©ACM 28/06/2015Dr Andy Brooks26 Experiment 3 maximum test score 10

Results & Discussion No statistically significant difference found in average times. –Version 1, 9.52 minutes –Version 2, 8.84 minutes Micro-typographic style improved programmer comprehension in terms of accuracy but not time. –Using the book format listing resulted in 1 more correct answer on average. Subjects did not rate higher the readability of the book format listing, perhaps because subjects knew the traditional listing was from the C book by respected authors Kernighan and Ritchie. 28/06/2015Dr Andy Brooks27 Experiment 3 Andy asks: What were the questions?

Experiment 4: Complete Book Format Listing A portion of the X_Windows package: 1,057 lines of C code. –traditional listing (laser printed with pagination) –complete book format listing The table of contents and index added 269 lines of comments, but the micro-typographic rearrangement meant the complete book format listing was only 41 lines longer than the original code. 28/06/2015Dr Andy Brooks28

Subjects/Þátttakendur 12 professional programmers with at least two years of C programming experience. Subjects were paired by experience and job function, creating 6 pairs of subjects. –6 subjects received program version 1 –6 subjects (the other members of the pairs) received program version 2 (book format) –within a pair, assignment was random (coin flip) Two subjects, highly experienced with the X_Windows system, helped establish expert performance levels. –subjects Xt and Xb (traditional and book) 28/06/2015Dr Andy Brooks29 Experiment 4 Andy says: good.

Tasks 30 minutes of think-aloud in which the task was to become familiar with the program. Oral comprehension test –time up to 30 minutes –accuracy (maximum score 10) Complete an incomplete call graph –time up to 30 minutes –accuracy (maximum score 39) one point for each missing node (11) and edge (28) The same instructions were given to everybody. 28/06/2015Dr Andy Brooks30 Experiment 4 Andy asks: What were the questions?

Results ©ACM 28/06/2015Dr Andy Brooks31 Experiment 4 maximum test score 10 t traditional b book format

Comprehension Test Results Small sample sizes of 6 and 6, so no statistical tests were performed. For each pair of subjects, the subject working with the book format listing had better accuracy and took less time than the subject working with the traditional listing. The subjects working with the book format listing performed well compared to the two experts. 28/06/2015Dr Andy Brooks32 Experiment 4

Results ©ACM 28/06/2015Dr Andy Brooks33 Experiment 4 maximum score 39 t traditional b book format

Call Graph Results Small sample sizes of 6 and 6, so no statistical tests were performed. For four pairs of subjects, the subject working with the book format listing had better accuracy and took less time than the subject working with the traditional listing. Et presented exceptional behaviour. Et worked the full 30 minutes tracing every procedure call. The subjects working with the book format listing performed well compared to the two experts. 28/06/2015Dr Andy Brooks34 Experiment 4

Imperfect experimental design/ Ófullkomið tilraunasnið 28/06/2015Dr Andy Brooks35 Experiment 4 “Before starting the exercise, five of the six subjects working with the version 2 listing indicated that they could use the index to complete the call graph without looking at the code. They were told it was an exercise in code reading and they were to build the call graph from the code, not the index. They were permitted to use the index and table of contents only to find modules when tracing the execution of the code they were reading.” Andy says: experiment 4 is flawed. The book format listing had all the necessary calling information in the index.

Andy´s View In Experiment 1, the Pearson chi-square test was used. This test is approximate. Nowadays, Fisher´s exact test would used and the resulting p value may not be less than –see In Experiments 2, 3, and 4, without knowledge of the questions actually asked in the comprehension tests, it is difficult to form an opinion about how useful the book format listing really is. I prefer experiments where subjects actually perform a maintenance task rather than answer a comprehension test. The call graph exercise of Experiment 4 was flawed. The “statements as sentences” principle is interesting and I personally prefer this arrangement of program text. 28/06/2015Dr Andy Brooks36