1 Debugging Integrated Systems: An Ethnographic Study of Debugging Practice Thomas Østerlie & Alf Inge Wang Debugging Integrated Systems

2 Overview Background –Debugging –Motivation –Sensemaking Research setting –Overview Gentoo –Gentoo’s formal debugging process Five characteristics of debugging practice in Gentoo –C1: Spans a variety of operating environments –C2: Collective –C3: Social –C4: Heterogeneous –C5: Ongoing Concluding remarks –Brief summary –Transferability of findings –Implications

3 Debugging: The traditional view Trace a causal chain from reported failure to its corresponding fault Access to and control over source code critical for this form of debugging

4 Motivation Increased focus on systems integration –Component-based development –Information systems and enterprise integration –Service-Oriented Architecture Debugging integrated systems –Software being integrated is developed and maintained by third- parties –Integrators have to debug systems without access to the source code of the components being integrated What do systems integrators do in practice when debugging? –Little is known about the debugging of integrated systems –Need to understand what is going on before improving on the process

5 Sensemaking Problem solving –Individual cognitive activity –Clearly defined problems and resources to solve the problem –Problem: software failure –Resources: tools and techniques for locating the fault Problem setting (Sch ö n 1991) –Problems do not present themselves as givens –Make a situation that is puzzling, troubling, uncertain into something that makes sense –Constructing problems from the materials of problematic situations Sensemaking (Weick 1995) –‘What is going on?’ A group’s collective experiences of a problems situation progressively clarified –Action precedes understanding Active engagement with the problem situation Understanding is retrospective

6 Research setting: Gentoo Community of volunteer software developers –320 official Gentoo developers –Distributed across 38 countries, 17 time zones –Develop and maintain a software system for integrating third-party OSS packages with various Unix operating systems The Gentoo software distribution – Integrator’s view –8,480 third-party packages supported –One installation script for every version of each supported package –Total of 23,900 installation scripts in package database –Total SLOC of 671,971 in package database –Supports 6 different Unix operating systems (5 processors on GNU/Linux) Gentoo installation – User’s view –Runs on a local computer –Local copy of the Gentoo package database –Uses the Gentoo package manager to integrate third-party OSS packages with the local system

7 Formal process of debugging in Gentoo

8 C1: Spans a variety of operating environments Variety of operating environments among individual Gentoo installations: –Configuration of individual packages Optionals Virtuals –Operating system –System evolution “Variation kills reproducibility” –State of individual Gentoo installations often impossible to replicate –Reproduction of reported failure therefore often impossible Problem situation, rather than clearly defined software failures

9 C2: Collective User and developers work together to make sense of the problem situation Collaborative debugging –User provides data - creates the materials of the problem situation –Developers interpret data, request new data Collaborative environment –User to developer communication: Problem reports –Developer to developer communication: IRC channel

10 C3: Social Workload Priorities –Reproducible vs. irreproducible –Curb work load vs. retaining users’ interest Responsibilities –Formal roles –Actual problem –Negotiate to bridge reality of formal roles with reality of actual problem –Determining responsibility inherent in sensemaking process DateNew reportsReports closedOpen reportsNumber of developers January 6, Not avail January 5, January 3, Not avail. January 16,

11 Exhibit: Enacting responsibilities StatementResearchers’ commentary Reporting user: I have installed my system from scratch The problem is related to the way Gentoo integrates software, and therefore the Gentoo developers' responsibility Developer A: [making reference to the systems information provided with the problem report] Is using an x86 profile for an amd64 machine troublesome? The reported problem is related to the way the user's Gentoo systems configuration; therefore the user's responsibility Developer B: [making reference to the installation script] Turning off the optional esound support might solve the problem. The problem may be related to how the package integrates with the esound package, and the third-party provider's responsibility. Developer A: [making reference to the compiler error provided with the problem report] Why is it that the thing can't find pthread? is that because of a missing - pthread The problem is related to the use of the pthreads library, and therefore the responsibility of another herd. Developer B: sounds like the glibc library was upgraded Related to the user's system configuration, and his responsibility

12 Summary of findings Debugging integrated systems is a collective sensemaking process Cyclic rather than linear Influenced by technical as well as social factors More than individual problem solving Process driven by plausibility rather than accuracy

13 Transferability of findings Presented findings to systems integrators –International network of researchers and practitioners –R&D department of a global telecom –National consultancy specializing in systems integration Similarities between commercial and volunteer context –Finding the problem is the problem –The role of organizational issues –The precarious relationship between systems integrator as provider and customers Similarities between systems integration and application development (tentative) –Finding the problem is the problem –The role of organization issues

14 Implications For research #1: Distinction corrective maintenance and debugging –Not useful when studying practice –The practice of debugging closely intertwined with the administration of the corrective maintenance process For research #2: For systems integrators the software failure is not an unproblematic phenomenon –Subject to interpretation and negotiation –What constitutes a software failure contingent upon the situation (workload, priorities, responsibilities, access to technical data) –Clear relation between error in code and observed failures too simplistic –Need better models for understanding software failures in practice Implications for practice –Difficult to determine what data is relevant prior to engaging with the problem –Comprehensive classification schemas of limited use for users reporting problems –Defect tracking systems need to support interaction between the reporting user and the system integrators debugging the problem

15 Thank you for your time. Questions?