1. Enhancing Configuration Management Workspaces with Information of Parallel Activities Topic Proposal Anita Sarma October 2005

2. 2 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace DCE

3. 3 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace DCE Pete and Ellen modify different artifacts with no dependencies No conflicts

4. 4 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace DCE Ellen starts to modify artifact “C” No conflicts C

5. 5 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace DCE Pete and Ellen modify entirely different files with no dependencies No conflicts C Direct conflicts Pete and Ellen concurrently modify the same artifact “C” lines 20-40 changed lines 5-10; 30- 40 changed

6. 6 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace DCE Pete and Ellen modify entirely different files with no dependencies No conflicts C Direct conflicts Pete modifies artifact “B” on which E depends B

7. 7 A Typical Development Scenario CM repository Pete’s workspace A Ellen’s workspace D Pete and Ellen modify entirely different files with no dependencies No conflicts C C Direct conflicts Pete and Ellen modify the same file BE No conflicts Direct conflicts Indirect conflicts Pete modifies “B” that affects “E”; Ellen modifies “E” in parallel signature of interface I 1 changed Changes to method body that calls interface I 1

8. 8 Goal Help mitigate the impact of direct and indirect conflicts

9. 9 Traditional CM Approaches Coordination mechanism Direct conflicts Indirect conflicts Pessimistic (RCS, DSEE) Locking before changes are made Avoided, at the expense of project delays Not addressed Optimistic (CVS, ClearCase) Automated merging after changes have been made Resolved, except for overlapping changes Not addressed

10. 10 Field Studies Perry, et al. – 1994 –Number of conflicts proportional to amount of parallel development Grinter – 1995 –Use information from CM systems to pace their development De Souza, et al. – 2003 –Send email detailing changes and their expected effects before check-in

11. 11 Inferences from Field Studies Conflicts regularly occur and affect productivity – (Perry, et al.,1994) Developer responses: –Actively obtain information from the CM systems – (Grinter,1995) –Pace their development to avoid performing conflict resolution – (Grinter,1995) –Send extra information to enable detection of effects of changes on other artifacts – (De Souza,2003) –Place their work in the context of others’ changes – (De Souza,2003)

12. 12 Inference – Informal Coordination CM repository Pete’s workspace CBA Ellen’s workspace CED Informal coordination conventions

13. 13 Hypothesis Providing information of parallel development activities and their effects enables developers to: –place their work in the context of others’ –self-coordinate their actions –reduce the magnitude and occurrence of direct and indirect conflicts

14. 14 Inference – Informal Coordination CM repository Pete’s workspace CBA Ellen’s workspace CED Informal coordination conventions

15. 15 More efficient and effective coordination Proposed Solution – Overview CM repository Pete’s workspace CBA Ellen’s workspace CED Enhanced workspace

16. 16 Proposed Solution – Context Setting Traditional CM approaches Problems – late conflict detection Proposed solution – early conflict detection CM infrastructure – Private workspaces – Well structured coordination protocols Isolated workspacesInsulated workspaces “Pull-based”“Push-based” Information only at specific synchroniza- tion points Relevant real-time information provided continuously Syntactic correctness – Pessimistic locks – Automated merge tools Overlapping direct conflict not resolved Identify and characterize direct conflicts through severity analysis Semantic correctness – Continuous builds – Integration testing Indirect conflicts undetected until build, test, or even deployment stage Identify and characterize indirect conflicts through impact analysis

17. 17 Overarching Plan of Attack Research question: Does information of parallel activities and their effects provide an effective context in which to place one’s own work? Validation: –Implicit validation from existing field studies –Evaluation results:  Usability case studies – observations on how users monitor information provided and interact with each other  Conflict detection case studies – analysis of the number and magnitude of conflicts detected in a real-life project

18. 18 Research Questions 1. How can a tool provide this information in a usable, scalable, and effective manner? 2. To what extent can this information affect self-coordination? 3. To what degree can self-coordination reduce the occurrence of conflicts? 4. To what degree can self-coordination reduce the magnitude of conflicts?

19. 19 Approach embodied by workspace awareness prototype – Palantír Approach Provide information of parallel activities continuously at real time Identify direct and indirect conflicts Provide metrics to denote the size and effect of conflicts Allow detection of conflicts earlier while changes are still in progress Build on existing CM infrastructure Present information through visualizations that are unobtrusive and contextualized Filter events based on relevance

20. 20 Palantír Architecture Visualization Extractor Internal State Palantír Client Visualization Extractor Internal State Palantír Client Event Database Palantír Server BootstrapCapture Workspace Wrapper CM System CM Server Repository The Eclipse Platform Event Listeners CM Plug-in Workspace The Eclipse Platform Event Listeners CM Plug-in Workspace Pete’s WorkspaceEllen’s Workspace

21. 21 Palantír Client – Visualizations

22. 22 Severity Analysis Severity Analysis – The amount (size) of change between two versions of an artifact –Artifact severity: the percentage of lines of code that have changed  Example: 25 lines on 100 = 25% –Directory severity: ∑Actual artifact severities ∑Possible artifact severities  Example: dirA – 12.5% dirB – 25% foo.c – 50 lines of 100 changed – 50% bar.c – 0 lines of 100 changed – 0% dirC – 0% head.c – 0 lines of 100 changed – 0%

23. 23 Impact Analysis – Remaining Research Impact of changes – “The effect of changes on my current workspace” Requirements –Identification of levels of analysis  Local workspace level  Remote workspace level  Repository level –Identification and presentation of effects of indirect conflicts  Outgoing conflicts – artifacts that affect others  Incoming conflicts – artifacts that are affected –Identification and implementation of metrics to denote the size of “effects” of a change

24. 24 Impact Analysis – Local Workspace CM repository Pete’s workspace CB A Ellen’s workspace CED Local workspace level Effect of local changes on local workspace (A → B) Pete’s view

25. 25 Impact Analysis – Remote Workspace CM repository Pete’s workspace CB A Ellen’s workspace C E D Local workspace level Remote workspace level Effect of changes in remote workspace on local workspace (E → A) Pete’s view

26. 26 Impact Analysis – Repository CM repository Pete’s workspace CBA Ellen’s workspace C E D Local workspace level Remote workspace level Repository level Effect of changes in repository on local workspace (E → A) E A Pete’s view

27. 27 Impact Analysis – Outgoing/Incoming Conflicts CM repository Pete’s workspace CB A Ellen’s workspace C E D Outgoing conflicts Incoming conflicts Outgoing conflicts – artifacts that affect others (E) Incoming conflicts – artifacts that are affected (A,B) Pete’s view

28. 28 Options for Impact Analysis Program analysis –Dependency analysis –Call graphs –Abstract syntax trees Eclipse IDE capabilities –Refactoring –Continuous build capabilities XML tags – Java-XML, Castor, Apache (XML Beans) …

29. 29 Evaluation – Criteria for Success 1. Usability study RQ#1: tool provides information in a usable, scalable, and effective manner RQ#2: information of parallel activities increases self- coordination RQ#3, #4: reduction in the occurrence and magnitude of direct and indirect conflicts 2. Conflict detection RQ#3: identification of potential direct and indirect conflicts RQ#4: calculation of magnitude of direct and indirect conflicts

30. 30 Evaluation 1 – Usability Study Five or more confederate case studies –At least one without Palantír Subjects: –Undergraduates, graduates, volunteer programmers –Role: Complete given programming assignment in teams within the time limit Confederate: –Fellow graduate students, research group members –Role: Act as a team member, but run predefined scripts that lead to conflicts in the assignment The subject is not aware of the identity and intentions of the confederate

31. 31 Usability Studies – Goals How subjects monitor and interact with Palantír Whether subjects monitor conflict warnings How subjects interact with their partners on detecting potential conflicts Identify differences (if any) in coordination efforts: –Only direct conflicts –Direct and indirect conflicts Conduct structured interviews

32. 32 Evaluation 2 – Conflict Detection Create light-weight Palantír client –Use in research group, open-source and commercial companies –Capture relevant Palantír events onsite and investigate them later offsite –Compare direct conflict detection with SCM merge conflicts –Compare indirect conflict detection with build logs, bug trackers, build managers,…

33. 33 Timeline Fall 05: Preliminary research on Impact Analysis (Oct – Dec 05) Investigate and design Impact Analysis metrics (Oct – Dec 05) Restructure Palantír events (Nov – Dec 05) Winter 06: Build Impact analysis tool / algorithm (Jan – Mar 06) Create Palantír client wrapper (Feb – Mar 06) Spring 06: Test Impact analysis tool (Apr 06) Reconstruct Palantír to be plug-in oriented (May 06) Integrate Impact analysis component with Palantír (May 06) Evaluate Palantír with Severity and Impact (Jun 06) Release Palantír client wrapper (Jun 06) Summer 06: Collect data from Palantír client wrapper (Jul – Sept 06) Create scenario for evaluation (Jul – Sept 06) Fall 06: Schedule user case studies (Jul – Sept 06) Analyze data from Client wrapper (Oct – Dec 06) Winter 07: Conduct user case studies (Jan –Apr 07) Dissertation writing (Jan – Mar 07) Spring 07: Evaluate scenario case studies (Apr – May 07) Complete dissertation (Apr – Jun 07)

34. 34 Contributions A coordination approach that … –enhances traditional CM workspaces with information of parallel activities –allows placement of one’s work in the context of others –enables early detection of potential conflicts …leads to reduction in occurrence and magnitude of direct and indirect conflicts This approach embodied by prototype – Palantír –Mechanisms for detection of potential conflicts –Metrics to denote magnitude of conflicts –Set of visualizations with varying degrees of obtrusiveness –Easy adoption into current practices

35. 35 Questions???

36. 36 Directory Severity Choices 1. Directory Severity = max {Artifact severities} –Grabs attention: max {75, 0, 0} = 75 –Multiplicity has no effect: max {75, 25, 50} = 75 2. Directory Severity = –Every artifact has an equal weight –Can degrade to zero when a majority of the artifacts have severity 0, pushing the combined severity to < 1: 3. Directory Severity = –Does not degrade to 0 with many 0 severity artifacts –Additional artifacts with smaller severity lowers directory severity: ∑Actual artifact severities ∑Possible artifact severities 1+0+0 100+100+100 ∑ severity>0 Actual artifact severities ∑ severity>0 Possible artifact severities 75 100 = 75, but 75+25+25 100+100+100 = 42 = 0

37. 37 Examples of Indirect Conflicts Interface signature changed –Return type –Input parameters Object type changed –Return type –Input parameters The behavior of the method changed Global variable setting within the method changed The order of the input parameter changed, while keeping their types constant The interface/method deleted from the class Splitting the module on which another depends into two or more parts

38. 38 Usability Case Study Set Up Programming assignment –Java programming language –10-15 classes –1.5 – 2 hour long sessions Direct conflictsIndirect conflicts – 1 (C) + 3 (S) = 4 – 2 (C) + 5 (S) = 7 – 3 (C) + 7 (S) = 10 – 2 (C) + 5 (S) = 7 – 3 (C) + 7 (S) = 10 – 3 (C) + 7 (S) + 5 (V) = 15 – 3 (C) + 7 (S) + 10 (V) = 20 Legend: C – Confederate; S – Subject; V – Virtual

39. 39 Work Thus Far Identified set of events to represent development activities Integration with three stand alone CM systems (RCS, CVS, JSVN) Workspace monitor Integration with Eclipse IDE (Eclipse-CVS, Subclipse) Set of four visualizations Severity analysis and directory severity analysis Robustness against loss of connectivity Ongoing work with project management visualization Ongoing data collection from software archives

40. 40 Future Work Use insights into workspace activity and development patterns –Workspace management –Project management Create tools to facilitate coordination across teams –Abstraction of development activities to be useful for gatekeepers Trust based CM Towards continuous coordination Coordination support for all phases of the software life cycle

41. 41 Continuous Build Systems Apache gump –Builds and compiles software against the latest version –Detects potentially incompatible changes very soon after a check-in –Notifies the project team of failed builds Tinderbox provides information of –What platforms has built successfully –Which platforms failed and how they failed –State of the files that made up the failed build

42. 42 An Alternative View Continuous coordination, collaborative architecture, seamless development environments, Asynchronous communication Access to common set of artifacts, isolated workspaces and version control Parallel development, roles and access rights Passive awareness of development activities and developers, manage information overload Task allocation and assignment Communication archival along with artifacts Communication Collocation benefits to distributed development Organizational memory, knowledge acquisition and dissemination, social navigation Prescribed and defined coordination support Artifact ManagementTask Management Advanced conflict detection Basic Functionality Rigid Process Information Discovery Information Provision Integrated Operations Fine grained versioning, conflict resolution Instant Messaging, monitoring changes to artifacts

43. 43 Back to Configuration Management Continuous coordination, collaborative architecture, seamless development environments, Asynchronous communication Access to common set of artifacts, isolated workspaces and version control Parallel development, roles and access rights Passive awareness of development activities and developers, manage information overload Task allocation and assignment Communication archival along with artifacts Communication Collocation benefits to distributed development Organizational memory, knowledge acquisition and dissemination, social navigation Prescribed and defined coordination support Artifact ManagementTask Management Advanced conflict detection Basic Functionality Rigid Process Information Discovery Information Provision Integrated Operations Fine grained versioning, conflict resolution Instant Messaging, monitoring changes to artifacts

44. 44 A Glimpse into the Future Continuous coordination, collaborative architecture, seamless development environments, Asynchronous communication Access to common set of artifacts, isolated workspaces and version control Parallel development, roles and access rights Passive awareness of development activities and developers, manage information overload Task allocation and assignment Communication archival along with artifacts Communication Collocation benefits to distributed development Organizational memory, knowledge acquisition and dissemination, social navigation Prescribed and defined coordination support Artifact ManagementTask Management Advanced conflict detection Basic Functionality Rigid Process Information Discovery Information Provision Integrated Operations Fine grained versioning, conflict resolution Instant Messaging, monitoring changes to artifacts

45. 45 Restructure Palantír Restructure Palantír events –To make them generic –To make it easier to query and parse Restructure Palantír architecture –To make it plug-in oriented –To make it more modular –To enable users to experiment with different versions of Palantír components (wrappers, analyzers, visualizations) –To encourage open source community developers to contribute

46. 46 Workspace Wrapper Tasks –Intercept workspace activity (both CM commands and local operations) –Interpret workspace activity –Determine whether the activity is relevant to Palantír –If relevant, gather information regarding the activity –Transmit the event to the Palantír server Unique per CM system Unique per editor Is and should remain unobtrusive Visualization Extractor Internal State Visualization Extractor Internal State Event Database Palantír Server BootstrapCapture Workspace Wrapper The Eclipse Platform Event Listeners CM Plug-in Workspace CM System CM Server Repository The Eclipse Platform Event Listeners CM Plug-in Workspace Palantír Client

47. 47 Palantír Server Tasks –Stores events from all workspaces –Maintains the order of the events –Bootstraps new clients and clients on a reconnect Common for all SCM systems Is unobtrusive Visualization Extractor Internal State Visualization Extractor Internal State Event Database Palantír Server BootstrapCapture Workspace Wrapper The Eclipse Platform Event Listeners CM Plug-in Workspace CM System CM Server Repository The Eclipse Platform Event Listeners CM Plug-in Workspace Palantír Client

48. 48 Palantír Client Internal State –Maintain overview of activities in both the local and remote workspaces Extractor –Allow users to configure the events to be viewed  Set of workspaces  Set of authors  Set of artifacts  … Visualizations –Organize and display events Visualization Extractor Internal State Visualization Extractor Internal State Event Database Palantír Server BootstrapCapture Workspace Wrapper The Eclipse Platform Event Listeners CM Plug-in Workspace CM System CM Server Repository The Eclipse Platform Event Listeners CM Plug-in Workspace Palantír Client

49. 49 Integration Experience Standalone wrappers –RCS (pessimistic) –CVS (optimistic) –Subversion (optimistic) Development editor – Eclipse –Eclipse-CVS –Subclipse

50. 50 Event TypeIndication Populated Artifact has been placed in a workspace Unpopulated Artifact has been removed from a workspace Synchronized Artifact has been synchronized with repository ChangesInProgress Artifact has changed in a workspace ChangesCommitted Artifact has been stored in repository ChangesReverted Artifact has been returned to its original state Added New artifact has been added to workspace Removed Artifact has been removed from workspace Renamed Artifact has been renamed Moved Artifact has been moved from one artifact to another Metrics Severity / impact of changes to an artifact List of Events

51. 51 Typical Sequence of Events Pessimistic Populated ChangesInProgress MetricsChanged … ChangesCommitted ChangesInProgress MetricsChanged … ChangesCommitted UnPopulated Optimistic Populated ChangesInProgress MetricsChanged … ChangesCommitted ChangesInProgress MetricsChanged … ChangesCommitted UnPopulated