1. 2IS95 Seminar Software Engineering and Technology
Software Evolution

2. Software Evolution. In the beginning.
Royce 1970, “Waterfall model”.
Maintenance = bug fixes and minor adjustments
What are the main shortcomings of this model, in your opinion?
Maintenance is restricted to bug fixes and minor adjustments, i.e., the model does not reflect more significant modifications after the initial development phase.
NB: Still in use anno 2009!
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

3. 1974. Lehman’s laws. E-type system: Evolution laws:
Systems operate in the real world (or address it)
They should be evolved since the real world evolves!
Evolution laws:
E-type systems must be continually adapted else they become progressively less satisfactory.
As an E-type system evolves its complexity increases unless work is done to maintain or reduce it.
The quality of E-type systems will appear to be declining unless they are rigorously maintained and adapted to operational environment changes.
There are more…
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

4. Macro look: Bennett and Rajlich (2000)
architecture decay
Loss of
architectural
integrity
Patches
too costly?
As early as possible, by preference during the servicing stage.
Legacy system = the system that is still valuable despite its degraded quality.
Migration = keep functionality, improve quality!
At what stage would you like to plan migration?
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

5. Micro look: Yau et al. (1978)
“Evolution” stage of the Bennett-Rajlich model
Refactoring = keep functionality, improve structure!
What is the difference between refactoring and restructuring as defined in the Yau model?
Restructuring is seen as a part of the change implementation while refactoring can be carried out, e.g., to improve readability without implementing new changes
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

6. Going extreme Software evolution (iterative, incremental) +
short time periods
high degree of collaboration
explicit accommodation of changing needs
agile software development
Extreme programming is an example of agile.
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

7. Software Evolution: Definition
Look at the discussion so far.
How would you define “software evolution”?
Software evolution is the set of activities, both technical and managerial, that ensures that software continues to meet organisational and business objectives in a cost effective way.
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

8. Where do we start? Versioning repositories (CVS, SVN, …)
Software but what about models, requirements, etc?
Bug tracking systems
Change logs
Documentation
Mailing list archives
Huge amount of information
Tooling is essential!
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

9. Research Themes in Software Evolution: Science
What is the nature of software evolution?
Lehman’s law:
Evolution = a multi-loop, multi-level and multi-agent feedback system
Understanding the evolution requires understanding
Human psychology
Social interaction
Organizational aspects
Economics and management
Legislation
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

10. Linchpin developers create a social network!
Social interaction?
Open source
Linchpin developers create a social network!
Madey, Freeh and Tynan 2002
/ SET / W&I

11. Research Themes in Software Evolution: Engineering
Technology, means and tools to direct, implement and control software evolution.
Some relevant questions:
What artefacts evolve?
Requirements, design, code, tests, …
What can we learn from evolution so far?
Quality change, software analysis
How evolvable are our artefacts?
How can we make them more evolvable?
Migration, reverse engineering and reengineering
How is the evaluation implemented?
Refactoring, architecture transformations …
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

12. What does evolve?
Requirements evolution
Design (architecture) evolution.
Data, code, documents, technology evolution
Evolution of different artefacts should be consistent.
This is called the co-evolution problem.
Tests and proofs evolution
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

13. Consistency and co-evolution: Points of discussion
What should co-evolve?
Code and database table definitions
Code and design documentation
Code and tests
Different code elements (packages, modules, files…) …
What constitutes inconsistency?
How to detect inconsistencies?
How to ensure absence of inconsistencies?
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

14. Co-evolution is time-dependent!
Assumption: files committed together are co-evolving.
D’Ambros, Gall, Lanza, and Pinzger. “Analysing Software Repositories to Understand Software Evolution”
/ SET / W&I

15. Source/Test Files Co-evolution
Andy Zaidman, Bart Van Rompaey, Serge Demeyer and Arie van Deursen. “Mining Software Repositories to Study Co-Evolution of Production & Test Code”
At the very beginning of the project up until commit #280, there is only one test (with file ID 20), which is changed very frequently (visible through the yellow horizontal bar). At that point, a number of new tests are introduced.
From commit #440 onwards, a new testing strategy is followed, whereby the introduction of new production code (a red square) almost always entails the immediate addition of a new unit test (a green triangle).
From #670 onwards, integration tests appear (visible by the yellow diamonds at the top of the chart). This commit is also interesting because it shows a vertical yellow bar, indicating that a large number of unit tests are modified, suggesting that several of the unit test files are affected by the adoption of integration tests. This pattern returns around commit #780.
Furthermore, around #870 and #1375 test additions can be seen through the vertical bar of green triangles. Due to the massive number of unit tests involved this might indicate (i) a “phased testing approach”, where an increased test effort is taking place at certain points in time (with little or no testing in between); or
(ii) shallow changes to the test code (e.g., import-optimization).
Added Source Modified Source Added Test Modified Test
/ SET / W&I

16. What can we learn from the evolution so far?
What modules are stable?
How to predict bugs from history?
How to detect the architecture decay?
What can we say about the project organisation?
Is there a major developer? (risk if she leaves)
Is there correlation between #bugs, productivity and certain developers/teams?
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

17. Is there a major developer? Fractal Figures!
Fractal value (between 0 and 1)
Area
Quantative measure: between 0 and smaller than 1, due to the square.
0 = 1 developer
1 = many small developers
nc(X) = number of commits performed by author X
D’Ambros, M., Lanza, M., Gall, H.: Fractal figures: Visualizing development effort for CVS entities.
/ Mathematics and Computer Science

18. Work assignment in an industrial project
Package
Module
File
Number of Modification Requests (MRs) per file 1 30
Some modules have many red(dish) files
/ SET / W&I
Courtesy of Alexandru Telea

19. The “reddish” modules: Team assignment
Many strategic/problematic components (70%) are outsourced (to India – Team A). This team is responsible for many MRs!
Team A
Team B
7 of the 11 red(ish) modules are assigned to the red team
Team C
/ SET / W&I
Courtesy of Alexandru Telea

20. Problems in the development?
Time
MR id range (4000 – 5000; grouped on hundreds)
Ex: Number of file commits referring to MRs with IDs in the range [ )
In mid 2008, activity related to MRs addressed in still takes place
/ SET / W&I
Courtesy of Alexandru Telea

21. How do systems evolve? Evolution patterns!
4 open-source systems
Nakakoji et al. 2002
/ SET / W&I

22. How evolvable are our artefacts? Analysis!
Resilience to change caused by
Bad readability
Bad structure
Code clones (see next slide)
High complexity
Insufficient abstractness (OO) …
How to detect these situations?
Metrics (discussed earlier)
Dependencies
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

23. Special technique: Code clones elimination
Presence of code clones impedes evolution.
Why?
Points of discussion:
When A and B can be considered clones?
Identical
Up to variable/constant/number renaming
Allowing gaps/repetitions
How to detect clones?
When is it beneficial to remove the clones?
Why do people clone code?
Propagation of change
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

24. Code duplication Why? [Kim et al. 2004] How? [Kapser, Godfrey 2006]
Limitations of the programming language
Inability to predict future variability
Reuse of the existing code
Struggle against bad(?) modularization
Cross-cutting concerns and aspects
How? [Kapser, Godfrey 2006]
Forking
Templating
Customization
Ethnographic study (i.e., they observed the programmers at work)
Forking is cloning used to bootstrap development of similar solutions, with the expectation that evolution of the code will occur somewhat independently, at least in the short term. The assumption is that the copied code takes a separate evolution path independent of the original. In such a case, changes in the copy may be made that have no side effect on the original code.
Templating is used as a method to directly copy behavior of existing code but appropriate
abstraction mechanisms are unavailable. It was also identified as a main driver for cloning in Kim and Notkin’s case study [277]. Templating is often found when a reused library has a relatively fixed protocol (that is, a required order of using its interface items) which manifests as laying out the control flow of the interface items as a fixed pattern. For instance, the code in Fig. 2.1 uses a fixed iteration scheme for variable iter.
Customization occurs when currently existing code does not adequately meet a new
set of requirements. The existing code is cloned and tailored to solve this new
problem.
/ SET / W&I

25. How to make artifacts more evolvable?
Evolution strategies
Refactor
Reengineer
E.g., using models (see next slides)
Re-implement
How can one decide which strategy to follow?
Aversano, Esposito, Mallardo, Tortorella
Ransom, Somerville, Warren …
Tot hier
/ SET / W&I

26. Reengineering Analysis
The horse shoe model
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

27. How to make the artefacts more evolvable?
Analysis
Reverse engineering
The horse shoe model
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

28. How to make the artefacts more evolvable?
Analysis
Reverse engineering
Restructuring, model evolution
The horse shoe model
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

29. How to make the artefacts more evolvable?
Analysis
Reverse engineering
Restructuring, model evolution
The horse shoe model
Forward engineering
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

30. How to make the artefacts more evolvable?
Analysis
Reverse engineering
Restructuring, model evolution
The horse shoe model
Forward engineering
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

31. How to make the artefacts more evolvable?
Analysis
Reverse engineering
Restructuring, model evolution
The horse shoe model
Forward engineering
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

32. Reverse Engineering: From Code to Models
Depends on
What kind of model would we like?
structure / behaviour
precise / approximate
What kind of code do we have?
complete / incomplete
compilable / executable / neither
programming languages:
heterogeneous / homogeneous
“special cases”:
process models
business rules
We need tools!
No silver bullets!
/ SET / W&I

33. Example: Reverse Engineering Class Diagram
Bellflower (C++)
CPP2XMI
Coloured areas and arrows correspond to details present in the inferred class diagram and not in the original one. Differences are: more methods and fields in certain classes (green), and aggregation arrows (red).
Inferred class diagram contains more details than the original one:
Additional fields and methods in certain classes
Additional relationship: aggregation
/ LaQuSo / Mathematics & Computer Science

34. Example: Reverse Engineering State Machines
Flow System Controller (C), module SPUR
CPP2XMI
/ SET / W&I

35. Example: Reverse Engineering Sequence Chart
Artificial (Java, EJB 3.0)
ctx:Invocation
Context
:Login
:Profiler
:Auditor
:ProductFacade
em:
EntityManager
:Product
productInfo(id)
access(ctx)
proceed()
measureDuration(ctx)
proceed()
doAudit(ctx)
proceed()
logMethods(ctx)
proceed()
productInfo(id)
find()
getDescription()
/ SET / W&I

36. How to make the artefacts more evolvable?
Analysis
Reverse engineering
Restructuring, model evolution
See talk about MDE!
The horse shoe model
Forward engineering
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

37. Summarizing… Software Evolution
A “hot” topic
Challenges:
Science and engineering
Different evolving artifacts
Different evolution patterns and strategies
Co-evolution
Analysis and visualization
/ SET / W&I

38. Software Evolution @ TU/e
What does evolve?
Repositories for models – Zvezdan Protic, MvdB
What can we learn from evolution so far?
Statistical approaches – Serguei Roubtsov, AS, MvdB
How evolvable are our systems?
SQuAVisiT – Serguei Roubtsov, Martin vd Vlist, Peter Schachtschabel
How to make the artefacts more evolvable?
Reverse engineering – Dennie van Zeeland, MvdB, AS
How is the evolution implemented?
Model transformations – Marcel van Amstel, Luc Engelen, MvdB
Migration coordination – Erik de Vink, Suzana Andova
and also YOU!
2IS99 – Software evolution – Research topics in software evolution
/ SET / W&I

39. Questions?
/ SET / W&I