1. Initial Development of a Theory of Software Evolution TUM Munich 19 January 2004 Meir (Manny) Lehman School of Computing Middlesex University White Hart Lane London N17 8HR, U.K mml@mdx.ac.uk http://www.cs.mdx.ac.uk/staffpages/mml

2. 22 January 2004 - 2 - © 715-charts-mml] Studies have consistently demonstrated that software evolution is a disciplined phenomenon of major practical significance and worthy of systematic study Current understanding of software evolution reflects 35 years of data acquisition, empirical analysis, modelling, interpretation Introduction General Definition of Evolution The results provide foundations and a framework for development of a formal theory of software evolution

3. 22 January 2004 - 3 - © 715-charts-mml] Evolution Evolution is staged process of discrete, progressive, change over time in the characteristics, attributes, properties of some material or abstract, natural or artificial, entity or system or of a sequence of these* Examples * Software Evolution and Software Evolution Processes, MM Lehman, JF Ramil, Ann. of Softw. Eng. v. 14, 2002 My work focused on nounal view of release-based evolution process Complementary, mutually supporting views of evolution - nounal - nature, causes, implications and management strategies - verbal - support, management and implementation

4. 22 January 2004 - 4 - © 715-charts-mml] Examples of Release-based Evolution Explanation, interpretation, validation of the patterns suggests empirical generalisations Common features striking Late 60s operating system Analysis to identify and interpret behavioural patterns, sources of differences Size in modules over release sequence numbers (RSN) and a large current real time system Exemplify E-type (real world) nounal software evolution studies

5. 22 January 2004 - 5 - © 715-charts-mml] E-type Programs, Systems, Applications Behaviour during execution, its consequences and wider results of execution determine acceptability of program Operate in and address real world problems and activities For satisfactory results programs must continue to reflect all relevant application and domain properties despite any changes that occur in the latter Stakeholder satisfaction is ultimate criterion of acceptability

6. 22 January 2004 - 6 - © 715-charts-mml] E-type Programs and the Real World Real World Domains Specification abstraction Reification process develops specified program properties Program reification Eliminating properties by abstraction invokes assumption that factors omitted are irrelevant to satisfactory execution and achievement of acceptable results Real world operational domain is model of specification Operational domain properties are abstracted to yield specification is_model_ of, that defines required program as is program

7. 22 January 2004 - 7 - © 715-charts-mml] E-type Programs and the Real World But real world is dynamic, always changing Reification also introduces other properties Program must be validated with respect to that domain to ensure thatany such incompatibilities are not objectionable But program must be a model-like reflection of application in its required operational domain model-like reflection Incompatibilities with required real world properties may, however, be introduced These must not be incompatible with specification Specification Real World Domains abstraction Program reification is_model_ of validation If any are revealed they must be excluded by changes to the specification

8. 22 January 2004 - 8 - © 715-charts-mml] Implications As the real world changes, incompatibilities may arise To restore the acceptability of the results of execution, requires that the program, system, documentation and/or operational procedures must be changed -to avoid misbehaviour -ensure user satisfaction In a changing real world, system must be continually adapted, i.e. evolved, to remain valid Moreover, program properties previously neutral may become unacceptable, as a consequence of assumptions that have become invalid But program validity relates to the application and operational domain as at time of execution

9. 22 January 2004 - 9 - © 715-charts-mml] Assumptions These may be explicit or implicit, conscious or unconscious, by commission or omission, recorded or unrecorded As already indicated, abstraction and reification processes inherently embed assumptions in specifications and in programs Management and updating of the reflections of assumptions a key, indeed the key, to reliable, efficient, safer computer usage Whatever the case, if programs or application processes are not to become unsatisfactory, harmful assumptions must be rectified by changes to applications, programs or real world properties or procedures Key and inescapable role of assumptions illustrated by bounding process

10. 22 January 2004 - 10 - © 715-charts-mml] Bounding Elicit, reconcile, merge, limit restricted, subjective, often biased, views of individual stakeholders -all levels of management -organisational, individual, direct or indirect users -marketeers, suppliers, procurement personnel -domains and application experts -system, software engineers, developers -etc., etc. Process blends viewpoints, determines broad goals, agrees initial assumptions of application and domain Iterative convergence to consensus changes assumptions, application, bounds that must be maintained compatible with reality as world changes Release process Evolving Views Application Concept Application Domain

11. 22 January 2004 - 11 - © 715-charts-mml] Software Maintenance Maintenance of the validity of an implicitly embedded assumption set and, thereby, stakeholder satisfaction by evolution of the system

12. 22 January 2004 - 12 - © 715-charts-mml] Global process a feedback-system The Software Development Process Exogenous change All in changing real world domain Introduction into use closes feedback loop that drives global evolution To yield iterative sequence of releases Culmination: validated program Program Series of steps Step n Final steps: installation Then operation, changing application, which changes domain Application Application domain Step 2 Step 1 Step i Step i+1 Operational program and, possibly, the operational domain

13. 22 January 2004 - 13 - © 715-charts-mml] Disciplined Process Thirty five years of studies have shown that feedback-driven evolution is a disciplined phenomenon whose behaviour can be encapsulated in a series of laws* * At present (Jan 2004) observations have largely been restricted to the currently most widely used development data, but phenomenological analysis suggests that the observed phenomena will, in general, be more widely observed though possibly changed in details

14. 22 January 2004 - 14 - © 715-charts-mml] The Laws of Software Evolution Laws provide a base and framework for a theory of software evolution III 1974 Self Regulation Global E-type system evolution is feedback regulated II 1974 Increasing Complexity As an E-type system is evolved its complexity increases unless work is done to maintain or reduce it IV 1978 Conservation of Organisational Stability The work rate of an organisation evolving an E-type software system tends to be constant over the operational lifetime of that system or segments of that lifetime VI 1991 Continuing Growth The functional capability of E-type systems must be continually enhanced to maintain user satisfaction over system lifetime VII 1996 Declining Quality Unless rigorously adapted and evolved to take into account changes in the operational environment, the quality of an E-type system will appear to be declining VIII 1971, 1996 Feedback System (Recognised 1971, formulated 1996) E-type evolution processes are multi-level, multi-loop, multi-agent feedback systems V 1978 In general, incremental growth (growth rate trend) of E-type systems constrained by need to maintain familiarity Conservation of Familiarity I 1974 Continuing Change An E-type system must be continually adapted else it becomes progressively less satisfactory in use, more difficult to evolve No.Brief Name Law

15. 22 January 2004 - 15 - © 715-charts-mml] and validate against further observations Carnap's Approach to Theory Formation Iterative extension Derive practical implications Determination of Rules, Guidelines The latter provide inputs to formal theory formation Formal Theory Formation Observations and data provide continuing inputs to two level process which leads to observed behavioural patterns, models Definitions, Empirical Generalisation validate Continued Observation Interpretation Explanation Modelling Observational Theoretical Two levels Initial Data Predict behaviours on basis of emerging theory Prediction and empirical generalisations To produce generalisations, definitions, theorems, practical implications

16. 22 January 2004 - 16 - © 715-charts-mml] Entities Involved Five basic classes of entities involved in theory development:- Assumptions may become axioms; any class may prove to be a theorem Example of empirical approach

17. 22 January 2004 - 17 - © 715-charts-mml] Preliminary Definitions E-type operational domains are, respectively, sub-domains and sub-sets of the real world with determined or implied bounds An E-type specification abstracts an E-type application to identify the properties and behaviours that must be reflected in a system if it is to produce a solution to the application needs or terms of reference Program is satisfactory to the extent to which its execution fulfils those needs An E-type program is a set of computer executable instructions defining a valid solution to an E-type application Real world comprises the entire universe, its properties and all happenings in it An E-type application addresses a problem or supports an activity in a specified real world operational domain A specification is satisfactory if it defines a program that satisfies real world needs as reflected by the authoritative requirements of stated or implicit stakeholders

18. 22 January 2004 - 18 - © 715-charts-mml] The real world has an unbounded number of properties Early Observations It may be partitioned in an unbounded number of ways into domains some of which, at least, also possess an unbounded number of properties Properties of an E-type program are abstractions of properties of the applications and domains to be supported E-type application attributes must be accurately reflected in program for it to satisfy accepted specification The attribute set of an E-type program - as distinct from such programs in execution - is bounded Being dynamic, its attributes also change continually, independently Installation, use and changes to installed software, change those properties

19. 22 January 2004 - 19 - © 715-charts-mml] E–type operational domains have, in general, an unbounded number of attributes Relevant Inferences There will always be operational domain attributes not addressed by program E–type specifications and programs have a bounded number of attributes but reflect an unbounded number of assumptions Assumptions reflected in an E-type program - e.g. bounding any aspect of an application or operational domain - may become invalid E-type program execution entails uncertainty, satisfaction cannot be guaranteed

20. 22 January 2004 - 20 - © 715-charts-mml] That is:- Outline of a proof of the Principle of Software Uncertainty However often a system has successfully executed it may produce unsatisfactory results on the next execution

21. 22 January 2004 - 21 - © 715-charts-mml] Practical Relevance What, if any, is practical relevance of a theory of software evolution?

22. 22 January 2004 - 22 - © 715-charts-mml] One Example - Assumptions Management Develop and use tool support for the above and for system adaptation Much more could be said, but not today Given a theorem that invalidation of assumptions is an inevitable source of E-type system misbehaviour or even failure, good practice requires one to:- Identify, capture, structure, document and update the rationale, assumptions, decisions underlying it and review, revalidate these whenever a change occurs in the specification, design or implementation of the application or its domain Institute periodic and event-triggered reviews and assessments to anticipate or identify a need for changes in the assumption set Make search for and questioning of assumptions in all inspections and validations a specific and required activity Improve questioning of assumptions, for example, by using independent implementation and validation teams

23. 22 January 2004 - 23 - © 715-charts-mml] Summarising High-level process behaviour patterns share descriptions that provide a basis for empirical generalisation though details may be paradigm sensitive Such phenomena are more related to organisational and societal factors and the inherent feedback nature of the global process than to technology Hence underlying global phenomena are, in general, expected to remain Software global evolution process a multi-level feedback system Observed feedback-driven discipline and empirical generalisations indicate that a formal theory of software evolution may be developed and play an important role in software process improvement

24. 22 January 2004 - 24 - © 715-charts-mml] Relevant Papers A complete listing of papers is provided at http://www.doc.ic.ac.uk/~mml/feast Rules, Tools and Guidelines MM Lehman, Rules and Tools for Software Evolution Planning and Management, pos. paper, FEAST 2000 Workshop, Imp. Col., 10 - 12 Jul. 2000, DoC, Res. Rep. Nov 2000, a revised version to appear in Annals of Software Engineering, Spec. Issue on Software Management, vol. 11., 2001 MM Lehman, The Future of Software - Managing Evolution, inv. contr., IEEE Software, Jan-Feb. 1998, pp. 40-44 Theory MM Lehman and JF Ramil, Towards a Theory of Software Evolution - And Its Practical Impact, invited talk, ISPSE 2000, Intl. Symposium on the Principles of Software Evolution, Kanazawa, Japan, Nov 1-2, 2000 Process Modelling G Kahen, MM Lehman, JF Ramil and PD Wernick, Dynamic Modelling in the Investigation of Policies for E-type Software Evolution, ProSim 2000, International Workshop on Software Process Simulation and Modelling, 12 - 14 Jul. 2000, London, UK BW Chatters, MM Lehman, JF Ramil, P Wernick, Modelling a Software Evolution Process, ProSim'99, Softw. Process Modelling and Simulation Workshop, Silver Falls, Oregon, 28-30 Jun. 1999, also as Modelling a Long Term Software Evolution Process in J. of Softw. Proc.: Improv. and Practice, 2000 v. 5, iss. 2/3, Jul. 2000, pps. 95-102 MM Lehman, DE Perry and JF Ramil, On Evidence Supporting the FEAST Hypothesis and the Laws of Software Evolution, Proc. Metrics'98, Bethesda, MD, 20-21 Nov. 1998, pp. 84-88 P Wernick and MM Lehman, Software Process White Box Modelling for FEAST/1, ProSim '98 Workshop, Silver Falls, OR, 23 Jun. 1998. As a revised version in Journal of Systems and Software, Vol. 46, Numbers 2/3, 15 Apr. 1999 MM Lehman and P Wernick, System Dynamics Models of Software Evolution Processes, Proc. Int. Wrkshp. on the Principles fo Software Evolution IWPSE-98, ICSE-20, 20-21 Apr. 1998, Kyoto, Japan, pp. 6-10 MM Lehman, DE Perry, JF Ramil, WM Turski and P Wernick, Metrics and Laws of Software Evolution - The Nineties View, Proc. Fourth International Symposium on Software Metrics, Metrics 97, Albuquerque, New Mexico, 5-7 Nov. 97, IEEE Comp. Soc. or. n. PR08093, pp 20-32. Also as in K El Eman and N H Madhavji (eds.), Elements of Software Process Assessment and Improvement, IEEE CS Press, 1999 WM Turski, A Reference Model for the Smooth Growth of Software Systems, IEEE Trans. Softw. Eng., v. 22, n. 8, Aug. 1996, pp. 599 - 600 Estimation JF Ramil and MM Lehman, Exploring Cost Estimation Models in the Context of Continuing Software Evolution, FESMA-AEMES Software Measurement Conference 2000 "Management Excellence through IT Measurement", Madrid, Spain Oct. 18-20, 2000 Lessons Learnt (e.g., Modelling Methodology) JF Ramil and MM Lehman, Metrics of Software Evolution as Effort Predictors - A Case Study, Proc. ICSM 2000, Int. Conference on Software Maintenance, 11-14 Oct. 2000, San Jose, CA Component-based and Integration-intensive Processes Other MM Lehman and JF Ramil, Software Evolution Phenomenology and Component Based Software Engineering, IEE Proc. Softw., sp. issue on Component Based Software Engineering, scheduled for Dec. 2000, earlier version as Tech. Rep. 98/8, Imperial College, London, Jun. 1998 JF Ramil (ed.), Preprints of FEAST 2000 International Workshop on Feedback and Evolution in Software and Business Process, Imp. Col., London, 10 - 12 Jul. 2000, 124 pp. JF Ramil, MM Lehman and G Kahen, The FEAST Approach to Quantitative Process Modelling of Software Evolution Processes, Proc. PROFES'2000 2nd International Conference on Product Focused Software Process Improvement, Oulu, Finland, 20 - 22 Jun. 2000, in Frank Bomarius and Markku Oivo (eds.) LNCS 1840, Springer Verlag, Berlin, 2000, pp. 311 - 325. MM Lehman, Feedback in the Software Evolution Process, Keynote Address, CSR Eleventh Annual Workshop on Software Evolution: Models and Metrics. Dublin, 7-9 Sept. 1994, Workshop Proc., Information and Software Technology, sp. is. on Software Maintenance, v. 38, n. 11, 1996, Elsevier, 1996, pp. 681 - 686

25. 22 January 2004 - 25 - © 715-charts-mml] Bibliography A listing with additional papers and other material is provided at and some may be accessed via http://www.cs.mdx.ac.uk/staffpages/mml References indicated with an ‘ * ’ have been reprinted in Lehman MM & Belady LA, Program Evolution—Processes of Software Change, Acad. Pr., London 1985 *Belady LA & Lehman MM, An Introduction to Program Growth Dynamics, in Statistical Computer Performance Evaluation, W Freiburger (ed), Academic Press, New York, 1972, pp. 503 - 511 Boehm BW, Software Engineering, IEEE Trans. on Comp., v. C-5, n. 12, Dec. 1976, pp. 1226 - 1241 id., A Spiral Model of Software Development and Enhancement, Computer, v. 21, May 1988, pp. 61 - 72 id., Software Engineering Economics, Englewood Cliffs, N.J, Prentice-Hall, 1981 Brooks FP, No Silver Bullet - Essence and Accidents of Software Engineering, Information Processing 86, Proc. IFIP Congress 1986, Dublin, Sept. 1-5, Elsevier Science Pubs. (BV), (North Holland), pp. 1069 - 1076 *Lehman MM, The Programming Process, IBM Research Report RC 2722, IBM Research Centre, Yorktown Heights, NY, Sept. 1969, also in Program Evolution— Processes of Software Change q.v. *id., Programs, Cities, Students - Limits to Growth?., Imperial College. Inaugural Lecture Series, v. 9, 1970 - 1974, also. in [GRI78], pp. 42 - 69, Lehman MM & Belady LA, 1985, pp. 133 - 163, also in Program Evolution—Processes of Software Change q.v. *id., Laws of Program Evolution - Rules and Tools for Programming Management, Proc. Infotech State of the Art Conf., Why Software Projects Fail, - Apr 9 - 11 1978, pp. 11/1 - 11/25 *id., Programs, Life Cycles and Laws of Software Evolution, Proc. IEEE Sp. Iss. on Softw. Eng., v. 68, n. 9, Sept 1980, pp. 1060 - 1076 Lehman MM, Stenning V & Turski WM, (1984). Another Look at Software Design Methodology, ICST DoC Res. Rep. 83/13, Jun 1983. Also, Software Engineering Notes, v. 9, no 2, Apr 1984, pp. 38 - 53 Lehman MM & Belady LA, Program Evolution,- Processes of Software Change, Academic Press, London, 1985, 538 p. id,. Evolution - The Cause of Iteration, Third Process Workshop, Breconridge, CO, Nov. 1986. In Iteration in the Software Process - Proc. 3rd Int. Process Wrkshp., Dowson M (ed), IEEE Comp. Soc. Press, Mar. 1987, pp. 29 - 32 Lehman MM, Process Models, Process Programs, Programming Support, Inv. Resp. To A Keynote Addr. By Lee Osterweil, Proc. 9th Int. Conf. on Softw. Eng., Monterey, CA, 30 Mar. 2 Apr 1987, IEEE Comp. Soc. pub. n. 767, IEEE Cat. n. 87CH2432-3, pp. 14 - 16 id., Uncertainty in Computer Application, Comm. of the ACM, Vol. 33, No. 5, May 1990, pp. 584-586 id., Uncertainty in Computer Application and its Control through the Engineering of Software, J. of Softw. Maint., Res. & Pract., v. 1, 1 Sept 1989, pp. 3 - 27 id., Models and Modelling in Software Engineering, Ency. of Softw. Eng., J Marciniak (ed), Wiley and Co, 1994, vol. 1, pp. 698 - 702 id., Software Evolution, loc cit, vol. 2, pp. 1202 - 1208 Osterweil L, Software Processes are Software Too, Iteration in the Software Process, Proc. of the 3rd Int. Proc. Worksh., Breckenridge, CO, 17 - 19 Nov. 1986, IEEE cat. n. TH0184-2, IEEE Comp. Soc. order n. 709, 1987, pp. 79 - 80 Perry DE, Policy and Product-Directed Process Instantiation, Proc. of the 6th Int. Softw. Process Workshop, 28-31 October 1990, Hakodate, Japan Rajlich VT and Bennet KH, A Staged Model for the Software Life Cycle, Computer, July 2000, pp. 66 - 71 Turski WM, And No Philosophers' Stone Either, Inf. Processing 86, Proc. IFIP Congr., Dublin, Sept. 1 - 5, 1986, Elsevier Sci. Pubs, London, pp. 1077 - 1080 Wilkes M V, Wheeler D J & Gill S, The Preparation of Programs for an Electronic Digital Computer, Addison Wesley Press Inc., 1951, 167 pp. Wirth N, Program Development by Stepwise Refinement, CACM, v.14, n.4, Apr 1971, pp.221-227 *Woodside CM, A Mathematical Model for the Evolution of Software, J. of Sys. and Softw. vol. 1, no. 4, Oct 1980, pp. 337 - 345 Zurcher FW and Randell B, Iterative Multi-Level Modelling - A Methodology for Computer System Design, IBM Res. Rep. RC 1938, Nov. 1967, IBM Res. Centre, Yorktown Heights, NY 10594. Also in Information Processing 67, Proc. IFIP Congr. 1968, Edinburgh, Aug. 1968, pp. D138 - 142

26. 22 January 2004 - 26 - © 715-charts-mml] Backups

27. 22 January 2004 - 27 - © 715-charts-mml] Final Word - A Societal Challenge Growing societal dependence on software constitutes a threat and indicates need for wider understanding of the evolution phenomenon to provides framework for directing and driving coordinated software process improvement Empirical study of the newer process paradigms and of assumption management must also be high on the priority list Current approach to software process improvement relies on ad hoc proposals for development of new or improved methods, tools, procedures etc. The nounal approach to software evolution addresses questions, which seeks to understand the nature, causes, impact, implications, management, control of software evolution, deserves far more attention, must be more widely pursued

28. 22 January 2004 - 28 - © 715-charts-mml] Way forward to formation of theory of software evolution appears realistic though not without challenges Current state: behavioural, patterns invariants for current widespread process paradigm identified; models constructed, interpretations, empirical generalisations proposed Software Evolution Outline example Ultimately theory may be generalisable to cover wide range of: -Real world domains -Applications -Organisations -Software processes Initial challenge, selection of appropriate generalisations, determination and formalisation of definitions Early empirical generalisations provide foundation for development of definitions, axioms, candidate theorems, proofs in formal theory

29. 22 January 2004 - 29 - © 715-charts-mml] A Driver of Evolution They reflect application and operational domains, each unbounded in the number of their properties Such invalidation over time a key source of pressure for continual software and system evolution Programs are artefacts produced by humans in finite time and so are necessarily bounded, that is finite, in the number of their static properties Gap bridged by unbounded number of assumptions reflected in specifications, program, interfaces, documentation, operational procedures and so on Some of these will become invalid as a result of domain and application changes Thus, every E-type program is incomplete relative to the domains it addresses

30. 22 January 2004 - 30 - © 715-charts-mml] E-type System Evolution as a Phenomenon Potential for disciplined study and for theoretical base and framework suggested by common behavioural abstractions and invariants, across industrially developed software from variety of domains and evolved using variety of methods, tools Development of an empirical theory Core observations -real world domains and applications each have an unbounded number of properties -programs are finite in the number of their properties and, therefore, reflect an unbounded number of assumptions -gradual invalidation of some of these leads to an intrinsic need for continuing evolution -feedback system characteristics largely determine global evolutionary behaviour -consequence of domain and global process properties, rather than of technology -some key features encapsulated in or consequence of Laws and in Uncertainty Principle -source of relationships between laws related to feedback characteristics of process

31. 22 January 2004 - 31 - © 715-charts-mml] Introductory Observations Demonstration of software as a disciplined phenomenon and related empirical generalisations provide, at least, initial feasibility Whether a satisfying empirical theory can be developed and a formal theory derived remains to be demonstrated Some initial implications derived from the laws have been discussed* and exemplify what might be derived from such a theory * Rules and Tools for Software Evolution Planning and Management, Annals of Software Engineering, special issue on Software Management, v. 11, November 2001, pp. 16 - 44 Software System Maintenance and Evolution in an Era of Reuse, COTS and Component-Based Systems, Joint Keynote Lecture, ICSM. and WESS 99, Oxford, 3 Sept. 1999 and Software Evolution in the Age of Component-Based Software Engineering, IEE Proceedings Software, v. 147, n. 6, Dec. 2000 Empirical studies have yielded laws providing candidate observations, potential inferences

32. 22 January 2004 - 32 - © 715-charts-mml] Initial generalisations adopted over a long period of observation Example at Observational Level Generalisations and associated definitions A set of observations and one of inferences Believed to be sufficient to support principle of software uncertainty Associated definitions intuitive

33. 22 January 2004 - 33 - © 715-charts-mml] Metrics and Modelling Recalibrate, validate models as new data becomes available so as to reflect process, application, environmental changes For system as a whole and its parts, acquire, plot, model, interpret historical evolution data and determine patterns, trends, growth and their rates of change Model and exploit dynamics of global process to improve planning and process, identify interactions, evaluate and optimise policies, control strategies Develop tools to support data collection, modelling, etc. All the above relate to global process, apply to both technical and organisational activities

34. 22 January 2004 - 34 - © 715-charts-mml] Evolution Management Create, maintain comprehensive documentation, emphasising identification and recording of assumptions - critical for usability, system evolvability and longetivity When validating, check interaction with, impact on unchanged parts of system - impact of assumptions Assess likely functional and non-functional evolutionary trends in advance and review as part of release planning, taking system, domain volatility into account Develop tools for capture, structured recording and periodic review of assumptions and of changes to them Plan, manage, control release to users Establish baselines of key measures over time - to support review Involve application, domain specialists in assessment Capture, monitor, analyse, exploit metrics, patterns, trends, rates of change

35. 22 January 2004 - 35 - © 715-charts-mml] Observe safe change rate limits Follow established software engineering principles - e.g., information hiding - to minimise spread of change between system elements Release Management Assign resources to control and reduce complexity and its growth Plan for clean-up releases if excessive functional increments are unavoidable Alternate enhancement, extension and clean-up, restructuring releases When large functional increments appear unavoidable distribute across several releases as in evolutionary development - Gilb Constrain release increment scope, size as suggested by models of past growth using criteria such as ~ ≤ m, ~(m+s) i.e. ~(m ‹ incr ‹ (m+2s)), ~ ≥ (m+2s) - as in statistical process control - to determine whether increments are safe, risky, unsafe More generally, assess need and assign resources to anti-regressive activities Assumptions play key role in release planning and management CMP next chart

36. 22 January 2004 - 36 - © 715-charts-mml] Observe safe change rate limits Follow established software engineering principles - e.g., information hiding - to minimise spread of change between system elements Release Management Assign resources to control and reduce complexity and its growth If excessive functional increments are unavoidable, plan for clean-up releases Alternate enhancement, extension with clean-up, restructuring releases When large functional increments appear unavoidable, distribute across several releases as in Gilb’s evolutionary development And, using models, to anti-regressive activities Assumptions play key role in release planning, management

37. 22 January 2004 - 37 - © 715-charts-mml] Software Components Many issues, e.g.: -Functional and interface specifications -Embedded assumptions -How components are used -System evolution -Disparate users -Long term impact on cost, quality, integrity, reliability -Component developer ownership rights Identification, understanding of issues essential if components are to be effective over prolonged period despite assumption set changes Compatibility with disparate users, themselves evolving, and with concurrent evolution of system, application, domains, must be maintained Long term costs are likely to increase; quality, availability, evolvability decline Assumptions embedded in components essentially unknowable, constrained by ownership interests, clashes with evolving, disparate, systems unpredictable perhaps irreconcilable CMP next chart

38. 22 January 2004 - 38 - © 715-charts-mml] Software Components Issues relate to: -Components themselves -Way they are used -System evolution -Components in an evolving system -Long term impact on cost, quality, integrity, reliability In a world moving to component-intensive software and processes important to examine implications of evolution phenomenon to: -Determine if, why and how components are sensitive to it -Derive implications of laws of software evolution in component context -Investigate means to mitigate and control impact Present discussion restricted to issues related to assumptions Identification and understanding of issues essential if components are to be effective over prolonged period

39. 22 January 2004 - 39 - © 715-charts-mml] Bottom Line To be of lasting value components must be semantically and syntactically as clearly and completely defined as any operator in a programming language Problems must be identified, solved if assumptions are to be managed, component usage is to have lasting value Techniques outlined earlier apply also component-based domains New issues to be addressed: -Independence of system and component development organisations -Protection of commercial competitive market place interests -Multiple, possibly competitive, clients with conflicting interests, concerns Component-based architecture and design logically equivalent to programming in very high level programming language Must identify specific problems that will surely arise and devise means to overcome, or at least control, them

40. 22 January 2004 - 40 - © 715-charts-mml] Conclusion Decisions based on balancing cost, value, risk that arise from alternative strategies and approaches Views may appear pessimistic but I am not a Luddite Problems in the use of components, in particular those relating to assumptions must be identified, spelt out, solved to retain control of computer usage Eventually, problems that have arisen in conventional programming will re- emerge Intervening period must be used to identify specific problems that will surely arise and to devise means to overcome, or at least, control them

41. 22 January 2004 - 41 - © 715-charts-mml] Phenomenology In philosophy, the science of phenomena as perceived, as opposed to the study of being, the nature of things as they are - √ Philosophical investigation, description of conscious experience in all varieties without reference to question of whether experience is objectively real - √ Theory More or less plausible or scientifically acceptable general principle or body of principles offered to explain phenomena - √ Body of theorems presenting clear, rounded and systematic view of a subject - √ Analysis of a set of facts in their ideal relations to one another - ? General or abstract principles of a body of facts; pure as distinct from applied science - X Analysis of a set of facts in their ideal relations to one another - √

42. 22 January 2004 - 42 - © 715-charts-mml] Classes of Assumptions About operational, functional, geographical, linguistic, etc. bounds political, economic, social, operational environments unanticipated needs/desires/opportunities not satisfied by current system Number of potential candidates for change is unbounded, hence, in general, change and evolution are inevitable Many arise as a result of feedback, e.g. learning from experience, observation of unexpected behaviour

43. 22 January 2004 - 43 - © 715-charts-mml] Areas Addressed Much of what follows appears self-evident, elements in a code of good practice Example - assumption management Some recommendations* developed for: -process management -release management -assumption management -component-based development * Lehman MM and Ramil JF, Rules and Tools for Software Evolution Planning and Management, Annals of Softw. Eng., spec. iss. on Softw. Management, v. 11, Nov 2001, pp. 16 - 44 Can only consider one today, and then only very briefly Innovative in that recommendations are unified by common conceptual framework and, potentially, by formal theory all derived from real world observation

44. 22 January 2004 - 44 - © 715-charts-mml] And, therefore, of E-type software Final Word Evolution inevitable in computer application in a dynamic real-world Main current effort in software evolution R & D has largely adopted a verbal, tool oriented, approach that develops methods, processes, tools etc. More effort in study of the evolution phenomenon Rapidly increasing societal dependence on software represents a real threat and indicates need for wider understanding of the phenomenon to provide framework to direct and drive coordinated development, process improvement Interpretation of evolution as a noun leads to seeking understanding of its nature, causes, impact,implications and control, leads to more systematic progress Empirical study of the newer process paradigms and of assumption management high on the priority list but deserves and justifies much more attention yielding localised, ad hoc process improvement