1. Tempus Software Maintenance and Evolution JMSE-SM&E Unit 6: Reengineering and Maintenance in software cycle Prof. Mohammad A. Mikki Gaza, Palestine, 5.8.2014

2. Joint Master in Software Engineering “Software Maintenance and Evolution” Course Unit 6 Reengineering and Maintenance in software cycle Software Maintenance and Evolution 2 ( Source: Ian Sommerville, Software Engineering, sixth Edition, Addison Wesley Publishing Company, 2000)

3. Joint Master in Software Engineering Introduction  This unit introduces software reengineering which is the systematic transformation of an existing system into a new form to realize quality improvements in operation, system capability, functionality, performance, or evolveability at a lower cost, schedule, or risk to the customer. Course Title 3

4. Joint Master in Software Engineering Objectives  To explain why software re-engineering is a cost-effective option for system evolution  To describe the activities involved in the software re-engineering process  To present software reengineering strategies  To distinguish between software reengineering and data reengineering  To explain the problems of data reengineering

5. Joint Master in Software Engineering List of Topics  Overview  Reengineering strategies  Steps involved in software re-engineering  Source code translation  Reverse engineering  Program structure improvement  Program modularisation  Data re-engineering

6. Joint Master in Software Engineering There are several definitions of reengineering:  Software reengineering is the preparation or improvement to software, usually for increased maintainability, reusability or evolveability Source: R. S. Arnold, A Roadmap Guide to Software Reengineering Technology, Software Reengineering, 1994.  Software reengineering is the systematic transformation of an existing system into a new form to realize quality improvements in operation, system capability, functionality, performance, or evolveability at a lower cost, schedule, or risk to the customer Source: S. Tilley and D. Smith, Perspectives on Legacy System Reengineering, SEI White Paper, 1995  Software reengineering is the examination and alteration of a subject system to reconstitute it in a new form and the subsequent implementation of the new form Source: E. Chikofsky and J. Cross, Reverse Engineering and Design Recovery: A Taxonomy, IEEE Software 7(1) Jan 1990: 13-17. Software Reengineering Overview

7. Joint Master in Software Engineering  System must be regularly maintained  Needed when maintenance is becoming a large cost factor  Reengineering is less risk and cost than redevelopment  Reengineering is a cost-effective option for software system evolution.  Needed when some subsystems of whole system require frequent maintenance.  Reengineering makes it easier to maintain.  Improve the system structure to make it easier to understand and maintain 7 Reengineering Objectives

8. Joint Master in Software Engineering 8 Reengineering in software life cycle Requirements Design Implementation Reengineered system maintenance reengineering

9. Joint Master in Software Engineering  When system changes are mostly confined to part of the system then re-engineer that part  When hardware or software support becomes obsolete  When tools to support re-structuring are available  When some but not all sub-systems of a larger system require frequent maintenance When to Reengineer

10. Joint Master in Software Engineering  Reduced risk: As the software is already existing, the risk is less as compared to developing a new software.  Reduced cost: The cost of re-engineering is significantly less than the costs of developing a new software.  Improved program and documentation quality  Makes programs easier to learn  Reduces work required to maintain software  Software becomes easier to test and debug 10 Advantages of Reengineering

11. Joint Master in Software Engineering Disadvantages of Reengineering  Reengineering has practical limits  Major reengineering tasks have to be done manually  Reengineered system is not likely to be as maintainable as a new system developed using modern software engineering methods

12. Joint Master in Software Engineering 12 Restructuring Problems  Not enough comments in original system  Not enough documentation of original system  Reengineering needs heavy computational demands  Reengineering does not help with poor modularization where related components are scattered throughout the code  The understandability of programs may not be improved by reengineering  Short life expectancy: Software is not expected to last very long when first developed

13. Joint Master in Software Engineering  Ignore: discard legacy applications  Re-write from scratch: Re-write legacy applications from scratch  Integrate: consolidate legacy applications into the current and future applications by access in place  Data Warehouse: build a shadow system to house the frequently accessed data  Gradual Migration: re-architect and transition legacy applications gradually Course Title 13 Reengineering Strategies

14. Joint Master in Software Engineering Factors Influencing Reengineering Strategy  Business value of the legacy systems: support current as well as future business processes  Flexibility and growth requirements: if legacy system does not need extensive changes for flexibility and growth then minimal effort for integration may be appropriate  Technical status of the legacy system: represents the quality of the system in terms of modularity, error rates, flexibility and utilization of current technologies. Course Title 14

15. Joint Master in Software Engineering An existing system needs to be discarded in the following situations:  The business has changed and new methods of doing business have been adapted, existing methods are being replaced  It is too costly to try to reengineer the current system  Current systems are monolithic and not reusable  New COTS products can be used with the same or similar functionality for less cost than reengineering  Some organizations discard legacy systems by outsourcing them to software houses that specialize in operating and maintaining legacy applications. Course Title 15 “Ignore” Strategy

16. Joint Master in Software Engineering An existing system needs to be re-written from scratch in the following situations:  The business has changed and new methods of doing business have been adapted, and there is a need to build systems to implement these new methods  It is too expensive to understand current systems  It is too costly to try to reengineer the current systems  Time and resources are available to start from scratch Course Title 16 “Re-write from scratch” Strategy

17. Joint Master in Software Engineering  Organization management may not allow such large investments in re-writing a legacy system from scratch just to have a more flexible system  Business does not stand still while new systems are being developed from scratch  Specifications for legacy systems rarely exist and undocumented dependencies frequently exist  The rewritten system become legacy before completing their development as development takes a long time Course Title 17 “Re-write from scratch” limitations

18. Joint Master in Software Engineering A legacy system needs to be integrated (merged into the current and future applications by access in place) in the following situations:  A lot of the business knowledge and rules are built into the current systems and these need to be preserved  Reduced cost of reengineering current systems compared to developing new ones from scratch  Time and resources are not available to develop new systems from scratch Course Title 18 “Integrate” Strategy

19. Joint Master in Software Engineering A legacy system needs to be integrated (merged into the current and future applications by access in place) in the following situations:  Can not wait for access to needed data until completion of migration  Needed data can be accessed by client applications by employing COTS technologies  Requirements for queries to the data are low (usually not a good idea for large amounts of queries) Course Title 19 “Integrate” Strategy (Cont.)

20. Joint Master in Software Engineering Data warehouse strategy needs to be used for legacy system in the following situations:  The data is the most important part of the existing systems  There is a need for new ways to access and manipulate the data (web-based, distributed)  These is no need to re-implement existing system functionality  There is no time and resources to re-implement existing system Course Title 20 “Data warehouse” Strategy

21. Joint Master in Software Engineering  Operational data: Data that is used for day-to-day transaction processing  Decision support data: Data which are used for business analysis and report generation. Decision support data is extracted from the operational data periodically and downloaded, usually to a separate system, for report generation and analysis. Course Title 21 Format of data in organizations in “Data warehouse” Strategy

22. Joint Master in Software Engineering Gradual migration strategy needs to be used for legacy system in the following situations:  Integration is not cost-effective  Legacy systems are to be phased out  There is an immediate need to update some components in the legacy systems  Time and resources are not available to do the reengineering all at once  There is uncertainty of the effectiveness of how reengineering effort will proceed  Need for prototyping of different reengineering methods Course Title 22 “Gradual Migration” Strategy

23. Joint Master in Software Engineering  The legacy and target systems may coexist during the migration stage.  Gateways design is needed to isolate the migration steps so that end users do not know if the data is coming from old or new systems.  Gateways design that facilitates migration is usually an expensive process. Course Title 23 Properties of “Gradual migration” Strategy

24. Joint Master in Software Engineering 1.Source code translation 2.Reverse Engineering 3.Program structure improvement 4.Program modularization 5.Data rerengineering 24 Steps Involved in Software Reengineering

25. Joint Master in Software Engineering Steps Involved in Software Reengineering Process

26. Joint Master in Software Engineering 1. Source Code Translation  Source code translation is the automatic conversion of program from one language (or language version) to another, e.g., from C++ to Java  The translated language may be an updated version of the original language or a completely different language.  May only be realistic (e.g. cost and time effective) if an automatic translator is available  Manual fine tuning of new code is always required

27. Joint Master in Software Engineering Source Code Translation Motivation  Source code translation is necessary in the following situations: -hardware platform updates -staff skill shortages -organizational policy changes -Lack of software support

28. Joint Master in Software Engineering  Source code is analyzed using a restructuring tool.  Program logic modeled using Boolean algebra  Poorly designed code segments are redesigned  Series of transformation rules are applied to yield restructured logic  Violations of structured programming constructs are noted and code is then restructured  Code restructuring could be done automatically  The resultant restructured code is reviewed and tested to ensure that no anomalies have been introduced Code Translation Guidelines

29. Joint Master in Software Engineering  Internal code documentation is updated.  Create resource exchange diagram showing - data types - procedures - variables shared between modules  Restructure program architecture to minimize module coupling Code Translation Guidelines (Cont.)

30. Joint Master in Software Engineering Source Code Translation Problems  Loss of comments in code  Loss of documentation of code  Heavy computational demands of translation  Restructuring does not help with poor modularisation where related components are dispersed throughout the code  The understandability of data-driven programs may not be improved by re-structuring

31. Joint Master in Software Engineering Program Translation Process

32. Joint Master in Software Engineering Course Title 2. Reverse Engineering  Reverse engineering is the process of analyzing software with the objective of recovering its design and specification.  The program itself is not changed by the reverse engineering process.  The program source code is usually available as an input to reverse engineering.  Reverse engineering is not different from reengineering  May be part of a re-engineering process but may also be used to re-specify a system for re-implementation  Builds a program data base and generates information from it. 32

33. Joint Master in Software Engineering Course Title  Program understanding tools that may be used in reverse engineering: - browsers - cross-reference generators - etc. 33 Program Understanding Tools

34. Joint Master in Software Engineering Course Title 34 Reverse Engineering Diagrammatic Representation

35. Joint Master in Software Engineering Course Title  It is the process of improving the structure of the program to optimize memory use and the lack of understanding of s/w engineering.  The program may be automatically restructured to remove unconditional branches  Program structure improvement simplifies conditions and make them more readable  New conditions and associated actions are added without changing the existing control structure. 35 3. Program Structure Improvement

36. Joint Master in Software Engineering  Typically, program structure becomes more complex as it is modified during maintenance.  Maintenance tends to corrupt the structure of a program.  It becomes harder to understand program structure as it is maintained Program Structure Improvement Motivation

37. Joint Master in Software Engineering Restructuring Approaches

38. Joint Master in Software Engineering Condition Simplification Example of Program Structure Improvement -- Complex condition if not (A > B and (C F) ) )... -- Simplified condition if (A = D or E > F)...

39. Joint Master in Software Engineering Automatic Program Restructuring

40. Joint Master in Software Engineering Course Title 4. Program Modularisation  It is the process of reorganizing a program so that related program parts are grouped together and considered as a single module.  It is usually a manual process that is carried out by program inspection and re-organisation. 40

41. Joint Master in Software Engineering Module Types  Data abstractions: Abstract data types where data structures and associated operations are grouped  Hardware modules: Include all functions required to interface with a hardware unit  Functional modules: Modules containing functions that carry out closely related tasks  Process support modules: Modules where the functions support a business process or process fragment

42. Joint Master in Software Engineering Course Title 5. Data Reengineering  Data reengineering is the process of analyzing and reorganizing data structures and sometimes the data values in a system to make it more understandable.  Data objects and attributes are identified, and existing data structures are reviewed for quality. 42

43. Joint Master in Software Engineering Course Title Data Reengineering Objectives  Create a managed data environment  Clean up the data problems and inconsistencies,  Unifying multiple, inconsistent representations  Database migration  Restructure internal or external data - Example: Migration from RDBMS to OODBMS 43

44. Joint Master in Software Engineering Course Title  May be part of the process of migrating from a file-based system to a DBMS-based system or changing from one DBMS to another  In most cases, data restructuring begins with a reverse engineering activity.  Data reengineering results in either architectural or code-level changes.  Expensive and time-consuming process. 44 Data Reengineering Concepts

45. Joint Master in Software Engineering Course Title  Data degradation - Data quality problem  Inherent limits that are built into the program - Invalid built-in constraints  Architectural evolution - Evolution into its appropriate architecture 45 Why Data Reengineering of Legacy Systems?

46. Joint Master in Software Engineering Course Title When to Reengineer The data are reengineered in following cases:  When data structure is weak -e.g., flat files are currently implemented  When a relational approach would greatly simplify processing 46

47. Joint Master in Software Engineering legacy systems data problems include:  Data naming problems - names are hard to understand -heavy use of aliasing by programmers  Field length problems -different field lengths used for same data item in different programs  Record organization problems -different field orderings used by different programs  Hard-coded literals  No data dictionary Data Problems in Legacy Systems

48. Joint Master in Software Engineering  End-users want data on their desktop machines rather than in a file system. They need to be able to download this data from a DBMS  Systems may have to process much more data than was originally intended by their designers  Redundant data may be stored in different formats in different places in the system Data Problems in Legacy Systems (Cont.)

49. Joint Master in Software Engineering 49  Unlike code restructuring, which occurs at a relatively low level of abstraction, data structuring is a full-scale reengineering activity Difference Between Data Reengineering and code restructuring

50. Joint Master in Software Engineering Data Value Inconsistencies Data value inconsistencies types:  Inconsistent default values used in different programs  Inconsistent units used for same quantity  Inconsistent validation rules  Inconsistent representation semantics  Inconsistent handling of negative numbers

51. Joint Master in Software Engineering Data value Inconsistencies- Inconsistent Default values  E.g., especially missing data codes.  Different programs assign different default values to the same logical data items.  This causes problems for programs other than those that created the data.  The problem is compounded when missing values are assigned a default value that is valid.  The missing data cannot then be discovered.

52. Joint Master in Software Engineering Data Value Inconsistencies- Inconsistent Units  E.g., miles in one program and kilometers in another  The same information is represented in different units in different programs.  For example, in the US or the UK, weight data may be represented in pounds in older programs but in kilograms in more recent systems.  A major problem of this type has arisen in Europe with the introduction of a single European currency.  Legacy systems have been written to deal with national currency units and data has to be converted to euros.

53. Joint Master in Software Engineering Data Value Inconsistencies- Inconsistent Validation Rules  E.g., different standards used to reject values as bad.  Different programs apply different data validation rules.  Data written by one program may be rejected by another.  This is a particular problem for archival data which may not have been updated in line with changes to data validation rules.

54. Joint Master in Software Engineering Data Value Inconsistencies- Inconsistent Representation Semantics  E.g., multiple ways of using uppercase to convey meaning in text strings  Programs assume some meaning in the way items are represented.  For example, some programs may assume that upper-case text means an address.  Programs may use different conventions and may therefore reject data which is semantically valid.

55. Joint Master in Software Engineering Data value inconsistencies- Inconsistent Handling of Negative Numbers  E.g., data validation and representation issues.  Some programs reject negative values for entities which must always be positive.  Others, however, may accept these as negative values or fail to recognise them as negative and convert them to a positive value

56. Joint Master in Software Engineering  Analysis of source code  Data redesign  Data record standardization  Data name rationalization  File or database translation Data Reengineering Steps

57. Joint Master in Software Engineering Data Reengineering Process

58. Joint Master in Software Engineering Approaches to Data Reengineering  Data cleanup  Data extension  Data migration

59. Joint Master in Software Engineering Data Cleanup  The data records and values are analyzed to improve their quality  Does not require any associated program changes  Data records and values are analyzed to improve quality  Duplicate data is removed  Redundant information is deleted  Consistent format applied to all records

60. Joint Master in Software Engineering Data Extension  The data and associated programs are re-engineered to remove limits on the data processing.  Modify the limits on the data structures and the tables (e.g. size or field widths)  This may require changes to programs to increase field lengths, modify upper limits on the tables, etc.  The data itself may then have to be rewritten and cleaned up to reflect the program changes.

61. Joint Master in Software Engineering Data Migration  Moved into the control of a modern DBMS  Data is moved to modern DBMS  Data might be stored in separate files or an older DBMS

62. Data Migration Example After re-engineering … Database-centred System Before re-engineering … Database-centred System Data Migration

63. Tempus Thank you for your attention.