1. Construction by Configuration: An opportunity for SE research
Prof. Ian Sommerville St Andrews University Scotland

2. St Andrews Small Scottish town, on the north-east coast of the UK
Home of golf
Scotland’s oldest university (founded in 1413)
Small university focusing on research and teaching excellence

3. Software engineering 2008
Over the past 10 years or so, there has been a radical shift in development strategy for large-scale organisational information systems
Instead of developing software by programming in a conventional language, the primary development mode is now based around the reuse of off-the-shelf systems such as ERP systems or generic systems (COTS) for a specific application area
Existing approaches to software engineering have to evolve to take into account this approach to systems engineering
Why has this happened? Lower perceived risk by senior management (not necessarily true). Commodification of IT - not seen as a means of gaining competitive advantage; Difficulties in finding and retaining staff; Faster time to deployment. Hard sell by vendors.

4. Commercial Off-the-Shelf Systems (COTS)
COTS-based reuse relies on adopting a generic system that has been developed to deliver some business function and adapting that systems to the needs of a particular organisation
Increasingly, new systems are developed by integrating several existing COTS, sometimes with different owners. Systems are becoming systems of systems
Muse about researchers keeping in touch with practice. Problems if research only becomes relevant to niche areas of practice.

5. COTS-based reuse
COTS-based reuse is based around systems for a specific application domain (e.g. patient information system in healthcare).
This incorporates generic functionality and abstractions from the domain which are configured for each specific customer. COTS are designed for configuration.
This can be a (very) long-term process
The current Spanish en-route air traffic control system is being reconfigured for use in the UK. The process is anticipated to take 6 years.
Observation: This has already become the dominant mode of software development for business systems.

6. Construction by configuration (C-b-C)
COTS and ERP systems include a set of reusable abstractions that represent an application domain
The reusable abstractions have to be configured to adapt them to their local circumstances of use
This can range from simple parameter setting through the definition of business rules
Sometimes, configuration may require new, special purpose component development.
C-b-C also may also include configuring the process as well as configuring the software that is the way people work has to change to meet the model assumed by the COTS system
We are still programming - but using a system-specific programming model rather than a generic model.

7. Configuration
I use the term ‘configuration’ to cover both customisation and adapting software to a specific execution environment
Adapting the system to reflect:
The specific needs of a customer (e.g. a hospital);
The requirements of a user or group of users (e.g. maternity unit or physiotherapy or both);
Interaction with other systems;
The characteristics of the platform on which the software executes.

8. Configuration activities
Selecting required system modules
Defining a data model
Defining business rules
Defining workflows
Defining external interactions
Defining the user interface
Defining reports produced
Setting platform parameters
Data re-engineering
Re-defining business processes
Not conventional programming - how much is normally understood by SE researchers.

9. A patient management system
We have followed the deployment of a patient management system (PIMS) for mental healthcare in Edinburgh, Scotland
Based around a generic COTS-package developed for hospitals in England
This was designed to be adapted for supporting different kinds of clinic including mental healthcare
Scotland has its own legal system and laws and healthcare is a devolved responsibility. The Scottish government sets its own targets and priorities for healthcare

10. Procurement issues
A major influence in choosing the particular COTS system was that it offered the opportunity for hospital managers (rather than clinical staff) to control how information was recorded
The government placed a tight deadline on hospitals for reporting against a set of targets
There was little time to carry out a detailed comparison of alternatives and this system had already been successfully deployed elsewhere

11. Software engineering
Configuring the data model required for a particular set of clinics
Configuring the menus for the particular type of clinic and the patient information that had to be recorded
Configuring the reports to be generated by the system
Configuring the rules that should be applied to the system data

12. Issues and problems Objectives conflict
Management objective - provide reports against a set of headings defined by the Executive
Clinical objectives - record patient information according to clinical classification
Invalid configuration assumptions
The language for defining the rules of the system was not expressive enough to cover the requirements of Scots law regarding the forced detention of patients who were a danger to themselves or others
Failure to configure the process
Local process differences meant that different clinics recorded different information about patients
Make point here about the system not having a workflow based model so didn’t force a single process. But, it did not allow multiple data models to reflect the different processes

13. The C-b-C process for COTS
There is often no clear distinction between specification, design and development
Systems are rarely completely configured before being put into use. The configuration process continues as the system is integrated with operational processes
There can be extensive (uncontrolled) “user” configuration of the system
The expectations of system stakeholders have to be configured to reflect the realities of he system

14. Process characteristics
Two-stage system requirements process
Identify general business requirements to chose a reusable system;
Identify specific requirements from the settings where the system will be used.
Business process redesign
The business processes have to be redesigned to r4flect the constraints of the selected system
Co-design of process and software
May be more active stakeholder involvement in the development process.
Limited testing - often post-deployment’ testing to iron out problems
Good practices such as configuration management, reviews, etc. are very difficult to implement
Testing - no specification to test against. System doesn’t break but is just unsuitable.

15. Choosing a COTS system
The decision on which COTS system to use is rarely a transparent process, based on a detailed analysis of the requirements in a specific setting
Issues that affect the decision include:
Political issues;
Platform issues;
Cost and schedule issues;
Availability of expertise;
Prejudice!
The end result is often like the old joke where a man asks for directions to how to get to Dublin..
The response is ‘If you’re going to Dublin, then I wouldn’t start from here’.

16. Co-design
For C-by-C, separating requirements and development/deployment is unlikely to be successful
Requirements compromises are essential because the stakeholder’s real needs need to be matched with what the system actually provides
Configurability makes co-design, with stakeholder involvement in the design process, much easier
However, users may misunderstand the (lack of) system flexibility

17. System testing Testing is a particular problem for COTS-based systems
Systems are not designed for running automated test suites
There is no specification that can serve as a basis for deriving tests
Failures are often not observable by testers as they are failures in process support that can only be recognised by users
Problems that arise are often a consequence of interactions between the process of use and the system rather than system failure.

18. System evolution
Handover process from consultants to local IT staff. Change of system ‘ownership’
Limited expertise with system - may be managers rather than software engineers
Problems of change management exacerbated because of the system configurability
Increasing system entropy as local configuration changes are implemented
Evolution of underlying COTS system outside control of system owners

19. Configuration problems
Understanding the configuration possibilities
Knowing what can be configured is not easy, especially if more than one product is included in the system
Understanding how to configure a system
Typically, configuration requires both business knowledge and technical knowledge
Predicting the consequences of configuration decisions
It is often difficult to understand how changing a configuration will affect the use and behaviour of a system while it is in operation
Understanding how a system is configured

20. Multiple configuration possibilities
Ways in which MS Word can be configured (that I’m aware of)
Preferences screen
Customisation screen
Organiser screen
Definition of templates
Definition of styles
Definition of macros
Inclusion of add-ins (e.g. Endnote)

21. Understanding how to configure

22. Configuration predictability
If you change a program, it is generally possible to hypothesise how this will affect the behaviour of the executing system
When you change a configuration, the relationship is less obvious
Change becomes a process of ad hoc experimentation. Gurus evolve who can suggest what to do but who can’t explain why it should be done that way

23. Understanding the configuration
Once a system has been configured, how can others understand the configuration.
Requirements/design/code traceability is difficult enough in conventional systems but much harder in COTS where:
Requirements are not properly documented as a result of the co-design process;
Understanding requires knowledge of the COTS system + knowledge of the configuration.
Change costing and impact analysis is difficult
E.g. in the healthcare system we looked at, it was originally estimated that changing menus would take a few days. It ended up taking several months.

24. Research challenges Design for configuration Configuration engineering
Design principles for configurability
Configuration visibility
Configuration description
Configuration engineering
Knowledge management for system engineering:
System modelling
Methods and tools for COTS-based system testing
Adapt supporting software engineering processes for C-by-C

25. Principles for configurable design
We need a coherent set of tested principles in this area to guide design and implementation
Possible examples of principles
Principle of visibility - make configurations explicit?
Principle of low coupling - reduce dependencies across configurations?
Principle of scalability - separate configuration of system deployment from configuration of functionality?
Principle of localisation - localise volatile configuration entities?
The diversity of different approaches to C-b-C mean that identifying unifying principles across different systems is very difficult

26. Configuration visualisation
In most current systems, configurations are opaque - it is difficult or impossible to view the configuration as a whole, at different levels of abstraction or to navigate through the configuration space
There is a need for tools that allow engineers to see the ‘configuration state’ of a system and to explore dependencies across that state
What configuration information is most useful to system developers?
What is the best way to present configuration views to system developers?

27. Configuration description
Configuration policies
There is sometimes a wide gap between organisational policy (especially, security policy) and the realisation of that policy as a configuration
Configuration compilation
Methods and tools are needed to allow higher level configuration specification and the automatic generation of a configuration from that specification

28. Knowledge management
Configuration engineering requires a close collaboration between different classes of system user, business analysts and system developers
Our experience has been that major problems arise because of failure to communicate across and within the different groups in an organisation
Better knowledge management support is required to capture, classify and share knowledge about assumptions, problems, the organisation and the system itself

29. Configuration modelling
Modelling notations such as the UML are focused on developing object-oriented models of the system
While some UML models (such as Use-Case models) can be used for configuration engineering, the majority of the UML models are either inappropriate or cannot be applied in a natural way when configuring a generic system
What domain-oriented modelling techniques are most useful, how can they be used and adapted to support understanding and analysis of systems being built using CbyC?

30. System testing Workflow testing Test management Test-first development
How can workflow models built into the system be tested against the reality of work?
Test management
How can test coverage be assessed and improved?
How can regression testing be supported?
Test-first development
Is this a valid approach for systems developed using CbC?

31. Supporting processes
Users may be partly responsible for parts of the configuration process. This causes major problems with supporting processes
Configuration management
Existing tools are oriented to source code
Change management
Can users be stopped from making ad hoc changes? How do users find out about changes?
Quality management
What does a ‘high-quality’ configuration mean?

32. Conclusions
Construction by configuration is a reality for modern business software development
Political, technical and business factors mean that this approach will be increasingly used for all types of system
Our current ‘ad hoc’ approaches are not good enough - we need to adapt software engineering methods and techniques for this approach.