1. Selecting COTS Products Using a Requirements-Based Approach
Jaelson Castro, Carina Alves
Centro de Informática
Universidade Federal de Pernambuco

2. Motivations Development of large and complex systems
Reusable components or COTS
The potential benefits of using COTS are
increased product reliability and stability, at
shorter development time and reduced cost.

3. What is COTS? Commercial-Off-The-Shelf There is no agreed definition
“ COTS are products:
that are sold, leased or licensed to the general public;
that is usually available without source code;
that is supported and evolved by the vendor who returns the intellectual property rights ”
Software Engineering Institute

4. COTS-Based Development (CBD)
Large incentives from industry and academia
Improved productivity and quality of software development
Emphasizes the acquisition and integration of COTS components over in-house development

5. Main Characteristics of CBD
The nature of COTS suggest new life cycle models
The success of COTS-based systems largely depends on the successful selection of software products

6. COTS-based systems life cycle
Evaluation
Selection
Adaptation
Integration
Update ?

7. The Evaluation Process
Consists of determining the quality of a product in the context of its intended use
Decision making
Must accommodate uncertainty
Evaluation activities can span the entire lifetime of systems

8. The Selection Process Critical activity for COTS-Based systems
Oriented by evaluation criteria
COTS candidates must satisfy user requirements
Includes an accurate understanding of the features of each product

9. The Adaptation Process
COTS are Plug and Pray
Development of all necessary software adapters and enhancements to the selected COTS
Component Wrapping – includes a software barrier around the components; limiting what it can do

10. The Integration Process
Includes all development efforts required to interconnect different COTS into a single integrated system
Consists of developing additional parts not supported by available products
Conventional development efforts

11. The Update Process
Like other systems, COTS-based systems need successive updates
Repair an error
New required functionality
Acquisition of new releases

12. Current Problems with COTS-Based Development
Products from different vendors have to be integrated and tailored to provide complete system functionality
Customers have limited access to product´s internals design
COTS lifecycle is determined by vendor
When using COTS products, customers are placed
in a situation over which they have no control

13. The impact of these problems can be
minimized if adequate attention is paid to the
process of COTS evaluation and selection

14. The importance of the Selection Process
Includes the understanding of user requirements
Careful analysis of the capabilities and limitations of each COTS candidate
Assessment of products´ quality

15. Main Dimensions of the Selection Process

16. Selection Process Challenges
Lack of well defined process
Use of inappropriate evaluation criteria
Back-box nature of COTS components
Unclear system expectations
Rapid rate of changes of COTS
This means that any evaluation will
typically have some degree of error

17. Related Works - *  OTSO STACE PORE
Product Identification
Requirements Acquisition
Non-functional requirements description
Product evaluation
Decision making analysis
OTSO
(Off-The-Shelf Option)  -
STACE
(Social-Technical Approach to COTS Evaluation) *
PORE
(Procurement-Oriented Requirements Engineering)
 Address fully * Deals with the issue but not fully – does not deal with the issue

18. The lack of a careful consideration of
non-functional requirements increases the
risks of COTS failure and costs of the
final system

19. Our Contribution
An approach that addresses the lack of requirements engineering methods during COTS evaluation/selection
Focus on non-functional requirements analysis to assist the evaluation of COTS products

20. The CRE Method
A systematic, repeatable and requirements-driven approach
Iterative process of requirements acquisition/
modeling and product evaluation/selection

21. Decreasing number of candidate products
CRE Iterative Process
The selection is made by rejection, products that do not meet user requirements are eliminated
Time
Number
Increasing number and detail of requirements statements
Decreasing number of candidate products

22. CRE Main Features
Goal oriented - each phase is oriented by predefined goals
CRE provides templates that include some guidelines
Interactive phases – The order is not rigid

23. The CRE Templates Template 1 Goals: Final result:
Information and requirements to be acquired:
Steps / sequence:
Important considerations:

24. The Phases
Identification – identify core requirements and candidates COTS products
Description – Describe each product and user requirements
Evaluation – Analyze products compliance with requirements
Acceptance – Verify legal issues

25. Identification
Define selection goals, include analysis of influencing factors
User Requirements
Application Architecture
Products availability
Organization infrastructure
Project objectives and restrictions
Metas
Goals
Evaluation Criteria
Functional Requirements
Non-functional Requirements
Architecture Issues
Domain Issues
Vendor Guaranties
Market variables
Products Features
Legal Issues

26. Identification Evaluation criteria definition COTS product searching
Elicitation of critical requirements
Interviews and questionnaires techniques
COTS product searching
Possible sources: in-house libraries, Internet, magazines, conferences, vendors.
Final result - list with all COTS candidates

27. Description Evaluation criteria must be elaborated in detail
Non-functional requirements play an important role to discriminate between similar functional products (ex. flexibility, reliability, portability)
It is usually difficult to check if a product satisfies non-functional requirements
Use NFR Framework to refine these requirements

28. Description
Feedback mechanism – information exchange between requirements process and products description

29. Requirements Document
Description
Requirements document is elaborated containing (all) relevant information about stakeholders needs
Requirements Document
Req_ID <unique identifier>
Type <functional, non-functional>
Description <information about requirement>
Priority <vary from 1 to 4>
Contributions <can interact in synergy and conflict>
Comments <additional observations>

30. Description
CRE method provides situation rules to deal with requirements conflict
IF <condition> THEN <action>
If Strong_Confl [Req1,Req2] and Req1_Prio > Req2_Prio
Then Deal with Req1
Else If Strong_Confl [Req1,Req2] and Req2_Prio > Req1_Prio
Then Deal with Req2

31. Description Checklist to help the elicitation of product information
Products Aspects
Vendor Aspects
Price
Maturity
Conformance with quality standards
Time delivery
Capacities
Stability
Benefits
Training
Constraints
Reputation
Version control
Support quality

32. Evaluation
Use of appropriate techniques to assist decision-making process
WSM (Weighted Scoring Method)
Simple but results are not accurate
AHP (Analytic Hierarchy Process)
Complex use, provides more confidence in decisions

33. WSM - Weighted Scoring Methodå
j=1
( weight * scoreaj )
Scorea =
Where a represents an alternative and n the number of criteria
Conformance
Score
Priority
Weight
Does not meet the requirement
Low 1
Meets with restrictions
Medium 2
Partially meets
High 3
Meets
Very High 4

34. AHP (Analytic Hierarchy Process)
Select Product
Product A
Product B
Product C
Goal
Criteria
Alternatives
Vendor Guaranties
Cost
Usability
Functionalities

35. Evaluation
Perform product demonstration sessions to obtain detailed information about COTS features and limitations
Obtain products compliance score with evaluation criteria
Important step during decision-making process

36. Evaluation
The cost of each COTS alternative extends the acquisition price
Apply cost estimation models
COCOTS proposed by B. Boehm
Provides a model to estimate the associated costs of COTS integration

37. Acceptance Negotiation of legal issues with COTS vendors
A license should minimally specify:
License grant
Payments to the supplier
Who owns the licensed product
The risks and liability each party assumes

38. Main Contributions
An effective approach to guide the selection of COTS products
An iterative process of requirements acquisition and product evaluation
Including a detailed description of non-functional requirements
Support for decision-making analysis

39. Future Work
Detailed treatment of requirements prioritization and negotiation
The interplay between software architecture and the selection of multiple COTS

40. Non-Functional Requirements (NFR)
Address important issues of quality for software systems
Related to constraints on system services
Play critical importance during system development
Have a global impact on systems
Referred as “ilities”

41. NFRs Main Features Subjective Relative Interacting
interpreted and evaluated differently by different people
Relative
importance may vary depending on the particular system
Interacting
Attempts to achieve one NFR can hurt or help the achievement of other

42. Non-functional requirements can be
difficult to deal with, yet dealing with
NFRs can be vital for the success
of a software system

43. Non-Functional Requirements
There is not a unique definition or complete list of non-functional requirements
Different researchers use different terminology
Previous works
Bohem, 1976
Roman, 1985
Keller, 1990
Standards ISO, ANSI, IEEE

44. Types of Non-Functional Requirements [sommerville 92]
Process requirements
Product requirements
External requirements
Usability requirements
Legal constraints
Delivery requirements
Reliability requirements
Economic constraints
Implementation requirements
Safety requirements
Interoperability requirements
Standards requirements
Efficiency requirements
Performance requirements
Capacity requirements

45. The NFR Framework Proposed by Chung, University of Toronto
Systematic and global representation of NFRs
Process-oriented approach
Qualitative approach
Represents NFRs explicitly as softgoals

46. Main Characteristics
Softgoals - are the basic unit for representing non-functional requirements
Interdependencies - state interrelationships among softgoals
Methods - offer operationalization techniques
Correlations - provide catalogues for inferring possible interactions

47. The notion of Softgoals
A goal that has no clear definition
Qualitative reasoning
Degrees of satisfaction
Interact in synergy or conflict
Decomposed through AND or OR relationship
AND - the softgoal is satisfied if all of its sub-goals are
OR - the softgoal is satisfied if any of its sub-goals are

48. NFR Framework can be used to
Acquire knowledge about:
the particular domain
functional requirements
particular kinds of NFRs
Identify particular NFRs for the domain
Decompose NFRs
Identify operationalizations (satisfice)

49. Using the NFR Framework (cont.)
To deal with:
ambiguities
tradeoffs and priorities
interdependencies among NFRs
Select operationalizations
Support decisions (design rationale)
Evaluate the impact of decisions

50. Catalogues Present knowledge about NFRs
Sources of knowledge: specialists, developers, textbooks, developer guides, etc.
Provide a broad range of expertise
Types of catalogues:
NFR types (organize NFRs into specialized hierarchies)
method (refine NFRs considering operationalizations)
correlation (show implicit interdependencies)

51. Catalogue of some NFR types
Performance
NFR Types
Time
Space
Security
Confidentiality
Integrity
Availability
Accuracy
Completeness

52. Softgoal Interdependency Graph (SIG)
Secure system
AND contribution
Confidentiality of
system
Integrity of
system
Availability of
system
OR contribution
Operationalization
Identification of
User
Authorization of
User

53. Implicit Interdependency SIG - Softgoal Interdependency Graph
Secure [system]
Integrity
[system]
Availability
Confidentiality
Identification
[user]
Authorization
User-friendly [system]
Accessibility
[capacities]
Learnability
Simplicity
[interface] -
negative
interdependency

54. Dealing with Priorities
Priority softgoals can be identified as:
Critical – vital for the success of the system
Dominant – deal with a significant part of the organization workload
Make appropriate tradeoffs among softgoals

55. Identifying a Softgoal as a Priority SIG - Softgoal Interdependency Graph
User-friendly [system]
Secure [system]
Confidentiality
[system]
Simplicity
[interface]
Accessibility
[capacities]
Learnability
[user]
Integrity
[system]
Availability
[system] + -
Identification
[user]
Authorization
[user] !
Simplicity
[interface]
Priority Softgoal

56. Recording Design Rationale
Design decisions should be supported by well-justified arguments
Reasons can be stated for making refinements, for selecting an alternative, etc.
A Claim softgoal can rationalize tradeoffs

57. Recording Design Rationale SIG - Softgoal Interdependency Graph
Secure [system]
Integrity
[system]
Availability
Confidentiality
Identification
[user]
Authorization
User-friendly [system]
Accessibility
[capacities]
Learnability
Simplicity
[interface] -
Claim Softgoal + !
Claim [User authorization will not hurt system simplicity much] ++

58. Selecting among alternatives
The refinement process continues until the possible solutions are sufficiently detailed
Evaluate the impact of decisions
Consider operationalizations and decide whether a chosen alternative meets a softgoal sufficiently

59. Evaluating the Impact of Decisions
Bottom-up process
Evaluation of softgoals are represented by labels (such as  and X)
Positive contribution
A satisficed offspring results in a satisficed parent
A denied offspring results in a denied parent
Negative contribution
A satisficed offspring results in a denied parent
A denied offspring results in a satisficed parent

60. Identifying a Softgoal as a Priority - SIG
User-friendly [system]
Secure [system]
Confidentiality
[system]
Simplicity
[interface]
Accessibility
[capacities]
Learnability
[user]
Integrity
[system]
Availability
[system] +  X - 
Identification
[user]
Authorization
[user] ++ ! 
Simplicity
[interface]
Claim [User authorization will not hurt system simplicity much]