1. Component-Based Software Engineering Testing
Ivica Crnkovic
Mälardalen University,
Department of Computer Engineering
Sweden
Page 1, 16 September 2018

2. Västerås
Stockholm
Page 2, 16 September 2018

3. Ivica Crnkovic Professor at Mälardalen Högskola, Västerås
Industrial Software Engineering
Computer Science Laboratory chair
Many years at ABB
CM and Development Environments
Development and management
Interests
Component-based Software Engineering
Software engineering in general
Software Configuration Management
Software Architecture
Software Engineering of RT and embedded systems
Page 3, 16 September 2018

4. Building reliable Component-based Software Systems
Lecture based on:
Building reliable Component-based Software Systems
Artech House Publisher, Boston, London
Ivica Crnkovic, Magnus Larsson – editors
Many known authors from CBSE
Related chapters
1,2, - Component specification
5 – development process
(10) – system trustworthiness
14 – testing reusable software components
Page 4, 16 September 2018

5. Topic overview 1 The challenges of SW- how can CBD help?
What is a software component?
Component models
Component-based Software Development Process
Component- based testing
References
Page 5, 16 September 2018

6. Part 1 The challenges of software development - how can component software help?
Page 6, 16 September 2018

7. SW Productivity Gap (ITEA)
Page 7, 16 September 2018

8. Software paradigm shift
Productivity
© 2001 ITEA Office Assoc
ITEA-Softec project
Agent-based development
Component-based development
Object-Oriented Development
Structured programming
Time
Design method
SA/SO
UML
ADL’s
???
We have been moving system organisation since the 70-ies from mainframe to client server to the current N-tier distributed systems in development, moving towards self organising SW systems.
We started software engineering technologies with structured programming (Structured Analysis and Organisation) and moved through Object-Oriented development now to Component-Based Technologies. The next step maybe Agent-based. Aspect based
OO and CBT allows us to improve the requirements and system modeling, to set up SW development platforms based on re-use and COTS, using Architecture Design Languages at design level
We are working at technologies to support distributed SW development.
However, the Component based development is not yet well understood and broadly used. Large problems remain, e.g. including non-functional requirements at the design phase, scalability of architectures and design patterns, testing and validation in particular in a distributed environment, reconfiguration.
At the same time we need to work towards the next step to allow connectivity for pervasive computing, dynamic reconfigurability, to create self-organising and self-correcting software.
We could call this “agent-based development”
We can be confident that the Information Society will come over us. How beneficial it will be and when the various steps will be made, depends on how we will address the challenges ahead.
The above considerations will certainly shape the planning currently ongoing for the sixth FP.
COBOL, C FORTRAN, ADA
Smalltalk C++
C++ Java
Agglets ?
Language
System organization
Mainframe
Client-server
N-tier distributed
Self-organizing
1970
1990
2000
2005
Page 8, 16 September 2018

9. Also, applications are not adaptable to changes
SW problems and CBD
SW is late
SW is buggy
SW is expensive
… because too often applications are created instead of constructed, requiring
re-invention
re-coding
re-testing
Component-based software engineering can help because it focuses on reuse of
subsystems
infrastructure
Also, applications are not adaptable to changes
Page 9, 16 September 2018

10. Component-based software construction (1)
components
Component #1
construction
application
Component
Component
Component #1 #2 #4
Component #2
Component
Component
Component #3 #3 #4
Page 10, 16 September 2018

11. Component-based software construction (2)
Application 1
Component #1 #2 #3 #4
components
Component
construction
Application 2 #1
Component #1 #5 #3 #6
Page 11, 16 September 2018

12. Component-based software construction (3)
application
Component
Component
component #1 #2
New
Component #4
Component
Component #3 #4
update
Page 12, 16 September 2018

13. Concentration on the business parts
“30 % of SW development effort is spent on infrastructure that adds no value”
Application
specific
Business issues
GUI
GUI
Standard
Reusable parts
Communication
Data model
Deployment
INFRASTRUCTURE
Page 13, 16 September 2018

14. Part 2 What is a component?
Page 16, 16 September 2018

15. Different understandings…
Architectural point of view
The software architecture of a program or computing system is the structure or structures of the system, which comprise software components [and connectors], the externally visible properties of those components [and connectors] and the relationships among them.”
Bass L., Clements P., and Kazman R., Software Architecture in Practice,
Page 17, 16 September 2018

16. Is a unix command a component?
Example - Unix
A general purpose OS, used as a platform for dedicated purposes
Standard API - POSIX
Commands uses as components in a shell-process
Example: sort out words from text files:
$ cat file1 file2 file | sed ’s/ /\ /g’ | sort -u >words.txt
TOO SIMPLE INTERFACE (stream)
Page 18, 16 September 2018

17. Frameworks - building “the real components”
Component Object Management - COM, Active X
Enterprise JavaBeans
CORBA components
.NET
Late binding - easy replacement
Word document
Excel document
My_application
Excel document
component
Page 19, 16 September 2018

18. Component Definition A software component is a unit of composition
with contractually specified interfaces
and explicit context dependencies only.
A software component can be deployed independently
and is subject to composition by third party.
Clemens Szyperski
Page 20, 16 September 2018

19. Composition unit How much costs the glue code?
A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party.
How much components fit together?
How much costs the glue code?
System
Glue code
Components
Page 21, 16 September 2018

20. What is a contract? More discussion later
A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party.
Contract - A specification attached to an interface that mutually binds the clients and providers of the components.
Functional Aspects (API)
Pre- and post-conditions for the operations specified by API.
Specification of invariants
More discussion later
Page 22, 16 September 2018

21. What is an explicit context dependency?
A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party.
Explicit specification of
Provided interface
Required interface
Subcomponent
Interfaces CC C1 s m C2 C3
Page 23, 16 September 2018

22. What does it mean deployed independently?
A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third party.
Late binding - dependencies are resolved at load or run-time.
Replacing of a component independent of the client (main application) if the contract is not broken.
Delegation - interaction with a weak coupling (for example no inheritance).
Which problems can occur in relation to late binding?
How can we guarantee that a replacement of a component will not affect other parts of the system?
Page 24, 16 September 2018

23. Components and Interfaces - UML definition
Component – a set of interfaces
required (in-interfaces)
provided (out-interfaces)
Interface – set of operations
Operations – input and output parameters of certain type
UML metamodel of the concepts used in syntactic specification of software components
Page 26, 16 September 2018

24. IDL Example interface ISpellCheck : IUnknown{
HRESULT check([in] BSTR *word, [out] bool *correct); };
interface ICustomSpellCheck : IUnknown
HRESULT add([in] BSTR *word);
HRESULT remove([in] BSTR *word);
library SpellCheckerLib
coclass SpellChecker
[default] interface ISpellCheck;
interface ICustomSpellCheck;
Page 27, 16 September 2018

25. Substitution
Substituting a component Y for a component X is said to be safe if:
All systems that work with X will also work with Y
From a syntactic viewpoint, a component can safely be replaced if:
The new component implements at least the same interfaces as the older components
From semantic point of view?
Page 28, 16 September 2018

26. Specifying the Semantics of Components
Current component technologies assume that the user of a component is able to make use of such semantic information.
Extension of Interface (adding semantics)
a set of interfaces that each consists of a set of operations.
a set of preconditions and postconditions is associated with each operation.
A set of invariants
Also called: Contractually specified interfaces
(it comes from component definition)
Page 29, 16 September 2018

27. Precondition, Postconditions, Invariants
an assertion that the component assumes to be fulfilled before an operation is invoked.
Will in general be a predicate over the operation’s input parameters and this state
Postcondition
An assertion that the component guarantees will hold just after an operation has been invoked, provided the operation’s pre-conditions were true when it was invoked.
Is a predicate over both input and output parameters as well as the state just before the invocation and just after
Invariant
Is a predicate over the interface’s state model that will always hold
Page 30, 16 September 2018

28. Semantic Specification in a UML metamodel
Page 31, 16 September 2018

29. Semantic Interface Specification
context ISpellCheck::check(in word : String, out correct : Boolean): HRESULT
pre:
word <> “”
post:
SUCCEEDED(result) implies correct = words->includes(word)
context ICustomSpellCheck::add(in word : String) : HRESULT
SUCCEEDED(result) implies words = (word)
context ICustomSpellCheck::remove(in word : String) : HRESULT
SUCCEEDED(result) implies words =
Page 32, 16 September 2018

30. Part 3 Component models
Page 33, 16 September 2018

31. Component models (technologies)
There are many component technologies
In-house developed
Domain specific
Platform specific
Language-related
Standards and de facto standards
COM
.NET
JavaBeans, Enetrpise JavaBeans
CORBA, CCM ……
Page 34, 16 September 2018

32. Part 4 Component-based software development process
Page 35, 16 September 2018

33. Time to Market – “Classical” Development Process?
Product Lifecycle
Problems:
Time To Market
High Costs
Meeting deadlines
Visibility
Requirements
Specification
Design
Implementation
Test
Operation & Maintenance
TIME
Page 36, 16 September 2018

34. Development process
COTS and outsourcing require different development processes
Requirements
Specification
Design
Implementation
Test
Find & Select
Adapt
Deploy
Page 37, 16 September 2018

35. Development process – emphasize reuse
Managing COTS in the early stage of the development process
Requirements
Specification
Design
Implementation
Test
Find & Select
Test
Adapt
Deploy
Page 38, 16 September 2018

36. CBD – separation of development processes
Requirements
Application
development
Specification
Find & Select
Design
Test
Implementation
Adapt
Test
Deploy
Operation & Maintenance
Component
development
Page 39, 16 September 2018

37. Component Selection and Evaluation
The components selected must therefore be evaluated:
Technically
Non-technically
Page 40, 16 September 2018

38. Technical and Non-technical Evaluation
Technical aspects of evaluation include
Integration
Validation
Verification
Examples of non-technical issues include:
The marketing position of the component supplier
Maintenance support provided
Alternative solutions, etc
Page 41, 16 September 2018

39. Evaluation Methods Procurement-oriented requirements engineering:
Identify several component candidates.
Investigate these.
Reject those not compliant with the main requirements.
Continue with the evaluation of the reduced number of candidate components.
If necessary refine the requirements, and repeat the evaluation process.
Page 42, 16 September 2018

40. Component-based testing
Page 43, 16 September 2018

41. Different levels of testing
Integration
Testing
Component
test
Component Development
Integration
Testing ?
Integration
Testing
Component
test
What is the system?
Integration
Testing
Standard techniques for testing
Providing extensive test documentation, test code, test environment
Page 44, 16 September 2018

42. Different levels of testing
Integration
Testing
Component
test
System Development
Integration
Testing
Component
test ?
What is a component?
Page 45, 16 September 2018

43. Testing Component test – a typical black box testing Interface testing
What does interface testing include?
Parameter interface ??
-Shared memory interface???
-Procedural Interface??
-Message passing Interface?
Is that enough?
Functional/procedural/messaging interface
Protocol testing
Contact-based testing
What is with non-functional attributes?
Page 46, 16 September 2018

44. Component testing What about contractual interface? Contractually
Specified
output
input
Input not specified by the contract
Input causing erroneous output (how to find it)?
Fault Injection testing
Page 47, 16 September 2018

45. Component testing Internal state testing A component – a grey box
Documentation
State diagram
Protocol documentation
Test results a 2
a,b 1 b 4 b a a 3 5 b
Page 48, 16 September 2018

46. Evaluation and test of the Assembly
It is necessary to evaluate and test the assembly;
Test of the individual components is not sufficient.
This implies that an investigation of the integration procedure may be a part of an evaluation.
Assembly
Page 49, 16 September 2018

47. Why do we need test assemblies?PA C1 P1
PA = F(P1, P2) C2 P2
Assembly Properties Determined by Component Properties
Page 50, 16 September 2018

48. Assembly Properties Problems of “hidden interface” PH1 A PA C1 P1
PA = F(P1, P2 PH1, PH2, f(PH1,PH2)) C2 P2
PH1
Page 51, 16 September 2018

49. System Testing
Do we need test component-based systems when components (and assemblies) have been tested?
Page 52, 16 September 2018

50. Re-testing cannot be eliminated in general.
Examples of correct components in a new environment
Ariane 5 example
Changes in
System environment
Can cause problems
Stress testing
Fault injection testing
system
Page 53, 16 September 2018

51. Reuse and Exhaustive Testing
How to minimize testing?
Provide evidence based on the component’s:
Contracts,
Experience accumulated
Requirements for the systems (component environments)
Page 54, 16 September 2018

52. Assuming that the environment is invariant
First Use
New Environment is not covered by the first use
Page 55, 16 September 2018

53. Overlapping Input Domain
P-G
First case
-27 -1
A-G
Second case
Covered by us1 and use 2
Page 56, 16 September 2018

54. Reliability and Confidence for a Input Domain
R(c)
C(c)
1027
I(c)
A graph representing the reliability and the confidence
for a input domain
Page 57, 16 September 2018

55. Lower Reliability Requirements
R(c)
C(c)
1027
I(c)
A component reused in a context with lower reliability requirements
Page 58, 16 September 2018

56. Reaching Desired Reliability
R(c)
C(c)
1027
I(c)
The component must be run for a longer time to reach the desired reliability
Page 59, 16 September 2018

57. Previously Experienced Reliability
R(c)
C(c)
1027
I(c)
Previously experienced reliability cannot be utilized if
input domains are outside historical use of the component
Page 60, 16 September 2018

58. Example: Deadline Requirements
If we reuse a component with only a deadline requirement in a new environment in which the execution time is shorter, the component can be reused without re-testing.
Page 61, 16 September 2018

59. Deadline Requirements
R(c)
C(c)
Worst case execution time
New
old
The deadline requirement is still fulfilled
since the new execution time is shorter
Page 62, 16 September 2018

60. Summary
In component-based approach testing plays even more important role
Separation of component from system testing
Not sufficient test components
Testing on assembly and system level is necessary
There are methods for reducing testing
Page 64, 16 September 2018

61. Books
Desmond Francis D'Souza, Alan Cameron Wills: Objects, Components, and Frameworks With Uml : The Catalysis Approach
Clemens Szyperski: Component Software : Beyond Object-Oriented Programming: (1998)
Peter Herzum and Oliver Sims: Business Component Factory : A Comprehensive Overview of Component-Based Development for the Enterprise
Alan W. Brown: Large-Scale Component-Based Development
Betrand Meyer: Object-Oriented Software Construction, 2nd
G.T. Heineman, W. Councill: CBSE Putting the Pieces Together
J. Cheesmam, J. Daniels: UML Components
K. Wallnau: Building Systems form Commercial Components
Ivica Crnkovic & Magnus Larsson: CBSE - Building reliable component--based systems
Page 65, 16 September 2018

62. References
Page 66, 16 September 2018

63. Journals IEEE Computer IEEE Software IEEE Internet Computing
IEEE Transactions on Software Engineering
IEEE Transactions on Computers
ACM Transactions on Programming Languages and Systems languages and programming systems.
ACM Transactions on Software Engineering and Methodology
ACM Transactions on Computer Systems
Software Development (
MSDN Magazine (msdn.microsoft.com/msdnmag)
… all major SW development magazines
Page 67, 16 September 2018

64. Conferences
International Workshop on Component-Based Software Engineering, ICSE, CBSE
Symposium on Engineering of Computer-Based Systems (ECBS), CBSE workshop
Euromicro CBSE track
Page 68, 16 September 2018

65. Conferences
ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) (oopsla.acm.org)
International Workshop on Component-Oriented Programming (WCOP)
Symposium on Generative and Component-Based Software Engineering
Technology of Object-Oriented Languages and Systems (TOOLS) (
International Conference on Software Reuse (ICSR)
Page 69, 16 September 2018