1. Testing and Management of Distributed Systems
Jie Chi Jini/Smart Homes
Pawan Devatha Jini/Smart Homes
Sudipto Ghosh Interface Mutation
Guergana Markova User Interface
Aditya P. Mathur Testing/Management
Sambhrama Mundkur Testing
Baskar Sridharan Management
Software Engineering Research Center
Purdue University
March 20, 2000

2. Structure of a Brokered Distributed Application
Component
Component
Client/Server
Client/Server
Request/data
Request/data
Stub/Skeleton
Stub/Skeleton
ORB
ORB
Communication over
network or within a process.
Component
12/30/2018
CS 690S

3. Objectives Testing: Monitoring and Control:
Assess the adequacy of a test suite prior to deployment.
Test and debug distributed objects while they are deployed.
Perform fault injection to assess the tolerance of a collection of objects to failures in components.
Monitoring and Control:
Monitor distributed objects.
Provide event-based control.
Gather system performance data.
12/30/2018
CS 690S

4. Adequacy Assessment Problem: Proposed Solution: Existing solutions:
Given a test suite T and a collection of components, assess the goodness of T.
Proposed Solution:
Use Interface Mutation
Existing solutions:
Measure code coverage
12/30/2018
CS 690S

5. What is Interface Mutation ?
Test Suite T contains Request-A.
Request-A
Original
Program
Client
Interface
Server
Response-A
Request-A
Mutant
Mutated Interface
Client
Server
Response-B
If Response-A is identical to Response-B for all requests in T
then T is inadequate and needs to be improved.
12/30/2018
CS 690S

6. Mutant Creation ? Mutating the interface: One change at a time.
Simple changes:
Parameter swap, increment and decrement parameters, nullify a string, zero a parameter, nullify a pointer, etc.
12/30/2018
CS 690S

7. Why Mutate ?
Why mutate?
Doing so will force a tester to develop test t to distinguish a mutant from the original program.
So what?
The new test t is likely to cause the original program to be executed in new ways thereby increasing the chance of revealing any hidden error.
Any experimental evidence?: Yes!!
12/30/2018
CS 690S

8. Experimental Methodology
Select components
Number of components = 3
Number of interfaces = 6
Number of methods = 82
Number of mutants = 220
Seed errors one by one in the components
Errors related to:
Portability, data, algorithm, language-specific
12/30/2018
CS 690S

9. Errors seeded Error Seeded Knuth Category 1
Wrong configuration variable used
Portability 2
Insufficient malloc (not enough memory)
Data 3
No malloc, using null pointer
Data 4
Forgetting to free memory
Data 5
Wrong offset in fseek(), fgets()
Data 6
Forget null string termination
Data 7
Forget strdup
Data 8
Wrong variable used in search
Blunder 9
Counts not incremented
Forgot
12/30/2018
CS 690S

10. Errors seeded Error Seeded Knuth Category 10 Wrong exception thrown
Blunder 11
Wrong activation mode set
Blunder 12
Forgot to remove object during exception
handling. This object was earlier created
in the code.
Algorithm 13
Similar to above: counter not decremented
during exception handling even though object
Was removed.
Algorithm 14
Object counter not incremented even though
object is created.
Algorithm 15
Error in loop termination condition.
Algorithm
12/30/2018
CS 690S

11. Error Revealing Capability
100%
Using
Mutation Adequacy
80%
Using
Statement/Decision
coverage
60%
40%
20%
Comp 1
Comp 2
Comp 3
Total
12/30/2018
CS 690S

12. Code Coverage Comp 1 Comp 2 Comp 3 Total 12/30/2018 CS 690S 100%
Block coverage with
Mutation Adequate Tests
Decision coverage with
Mutation Adequate tests
80%
60%
40%
20%
Comp 1
Comp 2
Comp 3
Total
12/30/2018
CS 690S

13. Tests Required Comp 1 Comp 2 Comp 3 12/30/2018 CS 690S 35
For Mutation Adequacy 30
For statement/decision
coverage 25 20 15 10 5
Comp 1
Comp 2
Comp 3
12/30/2018
CS 690S

14. Results
Interface mutation leads to fewer tests that reveal almost as many errors as revealed by statement and decision coverage.
Interface mutation is a scalable alternative to using code coverage.
12/30/2018
CS 690S

15. The Power of Interface Mutation
Reveals
programming errors in components.
errors in the use of component interfaces
Revealing certain types of deadlocks 4 1 2 5 3
Client Request
Client
Server
Server callback
12/30/2018
CS 690S

16. What remains to be done?
Testing for certain deadlocks and race conditions that occur in:
Components using different threading models in CORBA, specifically
Deadlocks and race conditions that occur inside a component
Testing for system scalability and performance
12/30/2018
CS 690S

17. Architecture of TDS 2.0

18. Design Goals Non-intrusive and low overhead
Ability to turn on/off monitoring
Selective monitoring (filtering)
Runtime performance measures at a low cost
Exploit network topology to assure enhanced performance
12/30/2018
CS 690S

19. Service Region
Service Region S
Hosts S S
12/30/2018
CS 690S

20. Zone-based partitioning
Zonal Manager FT
Zone A S S
12/30/2018
CS 690S

21. Naming Hierarchy Server Level Object Level Iface Level Method Level
Project
Zone A
Zone B
Zone C
Server Level
Server1
Server2
Object Level
Interface1
Interface2
Object1
Object2
Iface Level
Method1
Method2
Interface1
Interface2
Method Level
Method1
Method2
12/30/2018
CS 690S

22. Architecture of TDS 2.0(1) Ar db LLI LL S O MC Zone A
TDS GUI
Zone A
Host 1
Host 2
Host 3
Host 4
Host 5
Host 6 db
Zonal Manager LL
MC: Master Controller
LLI: Local Listener Interface
Ar: Architecture Extractor
12/30/2018
CS 690S

23. Server - Performance Statistics
Up-time
Processing Time
Total number of hits
Number of transient objects
Number of persistent objects
State
12/30/2018
CS 690S

24. Object - Performance Statistics
Name
Up-time
Processing time
Total number of hits
Number of interfaces
State
12/30/2018
CS 690S

25. Interface - Performance Statistics
Name
Up-time
Processing Time
Total number of hits
Number of methods
State
12/30/2018
CS 690S

26. Method - Performance Statistics
Name
Total number of hits
Average, Maximum and Minimum Latency
State
12/30/2018
CS 690S

27. Coverage Details
Graphical view of the ratio of number of requests to total requests (at each level).
Graphical view of utilization (at each level).
12/30/2018
CS 690S

28. Event-based Control Events Actions Timed Request Arrival Any
ALLOW – Allow all incoming requests
DENY – Deny all incoming requests
ALARM – Raise an alarm
MAIL – Send mail
REDIRECT – Redirect requests
12/30/2018
CS 690S

29. Event-based Control Action Object Control Command
Server
Object
Interface
Method
Control Command
Command = (event, action, action_object)
Control command applied at all levels
Note: For request-arrival events, control action executed after arrival but before processing.
12/30/2018
CS 690S

30. Sample Commands Timed command Semantics:
(time 15:45, DENY, (“ZoneA”,”ector”,”bank_server”))
Semantics:
DENY control action started at 15:45 (local time at “ector”).
Any request arrival after successful completion of control action will be denied
(time 18:00, ALLOW, (“ZoneA”,”ector”,”bank_server”))
12/30/2018
CS 690S

31. Sample Event-Action pairs
Control Command
client_id = (“ZoneA”,”lisa”,”TxManager_Server”)
server_id = (“ZoneA”,”ector”,”bank_server”,”Green_Bank”)
event=(client_id,server_id, “IDL:/Bank/1.0”,”closeAccount()”)
action = DENY
action_object = server_id
Command = (event, action,action_object)
Semantics: When a request for “IDL:/Bank/ closeAccount()/1.0” arrives from client_id, DENY control action will be applied on server_id before request is sent to server_id.
12/30/2018
CS 690S

32. Dynamic Testing/Debugging
Server-2
1. Client sends a request R.
6 b
6 c
Client
Server-1
6 a
2. Server responds
3. Client discovers incorrect response.
4. Client reports problem to the SP.
6 d
5. SP becomes a client.
6. SP issues R with a
red token and monitors
the flow of R. SP
12/30/2018
CS 690S

33. Issues with Dynamic Testing
Following the message chain.
Isolating the set of faulty components.
Selectively deflecting messages to instrumented components under debugging.
12/30/2018
CS 690S

34. Current Work Addition of dynamic testing features.
Addition of monitoring and control features.
Management of Smart Homes via specialized TDS.
HomeOwner version of TDS for Palm Tops.
12/30/2018
CS 690S