1. Model-based testing of complex manufacturing systems: A case study
10th Dutch Testing Day, October 8, 2004
Niels Braspenning
Systems Engineering Group,
Eindhoven University of Technology

2. Outline ASML and Tangram Model-based testing (MBT) MBT framework
Case: ASML laser subsystem
Conclusions
Future work
Questions

3. ASML: TWINSCAN Key figures: Supervisory Control (SUN)
10-15 sub-systems (Power PC)
Lower level CPUs (firmware)
50 processors
400 sensors,
500 actuators,
12,5 MLOC
Language: C (Java, Python,
Matlab)
Presentation ‘Testing High Technology Developments at ASML’
by Tom Brugman, Dutch Testing Day 2003

4. Tangram Development activities effort Test time Shipment date

5. Model-based testing (MBT)
Testing is an operational way to check whether a system implementation is correct
There is a need for unambiguous specifications and for test automation  MBT:
Specification is described in a formal model (unambiguous)
Tests can be generated from this formal model (automation)
However:
Unambiguous specification takes a lot of time (thus modeling also)
Current (worldwide) automatic testing covers small and specific test domains

6. MBT framework documentation, mental models System interface access
formal, suitable input for test tool
automatic generation and execution of tests
unambiguous description of correct system behavior

7. Case: laser subsystem Functional black box testing of laser subsystem
Objectives:
Show applicability of MBT with TorX (RU/UT) within ASML
Show usability of  (TU/e) specification models for MBT
Investigate limitation/shortcomings of the approach
communication
laser beam

8. Case: approach E L EP LP Spin LT TorX TDRV Laser Test rack Informal
specification E L
Specification model ()
model EP LP
Specification model (Promela)
translate
ASML docs
validate/verify
Spin
verification properties
validate/verify
Mental
model LT
convert
Test model
(Trojka, laser only)
Test environment
TDRV
TorX
Test tool
Test
rack
= model
= tooling
= ASML
Laser
SUT

9. Case: informal specification
Interface specifications
Operational sequences
State diagrams
(Confidential)

10. Case: specification model
Environment process:
Closed system
Specific traces for analysis
Laser processes:
IO: handle communication
LS: process commands, execute actions, create responses
CLS: hold current laser state
Error handling
Configurable behavior
Initially: Cymer ELS 7600 with laser state behavior only

11. Case: different models
Simulation only
Modeling expressivity (data, time, functions, stochastics, hybrid)
Reasonably easy to modify/configure
Promela
Simulation and verification
Less modeling expressivity (workarounds)
Less easy to modify/configure
Trojka
Testing only (open system)
Same characteristics as Promela

12. Case: results 1/2 Validation and verification Testing
Mainly good weather (operational) behavior
Simulation with  and SPIN
Model checking with SPIN
Testing
Also bad weather (exceptional) behavior
Discrepancies are detected by TorX, with different sources:
Documentation, model incompleteness, tooling, SUT

13. Case: results 2/2

14. Conclusions Proof of concept delivered: Shortcomings/limitations:
Applicability of MBT with TorX within ASML
Usability of  specification models for MBT
Shortcomings/limitations:
Manual translation from  to Promela
Tested functionality is limited due to limited interface access
Problems with data and timing
Error injection in SUT is not possible

15. Future work Laser case: Tangram: Test more functionality
Test real laser
Tangram:
Direct connection between  and TorX
Extensions for data, timed and hybrid testing
Using simulation models for (early) integration testing
Research in test strategy, test infrastructure, and model-based diagnosis

16. Questions/discussion