1. Work Item “Patterns in Test Development (PTD)” Re-start Meeting 17 March, Berlin Helmut Neukirchen neukirchen@informatik.uni-goettingen.de Institute for Informatics University of Göttingen Germany

2. Patterns in Test Development 2 Agenda  17 March 2005: 10:00 – 17:00 (18 March: 9:00 – 12:30) 1. Introduction  Introduction of attendees  Approval of agenda  Introduction to Patterns & Work Item 2. Status of current document  What has been achieved so far?  What needs to be done? 3. Presentations  Formal Test Purpose Language (from IPv6 testing STF)  Any further experience reports? 4. Future work  Work plan  Time Frame/Next meeting  Assignment of work 5. Any other Business  PTD mailing list at ETSI  Document area on a server  Further interested people? Lunch at 12:30

3. Patterns in Test Development 3 Agenda  1. Introduction  Introduction of attendees  Approval of agenda  Introduction to Patterns & Work Item

4. Patterns in Test Development 4 Introduction to Patterns (1/2)  Idea of patterns:  Proven solutions for problems arising again and again in a certain context.  Enable re-use,  Provide common vocabulary for specific kinds of solution.  Pattern usually defined using a pattern template consisting of three major parts: context, problem, solution.  Solution provided by a pattern is not new, since it is a proven solution.  “Pattern Mining”: solution qualifies for a pattern, if it is found at least three times.  Pattern provides abstract solution: in contrast to a library function, a pattern needs to be instantiated by implementing the abstract solution using some formal language.  Patterns are more flexible than a library.

5. Patterns in Test Development 5 Introduction to Patterns (2/2)  Pattern catalogues exist for various domains:  Alexander et. al.: A Pattern Language: Towns, Buildings, Construction, 1977  Gamma et. al.: Design Patterns, 1995  Buschmann et. al.: Pattern-Oriented Software Architecture, 1996  But: not much work on Test Patterns, i.e. patterns for the testing domain

6. Patterns in Test Development 6 Introduction to Work Item  Quote from ETSI Work Item description: “The result of the work should describe how Test Patterns are specified and how they are used in a test generation methodology framework.”  Within this Work Item: restrict Test Patterns to the test domain relevant for ETSI (i.e. protocol testing)  I.e. aim of Work Item deliverable DTR/MTS-00091: Provide guidance for test case developer:  how to use existing test patterns,  how to “mine” and specify new test patterns.  provide some examples for test patterns.  Not: provide full test pattern catalogue.

7. Patterns in Test Development 7 Specific Aspects of Test Patterns  Possible deviations from some concepts of classical “software” patterns:  Solutions may be language specific (e.g. TTCN-3, TPlan) (“idioms”).  Nevertheless, they may still be abstract, i.e. provide a solution which is not 100% copy & paste, but needs to be customized.  Pool of test suites which may be subject of “pattern mining” is small:  Deviate from the “solution found at least three times” rule, instead trust on (unwritten) experience.  Relate patterns from different test development phases together:  A pattern found in a certain test purpose might imply the usage of certain pattern during development of a test case.

8. Patterns in Test Development 8 Test Pattern Example Name Timer on transmission Class Behaviour Testing phases Specification Testing goals All Application Any domain Intend Avoid deadlock situation when transmitting data from test component Context For the testing of reactive systems tested typically via interfaces Parameter Timer duration Roles test component, source port, destination port Detailed After calling a method from a test component or sending a message on a description given port, a timer should be started to avoid deadlock in case the SUT does not reply to the function call or to the transmitted message Example timer t(duration); outPort.send(outMsg); t.start; alt{ [] inPort.receive(inMsg) { t.stop; return pass; } [] t.timeout { return fail; } } Consequences None Related Default pattern patterns Known uses Protocol testing

9. Patterns in Test Development 9 Agenda  2. Status of current document  What has been achieved so far?  What needs to be done?

10. Patterns in Test Development 10 Achieved Work (1/2)  DTR/MTS-00091 V1.1.1 (2004-03) (ETSI docbox 38TD13.doc): Draft TOC, some draft chapters:  Introduction (Motivation, Use of test patterns)  Test Pattern Concepts (Pattern classification/instantiation)  Test Pattern Definition (Pattern template)  Architectural Patterns  Behavioural Patterns  Data Patterns  Methodological Aspects (Pattern “mining”)  Informative Annex: Concrete Patterns  TR will not contain normative catalogue of test patterns:  Instead provide some examples  how test patterns may look like,  how to work with test patterns,  how to “mine”/create new test patterns on your own.

11. Patterns in Test Development 11 Achieved Work (2/2)  Let’s have a closer look at the DTR/MTS-00091 V1.1.1 (2004-03) document.

12. Patterns in Test Development 12 General Subjects of Further Study (What needs to be done?)  Elaborate on Behavioural and Data Test Patterns (Sections 8 & 9),  More examples for Test Patterns (Annex A),  More on methodological aspects (Section 10).  Investigate use of formal languages for test patterns:  TPlan (see later presentation),  TTCN-3.  Investigate usage of test patterns throughout whole test development process.

13. Patterns in Test Development 13 Agenda  3. Presentations  Steve/Anthony: TPlan (Formal Test Purpose Language from IPv6 testing STF)  Alain  Any further experience reports (VTT?, …?)

14. Patterns in Test Development 14 Agenda  4. Future work  Suggested Work Plan  Suggested Time Frame  Next meeting  Assignment of work

15. Patterns in Test Development 15 Suggested Further Work  Until next meeting:  Read DTR/MTS-00091 V1.1.1 (2004-03) and collect comments.  “Mine” for patterns on your own.  In existing test suites,  in your brain!  Format them using the proposed test pattern template:  Check whether template is suitable/needs modifications.

16. Patterns in Test Development 16 Suggested Milestone Schedule  Work Item Milestones:  “Stable” Draft available: October 2005 (MTS#41)  TB approval of final document: Spring 2006 (MTS#42)  Is this realistic?

17. Patterns in Test Development 17 Agenda  5. Any other Business  Communication via PTD mailing list at ETSI:  MTS-PTD@LIST.ETSI.ORG  I will subscribe the email addresses on the attendees list.  Not just for announcements, but also for active discussion?!  Document Area at ETSI docbox?  Further interested people?