Design science methodology for a PhD project Roel Wieringa 1IS Seminar 31st August

Goal Discuss one of our research projects 1.Subject 2.Methodological structure Relate this to your project 2IS Seminar 31st August

1. Subject Project goal: – Extending enterprise architecture design with goal-oriented requirements engineering Partners: – BizzDesign (Wilco Engelsman) – Novay (Dick Quartel) – UT (Roel Wieringa) 3IS Seminar 31st August

Work done so far KAOS and i* and Tropos and causal loop diagrams and.... ARMOR Light Armor Fusion: reasons to add elements Reduction: reasons to delete elements 4IS Seminar 31st August

Result (roughly) StakeholderDesire Goal RequirementSystem property 5 Goal is desire for which budget is made available If the goal is a system property, it is called a requirement Requirements must be satisfied by observed system property influence decomposition conflict IS Seminar 31st August Phenomenon Whatever is observable to stakeholders Desire is phenomenon ideal for a stakeholder Is-a

Status Concepts are stable for now... Fixed point in improvement iterations – Still to look at action goals and state goals – Measure of preferences? Relationships require a lot more work – Influence: state→state, action →state, action →action. Use + and – of causal loop diagram for state → state? – Decomposition: analytic and synthetic – Conflict: logically impossible, physically impossible, technically impossible, violation of standards, socially impossible, humanly impossible, too expensive... We will do some empirical work before we return to this 6IS Seminar 31st August

2. Methodological structure Engineering cycle (a.k.a. regulative cycle): Problem investigation Treatment design Treatment validation Implementation (transfer to Bizzdesign) Implementation evaluation 7IS Seminar 31st August “Solution” Artefact Technique Method....

Methodological structure of PhD thesis Problem investigation Treatment design Treatment validation Implementation (transfer to Bizzdesign) Implementation evaluation 8IS Seminar 31st August

What did we do so far Problem investigation: TBD Treatment design: light Armor Treatment validation: TBD Implementation (transfer to Bizzdesign) Implementation evaluation 9IS Seminar 31st August

Methodological options Problem investigation – What we know now: Mging large # of reqs, tracing EA to reqs, determining impact of req change or EA change are all problematic – To do: survey Among BizzDesign consultants? BizzDesign customers? In-depth interviews? Questionnaire Treatment design: light Armor Treatment validation: TBD 10IS Seminar 31st August

11 Initial treatment design Initial problem understanding Improved treatment design Improved problem understanding Treatment validation

Methodological options Problem investigation: Some survey Treatment design: light Armor Treatment validation: TBD 12IS Seminar 31st August

Methodological options Problem investigation: Some survey Treatment design: light Armor Treatment validation questions – Internal validity: LArmor in BizzDesign consultancy mitigates identified problems? – Trade-offs: All of LArmor? – Sensitivity: All of BizzDesign concultancy? Only Bizzdesign consultancy? 13IS Seminar 31st August

Methodological options Problem investigation Treatment design: light Armor Treatment validation methods – Simulation: Wilco uses LArmor with toy example – Simulation: Wilco uses LArmor with past project data – Action research: Wilco uses Larmor in a project – Opinion research: Wilco elicits opinion of Bizzdesign consultants about Larmor concepts & method – Experiment: Wilco studies BizzDesign consultants using Larmor on an artificial problem, compares to control group using current BizzDesign method – Case study: Wilco studies BizzDesign consultants using Larmor in a project 14IS Seminar 31st August

Methodological issues Generalization – Applicable to similar GORE methods? – Applicable to similar companies/projects? Theory – A theory is a dictionary plus a statement with scope of application; mechanism counterfactually supported Preferably, statement indicates mechanism Scope = generalization; always uncertain, possibly limited Mechanism Counterfactual – Theories applied? – Theory that will explain case study observations? 15IS Seminar 31st August

Organizational design science (Van Aken) – Elaborate problem investigation (diagnosis) – Like a physician Artefact design science (Wieringa) – Elaborate treatment design and validation – Like a medicine manufacturer 16IS Seminar 31st August

Discussion What about your project? Written presentation structure? – Problem – treatment – validation, with varying emphases Oral presentation structure? – Table of contents of two items – Context: the stage – Goals: what are you going to do – Why interesting for audience – What is new wrt related work – Substance 1: the version that everyone can understand – Substance 2: brief indication of some technical detail – Please keep your slides readable: minimum point size is 18 17IS Seminar 31st August

Discussion What about your project? Written presentation structure? – Problem – treatment – validation, with varying emphases Oral presentation structure? – Table of contents of two items – Context: the stage – Goals: what are you going to do – Why interesting for audience – What is new wrt related work – Substance 1: the version that everyone can understand – Substance 2: brief indication of some technical detail – Please keep your slides readable: minimum point size is 18 18IS Seminar 31st August

Discussion What about your project? Written presentation structure? – Problem – treatment – validation, with varying emphases Oral presentation structure? – Table of contents of two items – Context: the stage – Goals: what are you going to do – Why interesting for audience – What is new wrt related work – Substance 1: the version that everyone can understand – Substance 2: brief indication of some technical detail – Please keep your slides readable: minimum point size is 18 19IS Seminar 31st August