1. Eindhoven Technische Universiteit Measuring User Satisfaction through Experiments B. de Vries

2. Eindhoven Technische Universiteit Domotica Computable: 5 November 2004 Wildgroei Toekomst droom Losse deelmarkt en

3. Eindhoven Technische Universiteit Innovatie Juiste product ? Juiste doelgroep ? Juiste distributie ? Juiste tijd ? Juiste marketing ? …

4. Eindhoven Technische Universiteit Evaluation Observational> Case studies Experimental> Research

5. Eindhoven Technische Universiteit Characteristics Empirical: Gather evidence through observation and measurement that can be replicated by others Measurement Replicability Objectivity

6. Eindhoven Technische Universiteit Variables Independent: Cause Dependent: Effect

7. Eindhoven Technische Universiteit Scientific research Validity: Are you measuring what you claim to measure ( measuring the right thing) Reliability: The ability to produce the same results under the same condition (Measuring things right) Error: The difference between our measurements and the value of the construct we are measuring

8. Eindhoven Technische Universiteit Validity Internal validity problems Group threats, regression to the mean, time threats, history, maturation, instrumental change, differential mortality, reactive and experimenter effects External validity problem Over-use of special participants group, restricted number of participants

9. Eindhoven Technische Universiteit Between groups Treatment (experimental gp.) No Treatment (control gp.) Measurement Random allocation

10. Eindhoven Technische Universiteit Measuring User satisfaction Virtual Reality Bayesian Belief Networks

11. Desk-Cave

15. Eindhoven Technische Universiteit Set-UP 2 synchronized PC’s with dual monitor output 4 LCD Projectors

16. Eindhoven Technische Universiteit Features 1 : 1 Scale 3DS import Immersion Interaction

17. Eindhoven Technische Universiteit Bayes Theorem From: Evaluation and Decision (7M834)

18. Eindhoven Technische Universiteit Bayesian Belief Network Norma n Late Martin Late Train Strike

19. Eindhoven Technische Universiteit Node Probability Table Norma n Late Martin Late Train Strike Train Strike Norman late TrueFalse True0.80.1 False0.20.9

20. Eindhoven Technische Universiteit NPT’s Train Strike Martin lateTrueFalse True0.60.5 False0.40.5 Train strike True0.1 False0.9

21. Eindhoven Technische Universiteit Analyzing a BBN p(Norman late) = p(Norman late | train strike) * p(train strike) + p(Norman late | no train strike) * p(no train strike) = (0.8 * 0.1) + (0.1 * 0.9) = 0.17 Marginal probability Conditional probability p(Train strike|Norman late) = ( p(Norman late|train strike) * p(train strike) ) / P(Norman late) = (0.8 * 0.1) / 0.17 = 0.47

22. Eindhoven Technische Universiteit Measuring User Satisfaction Using Virtual Reality and Bayesian Belief Networks. 01.11.2004 Maciej A. Orzechowski

23. Eindhoven Technische Universiteit Motivations, aims Current techniques for measuring user preferences (CA, MM, interview) are artificial, lengthy or expensive. For good results we need to get the respondents more involved in the measurement. Can Virtual Reality (VR) improve the quality of measuring preferences: more involved and higher reliability? The aim of this project was to develop and test an interactive VR tool for measuring housing preferences.

24. Eindhoven Technische Universiteit VR System MuseV3 – a Virtual Reality application with functionality of a simple CAD system. Two categories of modifications: Structural modifications (change layout). Textural modifications (change visual impression).

25. Eindhoven Technische Universiteit Structural Modifications Change of internal and external dwelling’s layout. The most important for estimating user preferences. Include following commands: create/resize space; insert openings. Direct impact on overall costs of the dwelling.

26. Eindhoven Technische Universiteit MuseV3 in Desktop CAVE

27. Eindhoven Technische Universiteit Bayesian Belief Network Non-obtrusive interactive method to collect housing preferences. Potential advantages Interaction with the model during the time of preferences estimation. Incremental learning. Possibility to assess:  where the knowledge about preferences is most uncertain.  consistency of measurements.

28. Eindhoven Technische Universiteit Bayesian Belief Network cont. A Bayesian Belief Network (BBN) captures believed relations (which may be uncertain, stochastic, or imprecise) between variables, which are relevant to some problem. Lounge Ext (β1) Garage Ext (β2) Extra Kitchen (β3) 2 Bedrooms (β4) First Floor Ext (β5) Dormer Window (β6) Choice of Lounge Ext Choice of Garage Ext Choice of Extra Kitchen Choice of 2 Bedrooms Choice of First Floor Ext Choice of Dormer Window Price (γ) Family SituationAge

29. Eindhoven Technische Universiteit CPT calculation

30. Eindhoven Technische Universiteit Learning process

31. Eindhoven Technische Universiteit Convergence

32. Eindhoven Technische Universiteit Utility Convergence

33. Eindhoven Technische Universiteit Experiment 1600 letters -> 100 answers -> 64 respondents. Respondents were people searching for a house or who just bought one. 4 kinds of 2 types of tasks (2 traditional, 2 based on MuseV3):  CA: Verbal Description Only (VDO) Multimedia Presentation (MM).  BBN: Preset Options (PO) Free Modification (FM). Each respondent completed both types of tasks.

34. Eindhoven Technische Universiteit Observed-Predicted

35. Eindhoven Technische Universiteit Conclusions The results support the potential of the suggested approach. The results suggests higher involvement of respondents. This approach is non-obtrusive compared to different preference measurement techniques. The system (tool) can be used to:  To assist individual users in creating their own design.  To derive market potential of housing designs at aggregate level.

36. Eindhoven Technische Universiteit Domotica Experiments Alarmering: inbraak, zorg, brand Autom. Verlichting Autom. Zonwering …