Eindhoven Technische Universiteit Evaluation Observational> Case studies Experimental> Research
Eindhoven Technische Universiteit Characteristics Empirical: Gather evidence through observation and measurement that can be replicated by others Measurement Replicability Objectivity
Eindhoven Technische Universiteit Variables Independent: Cause Dependent: Effect
Eindhoven Technische Universiteit Scientific research Validity: Are you measuring what you claim to measure ( measuring the right thing) Reliability: The ability to produce the same results under the same condition (Measuring things right) Error: The difference between our measurements and the value of the construct we are measuring
Eindhoven Technische Universiteit Validity Internal validity problems Group threats, regression to the mean, time threats, history, maturation, instrumental change, differential mortality, reactive and experimenter effects External validity problem Over-use of special participants group, restricted number of participants
Eindhoven Technische Universiteit Between groups Treatment (experimental gp.) No Treatment (control gp.) Measurement Random allocation
Eindhoven Technische Universiteit Measuring User Satisfaction Using Virtual Reality and Bayesian Belief Networks. 01.11.2004 Maciej A. Orzechowski
Eindhoven Technische Universiteit Motivations, aims Current techniques for measuring user preferences (CA, MM, interview) are artificial, lengthy or expensive. For good results we need to get the respondents more involved in the measurement. Can Virtual Reality (VR) improve the quality of measuring preferences: more involved and higher reliability? The aim of this project was to develop and test an interactive VR tool for measuring housing preferences.
Eindhoven Technische Universiteit VR System MuseV3 – a Virtual Reality application with functionality of a simple CAD system. Two categories of modifications: Structural modifications (change layout). Textural modifications (change visual impression).
Eindhoven Technische Universiteit Structural Modifications Change of internal and external dwelling’s layout. The most important for estimating user preferences. Include following commands: create/resize space; insert openings. Direct impact on overall costs of the dwelling.
Eindhoven Technische Universiteit MuseV3 in Desktop CAVE
Eindhoven Technische Universiteit Bayesian Belief Network Non-obtrusive interactive method to collect housing preferences. Potential advantages Interaction with the model during the time of preferences estimation. Incremental learning. Possibility to assess: where the knowledge about preferences is most uncertain. consistency of measurements.
Eindhoven Technische Universiteit Bayesian Belief Network cont. A Bayesian Belief Network (BBN) captures believed relations (which may be uncertain, stochastic, or imprecise) between variables, which are relevant to some problem. Lounge Ext (β1) Garage Ext (β2) Extra Kitchen (β3) 2 Bedrooms (β4) First Floor Ext (β5) Dormer Window (β6) Choice of Lounge Ext Choice of Garage Ext Choice of Extra Kitchen Choice of 2 Bedrooms Choice of First Floor Ext Choice of Dormer Window Price (γ) Family SituationAge
Eindhoven Technische Universiteit Experiment 1600 letters -> 100 answers -> 64 respondents. Respondents were people searching for a house or who just bought one. 4 kinds of 2 types of tasks (2 traditional, 2 based on MuseV3): CA: Verbal Description Only (VDO) Multimedia Presentation (MM). BBN: Preset Options (PO) Free Modification (FM). Each respondent completed both types of tasks.
Eindhoven Technische Universiteit Conclusions The results support the potential of the suggested approach. The results suggests higher involvement of respondents. This approach is non-obtrusive compared to different preference measurement techniques. The system (tool) can be used to: To assist individual users in creating their own design. To derive market potential of housing designs at aggregate level.