1. IdMRC Social Research Methods Autumn Lecture-Workshop Series

2. Science Henri Christiaans Aim? When is knowledge scientific knowledge?
Criteria?
Knowledge sources?
When is research scientific research?
Henri Christiaans

3. Science Realism What we observe is real Instrumentalism
What we observe doesn’t need to be real
Social constructivism
Theories only get meaning through social and political context
Atomen en electronen als voorbeeld
Sociaal constr: objectiviteit niet belangrijk

4. What is Knowledge? Justified true belief (Plato’s Theaetetus)
The Greeks classify knowledge into 2 types:
Doxa (believed to be true)
Episteme (known to be true)
Doxa  Epistime
Through Scientific process of inquiry
How do we know what we know?
Define knowledge alternatively
Supported by evidence (usually empirical)
Conceive knowledge claims in a probabilistic sense
Knowledge is a matter of societal acceptance

5. How is Knowledge Acquired?
Role of science, where science is a convention, related to societal norms, expectations, values, etc.
Thus, is science equals any scholarly attempt at acquiring knowledge
Science requires conventions to be followed

6. How is Knowledge Acquired?
Role of science, where science is a convention, related to societal norms, expectations, values, etc.
Thus, is science equals any scholarly attempt at acquiring knowledge
Science requires conventions to be followed

7. Knowledge in design
Implicit prioritisation of the (language-based mode of) propositional knowledge (justified true beliefs) seems to exclude certain kinds or formats of knowledge associated with practice, which are often called practical, experiential, personal, or tacit knowledge and which evade verbal articulation.
Despite of continued criticism, the definition of knowledge as ‘justified true belief’ has remained the prevailing definition, and Niedderer (2007) has shown that this understanding of propositional knowledge is implicit in the definition of research because of additional requirements such as the textual/written presentation of an intellectual position (proposition, thesis – ‘true belief’), because of the logic of verification and defence of this intellectual position through argument and evidence (justification), and the requirement for generalisability/transferability and explicit and unambiguous communication.

8. Knowledge sources Observation The Reason Intuition Authority
Experiments/measurements
The Reason
Mathematics/logical reasoning
Intuition
Authority
(Divine) Revelation

9. Science based on empirism
Knowledge derived from how the world is experienced. Scientific statements are controlled by and derived from our experiences and observations. en
Scientific theories developed and tested by experiments and observations through empirical methods

10. Questions to be asked Which methods do we plan to use?
Which methodology defines the use of methods?
Which theoretical perspective do we start from in order to apply the right methodology?
Which epistemology feeds this theoretical perspective?
Therefore, I will try to bring structure, first to link the the relevant questions which have to be asked before we start the actual study.

11. Ontology
1. A systematic account of Existence. Nature of the world around us.
2. (From philosophy) An explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them.
3. The hierarchical structuring of knowledge about things by subcategorising them according to their essential (or at least relevant and/or cognitive) qualities.

12. Epistemology and ontology
The way of understanding and interpreting how we know what we know.
Particular methodologies tend to entail (subscribe to) particular epistemologies and, in their turn, particular forms of ontology

13. Ontology in Computing Terms
For AI systems, what "exists" is that which can be represented.
We can describe the ontology of a program by defining a set of representational terms. Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, an ontology is the statement of a logical theory.
A set of agents that share the same ontology will be able to communicate about a domain of discourse without necessarily operating on a globally shared theory. The idea of ontological commitment is based on the Knowledge-Level perspective.
Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms.

14. Epistemology
From the Greek words episteme (knowledge) and logos (word/speech) is the branch of philosophy that deals with the nature, origin and scope of knowledge.
Refers to our theory of knowledge, in particular, how we acquire knowledge (Hirschheim, 1992).

15. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretativism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Social-constructivism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
stat analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Modelling is not a research method, it’s a tool. Testing the tool is about research
Crotty, 1998

16. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretativism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Social-constructivism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
stat analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Reality. Ho do we know what we know?

17. Theoretical perspective
Philosophical point of view which feeds the methodology and offers a context for the process and the logics, and gives our criteria a basis.
Cultural differences play a role
Colombia:
Oslo: research on design education, describing cases from different countries: emancipation: making people aware of design issues.

18. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretivism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Social-constructivism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
statistical analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Crotty, 1998

19. Three Main Epistemologies
Positivist
Interpretivist
Critical

20. Interpretivism Interpretivism rests upon idealism:
the world is interpreted through the mind; e.g., classificatory schemes of species;
the social world cannot be described without investigating how people use language and symbols to construct what social practices; i.e., understand their experience;
the social world becomes the creation of the purposeful actions of conscious agents; and
no social explanation was complete unless it could adequately describe the role of meanings in human actions
Actions are not governed by discrete patterns of cause and effect (as in positivism), but by rules that social actors use to interpret the world

21. Positivist Science 5 Pillars Unity of scientific method
Causal Relationships
Empiricism
Science and its process is Value-Free
Foundation of science is based on logic and maths

22. Ontology of Positivism
Realism
Universe comprised of objectively given, immutable objects and structures, existing as empirical entities, on their own, independent of the observer’s appreciation of them.
Contrasts with relativism or instrumentalism, where reality is a subjective construction of the mind, thus varying with different languages and cultures.
While hugely successful in physical sciences, it is not as successful for social science.

23. Anti-Positivism Latter part of 19th century
Man as an actor could not be studied through the methods of natural sciences that focus on establishing general laws. In the cultural sphere man is free (Burrell and Morgan, 1979)

24. Post-Positivism
Based on the concept of critical realism, that there is a real world out there independent of our perception of it and that the objective of science is to try and understand it,
combined with triangulation, i.e., the recognition that observations and measurements are inherently imperfect and hence the need to measure phenomena in many ways.
The post-positivist epistemology regards the acquisition of knowledge as a process that is more than mere deduction. Knowledge is acquired through both deduction and induction.

25. Rational Solving Problem Paradigm Reflection in Action Paradigm
Simon versus Schon
Designer
Designer
objective Analysis
Objective Analysis
subjective Interpretation
design Problem
design Solution
Design Task
(= problem + situation+ teime)
design Solution
Rational Solving Problem Paradigm
Reflection in Action Paradigm
Rationalist Root
Constructivist Root
POSITIVISM
PHENOMENOLOGY

26. Methodology
Our strategy and action plans, the design process which defines what specific methods we will choose

27. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretativism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Social-constructivism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
statistical analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Crotty, 1998

28. Types of Research Analytical Historical Philosophical
Literature study Meta-analysis
Descriptive Survey (questionnaire, interview)
Case study Task analysis
Document analysis Correlation anal.
Observation Etnographics
Explorative Survey Correlational
Case study Experimental
Experimental Pre-experimental
True-experimental
Quasi-experimental

29. Ethnographics

30. Types of research methods
empirical
participatory
quantitative
inductive
prescriptive
idiographic
nomothetic
descriptive
deductive
unbiased
qualitative
rational

31. Fundamental Research: the Empirical cycle
testing
deduction
prediction
theory
generalising
modelling
Explaining/
interpreting
evaluating
describing/
evaluation
specifying
induction
hypotheses
knowledge problem
‘t Hart c.s.

32. Practice oriented Research: The regulative cycle
problem from practice
evaluation
intervention
plan
(problem solving)
generalising
modelling
designing
deciding
process evaluation
describing/
interpreting
action-process supporting
observing
evaluating
diagnosis
‘t Hart c.s.

33. Method The technique to gather data, related to the research question.
Surveys or interviews are not always the right techniques to answer our specific research questions.

34. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretativism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
stat analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Crotty, 1998

35. Qualitative Positivist Research versus Non-Qualitative Positivist Research
QPR Methods
Non-QPR Methods
Field experiment
Math Modeling (analytical modeling)
Lab experiment
Group feedback
Free simulation experiment
Participative research
Experimental simulation
Case study
Adaptive experiment
Philosophical research
Field study
Opinion research
Archival research
Table 1. QPR versus Non-QPR Methods (Click on the method for its definition)

36. Type of Research, General Research Approaches,
Data Collection Techniques, & Data Analysis Techniques

37. Participatory mindset
Design-Led
Design-Led
Critical
Design
Probes
generative tools
Design and Emotion
User-centered
Design
Participatory
Design
Expert mindset
contextual enquiry
Participatory mindset
Lead-user inovation
Dutch/Scandinavian design
Usability testing
applied
ethnography
Human factors and ergonomics
Sanders, 2002
Research-Led
Research-Led

38. Participatory mindset
Design-Led
Design-Led
Critical
Design
Probes
generative tools
Design and Emotion
User-centered
Design
Participatory
Design
Expert mindset
contextual enquiry
Participatory mindset
Lead-user inovation
Dutch/Scandinavian design
Usability testing
applied
ethnography
Human factors and ergonomics
Sanders, 2002
Research-Led
Research-Led

39. Research background Epistemology Theoretical perspective Methodology
objectivism
subjectivism
Theoretical perspective
positivism
Interpretativism
symbolic interactionism
phenomenology
hermeneutics
feminism
(post)modernism
Methodology
experimental
descriptive
survey
ethnography
heuristic
action research
discourse anal.
evaluation
Methods
scaling
questionnaires
observation
interview
focus group
case study
narratives
ethnographic
statistic. analysis
data reduction
cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Modelling is not a research method, it’s a tool. Testing the tool is about research
Crotty, 1998

40. Definitions
‘Research’ = the systematic inquiry to the end of gaining new knowledge
a ‘researcher’ = a person who pursues research (e.g., in design).
Practice’ = professional practice (e.g., in design) or to processes usually used in professional practice to produce professional work for any purpose other than the (deliberate) acquisition of knowledge.
‘Practitioner’ = anyone who works in professional practice.

41. Process (design methodology)
Design Knowledge
Process (design methodology)
First resides in people, especially designers. So, it’s obvious to study design ability/expertise + how they learn. Second, it resides I the process: tactics and strategies, and the techniques they use: methodology. Third, design knowledge is in the product: in one system relating the three dimensions Man – Artefact – Ambiance. 
product
people
designers

42. Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design

43. Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design.
Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.

44. Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design
Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.
The product dimension asks for forms and materials, and finishes with the embodiment of design attributes: both the intentional world (teleological and functional –wishes and needs–) in relation with the principal, partial and elementary function and the man’s connection with the systemic formal and material part (structure, organization, parts and connections).

45. Design Research Terry Love’s view:
Design Research is dominated by two contradicting incompatible approaches:
Scientific: design can be completely understood
Interpretive: design is an ‘intuitive’ activity, dependent on creativity, and scientifically inaccessible
The approaches are epistemologically and practically contradictory in that scientific empiricism and interpretivistic exploration regard each other’s central assumptions as invalid.
Empirical scientific research specifically excludes subjective reporting as reliable evidence.
Interpretive approaches deny that the scientific empirical approach addresses the central target of design research – the human internal creative design activities

46. Design Research Scientific Interpretive Theoretical perspective Focus
Scientific, usually based on physics
Interpretive, focusing on individuals’ experiences, their construction of understanding, perceptions and interpretation of reality. Often centres on individual creativity and subjective perceptions relating to being creative.
Focus
Empirical realities of the design processes, design objects, design brief and contexts.
The core concept of ‘design’ is defined in terms of these activities.
Experiences of designers and other design constituents. Tries to identify form of internal creative design activities from observation of externalities.
Typically defines design in terms of creativity, art, individual genius and socio-cultural influences

47. Design Research Scientific Interpretive View of Design
Design is a process.
May or may not include creativity.
Intuitive, involving hidden aspects of human subjective thinking and affective activity.
View of creativity
‘Something, or a specification for something, is “created”’.
Creation can be achieved mechanically, by automation or intuitively.
Human internal activity that results in ideas for new, unusual, highly valued, never before created things, emerging ‘magically’ from the genius of designers.
Focus on ‘individual creativity’ attributed to specific ‘designers’ and socio-cultural influences.

48. Design Research Scientific Interpretive Data collection
Similar to physics and natural sciences.
Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes self reporting data collection.
Analysis methods
Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes reflective analysis of self reports and self perception.
Knowledge focus
Discipline specific empirical information (along with)
elicited representations of tacit information and data that designers use.
Tacit and embodied skills of designers and users.
Culturally-determined knowledge.
Embedded meanings.

49. Scientific Interpretive Strengths
1. Techniques to investigate phenomena in ways that are transparent, repeatable, testable, and verifiable.
2. Research methods are expressed in a formal language that enables precise critique of the data collection techniques, methods of analysis, processes that lead to abstractions, and the theory abstractions and conclusions.
3. Correspondence between characteristics of phenomena and the formal defined symbolic language of concepts and operations in which mathematically theories and representations of the phenomena are expressed.
1. Focus on human considerations, such as the human creative aspects of design, and how users and other interpret designed outcomes.
2. Interpretive methods give space for designers and users to explain, in their own words, and from their own perspectives, how they design and use designed outcomes and how they communicate with others about designs.
3. Interpretive methods also allow exploration of opinions of users about cultural aspects of particular designs.
4. The interpretive approach can be extended to draw strength from the use of large data sets by which correlations and measures of confidence in them can be established between individuals’ ‘stories’ and the phenomena being studied.

50. Scientific Interpretive Weaknesses
Scientific empirical method does not adequately address human subjective, interpretive and experiential phenomena except via physiological substrates.
Main weakness is lack of reliability of individuals’ evidence, perceptions and interpretations i.e. lack of correlation between what people say and reality.
Evidence of this problem in studies of e.g. witness testimony, reliability of memory, relationships between reported thoughts and physiological evidence, influence of subconscious ‘thinking’, mental illusions and delusions in normal people.
‘False consciousness’: people’s representations of themselves are inaccurate or simply wrong.
Extends to individuals descriptions of processes, and the social activities that they undertake.

51. Scientific Interpretive Contradictions
There is an incompatibility between scientific modelling of design process and inclusion of a process element ‘create a new solution’ as a subjective human activity.
Claims that all sub-fields of design are incommensurate as they use different knowledge (and that the broader field of design is fundamentally fragmented) is at odds with scientific representation of designers working across disciplines and in multi-cross- and trans-disciplinary teams.
There is tension between interpretive approaches that focus on experiential subjective phenomenological aspects of human creative design activity and the frequent shift of emphasis onto aspects of design and creative activity that are more accessible empirically using a physical science approach.
There is an epistemological inconsistency in claims that Design exists of itself as a phenomena capable of creative agency and action.

52. Design Research Love’s proposal:
a unified basis for design theory bridging these two incompatible approaches.
Advantages
It provides a coherent epistemological basis for new theories
It recasts prior research and theory within a justified integrated framework with a clear epistemology and ontology.
This in turn provides the basis for developing a design field.

53. Foundations for a unified basis
Designs (i.e. the specification for creating or doing something)
Designed outcomes (after they are manufactured/actualised)
Design activity
Design processes
The skills of designers
The role of design activity
Cognitive design processes
Behaviour of designers as individuals and in social groups
Combinations of the above

54. Epistemologies Assumptions for Qualitative and Quantitative Research

55. Deductive logic of quantitative research
Researcher tests or verifies a theory
Researcher tests hypotheses
or research questions
Researcher defines and operationalizes
variables derived from the theory
Researcher measures or observes
variables using an instrument
to obtain scores
Creswell, 2003

56. Inductive logic of qualitative research
Generalizations or theories
to past experiences and literature
Researcher looks for broad patterns.
Generalizations or Theories from
Themes or Categories
Researcher analyze data to
form themes or categories
Researcher asks open-ended questions
of participants or records field-notes
Researcher gathers information
e.g. interviews, observations
Creswell, 2003

57. Qualitative vs Quantitative
General Laws
Test Hypotheses
Predict behavior
Outsider-Objective
Structured
formal measures
probability samples
statistical analysis
Qualitative
Unique/Individual case
Understanding
Meanings/Intentions
Insider-Subjective
Unstructured
open ended measures
judgement samples
interpretation of data
Purpose
Perspective
Procedures

58. Qualitative Research Triangulation
By using several data collecting methods – field notes, interviews, narratives – a complete picture of the phenomenon can be provided

59. Interpretation: observation of species- a - -
Difficulty in research with interpreting the results of a study. Even in ‘hard’ sciences.
Example:
Geologists have found fossiles in several layers of the surface of the earth.
This is the pattern they find.

60. Interpretation - a - b - - c - d -
Three groups of geologists (b, c and d) come up with an explanatory model for the findings of the geologists (a).
Different interpretations can be based on the same empirical data.

61. Interpretation: observation of discourse
J (reading) pack is firmly attached to the bike positioning of the backpack was alright fact that the centre of gravity of the backpack is placed rather far to the back of the bike (inaudible)
I do we have any … em...
J there's a problem with potholes .. the backpack tends to slide up and down which adversely influences stability I guess when you hit bumps
I isn't that in the negative?
J mm yeah well the product was considered ugly well that's solvable (laughter) we can fix that one if nothing else ... it takes a while to get used to cycling with this weight; mistakes are made attaching the fastening device to the bike so it has to be easy to attach
K with only one yeah gotta be fool proof so that's part of our
J yeah that should be in our spec
K functional spec
For qualitative data such as ‘thinking aloud’ protocol, the interpretation is even harder.

62. The role of interpretation
Gap between objects and our representations, in 3 forms ('methodological horrors', Woolgar '88):
1. Indexicality
2. Inconcludability
3. Reflexivity

63. THANK YOU!