Presentation on theme: "IdMRC Social Research Methods Autumn Lecture-Workshop Series"— Presentation transcript:
1 IdMRC Social Research Methods Autumn Lecture-Workshop Series
2 Science Henri Christiaans Aim? When is knowledge scientific knowledge? Criteria?Knowledge sources?When is research scientific research?Henri Christiaans
3 Science Realism What we observe is real Instrumentalism What we observe doesn’t need to be realSocial constructivismTheories only get meaning through social and political contextAtomen en electronen als voorbeeldSociaal constr: objectiviteit niet belangrijk
4 What is Knowledge? Justified true belief (Plato’s Theaetetus) The Greeks classify knowledge into 2 types:Doxa (believed to be true)Episteme (known to be true)Doxa EpistimeThrough Scientific process of inquiryHow do we know what we know?Define knowledge alternativelySupported by evidence (usually empirical)Conceive knowledge claims in a probabilistic senseKnowledge is a matter of societal acceptance
5 How is Knowledge Acquired? Role of science, where science is a convention, related to societal norms, expectations, values, etc.Thus, is science equals any scholarly attempt at acquiring knowledgeScience requires conventions to be followed
6 How is Knowledge Acquired? Role of science, where science is a convention, related to societal norms, expectations, values, etc.Thus, is science equals any scholarly attempt at acquiring knowledgeScience requires conventions to be followed
7 Knowledge in designImplicit prioritisation of the (language-based mode of) propositional knowledge (justified true beliefs) seems to exclude certain kinds or formats of knowledge associated with practice, which are often called practical, experiential, personal, or tacit knowledge and which evade verbal articulation.Despite of continued criticism, the definition of knowledge as ‘justified true belief’ has remained the prevailing definition, and Niedderer (2007) has shown that this understanding of propositional knowledge is implicit in the definition of research because of additional requirements such as the textual/written presentation of an intellectual position (proposition, thesis – ‘true belief’), because of the logic of verification and defence of this intellectual position through argument and evidence (justification), and the requirement for generalisability/transferability and explicit and unambiguous communication.
9 Science based on empirism Knowledge derived from how the world is experienced. Scientific statements are controlled by and derived from our experiences and observations. enScientific theories developed and tested by experiments and observations through empirical methods
10 Questions to be asked Which methods do we plan to use? Which methodology defines the use of methods?Which theoretical perspective do we start from in order to apply the right methodology?Which epistemology feeds this theoretical perspective?Therefore, I will try to bring structure, first to link the the relevant questions which have to be asked before we start the actual study.
11 Ontology1. A systematic account of Existence. Nature of the world around us.2. (From philosophy) An explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them.3. The hierarchical structuring of knowledge about things by subcategorising them according to their essential (or at least relevant and/or cognitive) qualities.
12 Epistemology and ontology The way of understanding and interpreting how we know what we know.Particular methodologies tend to entail (subscribe to) particular epistemologies and, in their turn, particular forms of ontology
13 Ontology in Computing Terms For AI systems, what "exists" is that which can be represented.We can describe the ontology of a program by defining a set of representational terms. Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, an ontology is the statement of a logical theory.A set of agents that share the same ontology will be able to communicate about a domain of discourse without necessarily operating on a globally shared theory. The idea of ontological commitment is based on the Knowledge-Level perspective.Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms.
14 EpistemologyFrom the Greek words episteme (knowledge) and logos (word/speech) is the branch of philosophy that deals with the nature, origin and scope of knowledge.Refers to our theory of knowledge, in particular, how we acquire knowledge (Hirschheim, 1992).
15 Research background Epistemology Theoretical perspective Methodology objectivismsubjectivismTheoretical perspectivepositivismInterpretativismsymbolic interactionismphenomenologyhermeneuticsfeminism(post)modernismSocial-constructivismMethodologyexperimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluationMethodsscalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstat analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.Modelling is not a research method, it’s a tool. Testing the tool is about researchCrotty, 1998
16 Research background Epistemology Theoretical perspective Methodology objectivismsubjectivismTheoretical perspectivepositivismInterpretativismsymbolic interactionismphenomenologyhermeneuticsfeminism(post)modernismSocial-constructivismMethodologyexperimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluationMethodsscalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstat analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.Reality. Ho do we know what we know?
17 Theoretical perspective Philosophical point of view which feeds the methodology and offers a context for the process and the logics, and gives our criteria a basis.Cultural differences play a roleColombia:Oslo: research on design education, describing cases from different countries: emancipation: making people aware of design issues.
19 Three Main Epistemologies PositivistInterpretivistCritical
20 Interpretivism Interpretivism rests upon idealism: the world is interpreted through the mind; e.g., classificatory schemes of species;the social world cannot be described without investigating how people use language and symbols to construct what social practices; i.e., understand their experience;the social world becomes the creation of the purposeful actions of conscious agents; andno social explanation was complete unless it could adequately describe the role of meanings in human actionsActions are not governed by discrete patterns of cause and effect (as in positivism), but by rules that social actors use to interpret the world
21 Positivist Science 5 Pillars Unity of scientific method Causal RelationshipsEmpiricismScience and its process is Value-FreeFoundation of science is based on logic and maths
22 Ontology of Positivism RealismUniverse comprised of objectively given, immutable objects and structures, existing as empirical entities, on their own, independent of the observer’s appreciation of them.Contrasts with relativism or instrumentalism, where reality is a subjective construction of the mind, thus varying with different languages and cultures.While hugely successful in physical sciences, it is not as successful for social science.
23 Anti-Positivism Latter part of 19th century Man as an actor could not be studied through the methods of natural sciences that focus on establishing general laws. In the cultural sphere man is free (Burrell and Morgan, 1979)
24 Post-PositivismBased on the concept of critical realism, that there is a real world out there independent of our perception of it and that the objective of science is to try and understand it,combined with triangulation, i.e., the recognition that observations and measurements are inherently imperfect and hence the need to measure phenomena in many ways.The post-positivist epistemology regards the acquisition of knowledge as a process that is more than mere deduction. Knowledge is acquired through both deduction and induction.
25 Rational Solving Problem Paradigm Reflection in Action Paradigm Simon versus SchonDesignerDesignerobjective AnalysisObjective Analysissubjective Interpretationdesign Problemdesign SolutionDesign Task(= problem + situation+ teime)design SolutionRational Solving Problem ParadigmReflection in Action ParadigmRationalist RootConstructivist RootPOSITIVISMPHENOMENOLOGY
26 MethodologyOur strategy and action plans, the design process which defines what specific methods we will choose
28 Types of Research Analytical Historical Philosophical Literature study Meta-analysisDescriptive Survey (questionnaire, interview)Case study Task analysisDocument analysis Correlation anal.Observation EtnographicsExplorative Survey CorrelationalCase study ExperimentalExperimental Pre-experimentalTrue-experimentalQuasi-experimental
30 Types of research methods empiricalparticipatoryquantitativeinductiveprescriptiveidiographicnomotheticdescriptivedeductiveunbiasedqualitativerational
31 Fundamental Research: the Empirical cycle testingdeductionpredictiontheorygeneralisingmodellingExplaining/interpretingevaluatingdescribing/evaluationspecifyinginductionhypothesesknowledge problem‘t Hart c.s.
32 Practice oriented Research: The regulative cycle problem from practiceevaluationinterventionplan(problem solving)generalisingmodellingdesigningdecidingprocess evaluationdescribing/interpretingaction-process supportingobservingevaluatingdiagnosis‘t Hart c.s.
33 Method The technique to gather data, related to the research question. Surveys or interviews are not always the right techniques to answer our specific research questions.
35 Qualitative Positivist Research versus Non-Qualitative Positivist Research QPR MethodsNon-QPR MethodsField experimentMath Modeling (analytical modeling)Lab experimentGroup feedbackFree simulation experimentParticipative researchExperimental simulationCase studyAdaptive experimentPhilosophical researchField studyOpinion researchArchival researchTable 1. QPR versus Non-QPR Methods (Click on the method for its definition)
36 Type of Research, General Research Approaches, Data Collection Techniques, & Data Analysis Techniques
37 Participatory mindset Design-LedDesign-LedCriticalDesignProbesgenerative toolsDesign and EmotionUser-centeredDesignParticipatoryDesignExpert mindsetcontextual enquiryParticipatory mindsetLead-user inovationDutch/Scandinavian designUsability testingappliedethnographyHuman factors and ergonomicsSanders, 2002Research-LedResearch-Led
38 Participatory mindset Design-LedDesign-LedCriticalDesignProbesgenerative toolsDesign and EmotionUser-centeredDesignParticipatoryDesignExpert mindsetcontextual enquiryParticipatory mindsetLead-user inovationDutch/Scandinavian designUsability testingappliedethnographyHuman factors and ergonomicsSanders, 2002Research-LedResearch-Led
39 Research background Epistemology Theoretical perspective Methodology objectivismsubjectivismTheoretical perspectivepositivismInterpretativismsymbolic interactionismphenomenologyhermeneuticsfeminism(post)modernismMethodologyexperimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluationMethodsscalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstatistic. analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.Modelling is not a research method, it’s a tool. Testing the tool is about researchCrotty, 1998
40 Definitions‘Research’ = the systematic inquiry to the end of gaining new knowledgea ‘researcher’ = a person who pursues research (e.g., in design).Practice’ = professional practice (e.g., in design) or to processes usually used in professional practice to produce professional work for any purpose other than the (deliberate) acquisition of knowledge.‘Practitioner’ = anyone who works in professional practice.
41 Process (design methodology) Design KnowledgeProcess (design methodology)First resides in people, especially designers. So, it’s obvious to study design ability/expertise + how they learn. Second, it resides I the process: tactics and strategies, and the techniques they use: methodology. Third, design knowledge is in the product: in one system relating the three dimensions Man – Artefact – Ambiance. productpeopledesigners
42 Design knowledgeDesign knowledge resides firstly in people: in designers especially. Therefore, we study human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design
43 Design knowledgeDesign knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design.Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.
44 Design knowledgeDesign knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to designDesign knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.The product dimension asks for forms and materials, and finishes with the embodiment of design attributes: both the intentional world (teleological and functional –wishes and needs–) in relation with the principal, partial and elementary function and the man’s connection with the systemic formal and material part (structure, organization, parts and connections).
45 Design Research Terry Love’s view: Design Research is dominated by two contradicting incompatible approaches:Scientific: design can be completely understoodInterpretive: design is an ‘intuitive’ activity, dependent on creativity, and scientifically inaccessibleThe approaches are epistemologically and practically contradictory in that scientific empiricism and interpretivistic exploration regard each other’s central assumptions as invalid.Empirical scientific research specifically excludes subjective reporting as reliable evidence.Interpretive approaches deny that the scientific empirical approach addresses the central target of design research – the human internal creative design activities
46 Design Research Scientific Interpretive Theoretical perspective Focus Scientific, usually based on physicsInterpretive, focusing on individuals’ experiences, their construction of understanding, perceptions and interpretation of reality. Often centres on individual creativity and subjective perceptions relating to being creative.FocusEmpirical realities of the design processes, design objects, design brief and contexts.The core concept of ‘design’ is defined in terms of these activities.Experiences of designers and other design constituents. Tries to identify form of internal creative design activities from observation of externalities.Typically defines design in terms of creativity, art, individual genius and socio-cultural influences
47 Design Research Scientific Interpretive View of Design Design is a process.May or may not include creativity.Intuitive, involving hidden aspects of human subjective thinking and affective activity.View of creativity‘Something, or a specification for something, is “created”’.Creation can be achieved mechanically, by automation or intuitively.Human internal activity that results in ideas for new, unusual, highly valued, never before created things, emerging ‘magically’ from the genius of designers.Focus on ‘individual creativity’ attributed to specific ‘designers’ and socio-cultural influences.
48 Design Research Scientific Interpretive Data collection Similar to physics and natural sciences.Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes self reporting data collection.Analysis methodsDrawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes reflective analysis of self reports and self perception.Knowledge focusDiscipline specific empirical information (along with)elicited representations of tacit information and data that designers use.Tacit and embodied skills of designers and users.Culturally-determined knowledge.Embedded meanings.
49 Scientific Interpretive Strengths 1. Techniques to investigate phenomena in ways that are transparent, repeatable, testable, and verifiable.2. Research methods are expressed in a formal language that enables precise critique of the data collection techniques, methods of analysis, processes that lead to abstractions, and the theory abstractions and conclusions.3. Correspondence between characteristics of phenomena and the formal defined symbolic language of concepts and operations in which mathematically theories and representations of the phenomena are expressed.1. Focus on human considerations, such as the human creative aspects of design, and how users and other interpret designed outcomes.2. Interpretive methods give space for designers and users to explain, in their own words, and from their own perspectives, how they design and use designed outcomes and how they communicate with others about designs.3. Interpretive methods also allow exploration of opinions of users about cultural aspects of particular designs.4. The interpretive approach can be extended to draw strength from the use of large data sets by which correlations and measures of confidence in them can be established between individuals’ ‘stories’ and the phenomena being studied.
50 Scientific Interpretive Weaknesses Scientific empirical method does not adequately address human subjective, interpretive and experiential phenomena except via physiological substrates.Main weakness is lack of reliability of individuals’ evidence, perceptions and interpretations i.e. lack of correlation between what people say and reality.Evidence of this problem in studies of e.g. witness testimony, reliability of memory, relationships between reported thoughts and physiological evidence, influence of subconscious ‘thinking’, mental illusions and delusions in normal people.‘False consciousness’: people’s representations of themselves are inaccurate or simply wrong.Extends to individuals descriptions of processes, and the social activities that they undertake.
51 Scientific Interpretive Contradictions There is an incompatibility between scientific modelling of design process and inclusion of a process element ‘create a new solution’ as a subjective human activity.Claims that all sub-fields of design are incommensurate as they use different knowledge (and that the broader field of design is fundamentally fragmented) is at odds with scientific representation of designers working across disciplines and in multi-cross- and trans-disciplinary teams.There is tension between interpretive approaches that focus on experiential subjective phenomenological aspects of human creative design activity and the frequent shift of emphasis onto aspects of design and creative activity that are more accessible empirically using a physical science approach.There is an epistemological inconsistency in claims that Design exists of itself as a phenomena capable of creative agency and action.
52 Design Research Love’s proposal: a unified basis for design theory bridging these two incompatible approaches.AdvantagesIt provides a coherent epistemological basis for new theoriesIt recasts prior research and theory within a justified integrated framework with a clear epistemology and ontology.This in turn provides the basis for developing a design field.
53 Foundations for a unified basis Designs (i.e. the specification for creating or doing something)Designed outcomes (after they are manufactured/actualised)Design activityDesign processesThe skills of designersThe role of design activityCognitive design processesBehaviour of designers as individuals and in social groupsCombinations of the above
54 Epistemologies Assumptions for Qualitative and Quantitative Research
55 Deductive logic of quantitative research Researcher tests or verifies a theoryResearcher tests hypothesesor research questionsResearcher defines and operationalizesvariables derived from the theoryResearcher measures or observesvariables using an instrumentto obtain scoresCreswell, 2003
56 Inductive logic of qualitative research Generalizations or theoriesto past experiences and literatureResearcher looks for broad patterns.Generalizations or Theories fromThemes or CategoriesResearcher analyze data toform themes or categoriesResearcher asks open-ended questionsof participants or records field-notesResearcher gathers informatione.g. interviews, observationsCreswell, 2003
57 Qualitative vs Quantitative General LawsTest HypothesesPredict behaviorOutsider-ObjectiveStructuredformal measuresprobability samplesstatistical analysisQualitativeUnique/Individual caseUnderstandingMeanings/IntentionsInsider-SubjectiveUnstructuredopen ended measuresjudgement samplesinterpretation of dataPurposePerspectiveProcedures
58 Qualitative Research Triangulation By using several data collecting methods – field notes, interviews, narratives – a complete picture of the phenomenon can be provided
59 Interpretation: observation of species -a--Difficulty in research with interpreting the results of a study. Even in ‘hard’ sciences.Example:Geologists have found fossiles in several layers of the surface of the earth.This is the pattern they find.
60 Interpretation - a - b - - c - d - Three groups of geologists (b, c and d) come up with an explanatory model for the findings of the geologists (a).Different interpretations can be based on the same empirical data.
61 Interpretation: observation of discourse J (reading) pack is firmly attached to the bike positioning of the backpack was alright fact that the centre of gravity of the backpack is placed rather far to the back of the bike (inaudible)I do we have any … em...J there's a problem with potholes .. the backpack tends to slide up and down which adversely influences stability I guess when you hit bumpsI isn't that in the negative?J mm yeah well the product was considered ugly well that's solvable (laughter) we can fix that one if nothing else ... it takes a while to get used to cycling with this weight; mistakes are made attaching the fastening device to the bike so it has to be easy to attachK with only one yeah gotta be fool proof so that's part of ourJ yeah that should be in our specK functional specFor qualitative data such as ‘thinking aloud’ protocol, the interpretation is even harder.
62 The role of interpretation Gap between objects and our representations, in 3 forms ('methodological horrors', Woolgar '88):1. Indexicality2. Inconcludability3. Reflexivity