Presentation is loading. Please wait.

Presentation is loading. Please wait.

Analysing and interpreting cognitive interview data: a qualitative approach.

Similar presentations


Presentation on theme: "Analysing and interpreting cognitive interview data: a qualitative approach."— Presentation transcript:

1 Analysing and interpreting cognitive interview data: a qualitative approach

2 Presentation structure background aims and objective of QT design of cognitive interviews review of methods of analysis NatCen approach issues for discussion

3 Background aims and objectives of QT does test question meet measurement objectives if not, what problems arise implications of problems for survey design of cognitive interviews probe sheet interviewing techniques sampling strategy

4 Review of methods of analysis little written on analysis of cognitive interview data two main methods cited: standardised coding scheme qualitative analysis standardised coding scheme approaches are documented (usually) qualitative analysis descriptions are often scant on detail

5 Standardised coding schemes can be used to as stand alone question appraisal tool (e.g. QAS) can incorporate some elements of behaviour coding interviewer has a problem reading the question or recording the answer focus on cognitive Q & A model comprehension/communication recall/computation bias/sensitivity (judgement issues) response category plus ‘logical issues’

6 Issues with standardised schemes PROS lend themselves to presenting data quantitatively (x Rs had this problem) perceived (by some) to be more robust process is replicable useful in cross-national/ cross cultural settings, where standardisation important CONS need a lot of detailed codes under each main heading (particularly comprehension) time consuming loose context: why did R interpret Q in that way? lend themselves to presenting data quantitatively (x Rs had this problem)

7 Qualitative approach “Just naming and classify what is out there is usually not enough. We need to understand the patterns, the recurrences, the whys. As Kaplan (1964) remarks, the bedrock of inquiry is the researcher’s quest for ‘repeatable regularities’.” Miles & Huberman (1994)

8 Approaches to qualitative analysis Ethnographic accounts detailed ‘thick’ descriptions of cultures or organisations Life histories analysed as individual cases or mined for common themes Content analysis identifies content and context of documents, often involves counting (not strictly qual) Grounded theory generates analytical categories and the links between them through an iterative process of collecting and analysing data

9 Approaches to qualitative analysis Narrative analysis examines how a story is told and the intention of the teller Conversation analysis examines the structure of (usually) naturally occurring conversations Discourse analysis focuses on how knowledge is produced through the use of language Interpretative analysis attempts to present and re-present the world of those studied, by identifying and describing substantive themes, and searching for patterns between them

10 Key stages of the analytical process Data management identifying themes sorting and reducing data Generation of findings describing classifying finding linkages and patterns identifying explanations Characteristics of ‘good’ analysis system Remains grounded in the data Transparent data reduction process Facilitates and displays ordering Permits within and between case analysis

11 Seeking wider applications Developing explanations Detecting patterns of association Establishing typologies Identifying elements & dimensions Summarising / synthesising data Sorting data Tagging data Identifying initial concepts / themes Primary data Data management Descriptive analysis Explanatory analysis The analytical hierarchy in qualitative research Data collection ?

12 Thematic analysis - purposes and principles Structured display of data by theme (Q) across all cases Creating categories and classifying data within them* Demonstrating range and diversity Using examples to illustrate and amplify Must be comprehensive Labelling and categorising must be valid

13 Carrying out thematic analysis - NatCen approach Familiarisation with data Identification of factors highlight, summarise, provisionally label Categorisation is this a different manifestation of that is this a subset of that is this of the same order as that Iterative process of refinement Start close to the data - become more abstract and interpretative Must be comprehensive Aim is analytical coherence

14 Looking for explanations Informed by: hunches and hypotheses reflections during fieldwork and analysis other research or theories Process involves: detailed within case analysis comparison between cases repeated interrogation of the data milking data moving back and forth between cases searching for rival explanations Expect multiplicity Must be comprehensive

15 Summary of NatCen approach Detailed notes made on interviews Notes reviewed Chart set up Notes charted (chart revised) Charts reviewed Data interpreted Findings emerge Recommendations made Report written

16 Example chart

17 Issues to consider better documentation of qualitative analysis approach integration of code frames within thematic approach development of best practice for analysis of cognitive data cognitive interview data as one component of testing strategy collaborate findings using other data sources (e.g. split ballot experiments)

18


Download ppt "Analysing and interpreting cognitive interview data: a qualitative approach."

Similar presentations


Ads by Google