Presentation is loading. Please wait.

Presentation is loading. Please wait.

Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language.

Similar presentations


Presentation on theme: "Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language."— Presentation transcript:

1 Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language Sciences University College London

2 Evidential reasoning  How do people assess and combine evidence to make decisions? –Legal, Medical, Financial, Social …  Cognitive science approach –What kinds of representations? –What kinds of inference patterns?  How do these compare with normative or formal methods of evidential reasoning? – Bayesian networks now used to model complex forensic evidence (Taroni et al, 2006)

3 Reasoning with legal evidence Legal domain Legal domain –E.g. juror, judge, investigator, media Complex bodies of interrelated evidence Complex bodies of interrelated evidence –Forensic evidence; witness testimony; alibis; confessions etc Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect) Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

4 Story model (Pennington & Hastie, 1986, 1991, 1992)  Evidence evaluated through story construction –‘Stories involve human action sequences in which relationships of physical causality and intentional causality between events are central’  Jurors use prior causal knowledge and expectations about story structure to fill in gaps in evidence  Active ‘sense-making’ process to construct an account of what happened

5 Evidential Reasoning  Reasoning from evidence –Use the evidence to construct ‘most plausible’ account of what happened –Generate a causal story based on the evidence  Reasoning about evidence –Assessing the strength/reliability/vali dity of the evidence –How well does the evidence support the putative hypotheses/stories? think-aloud protocols from jurors in simulated trials suggest that they predominantly engage in former

6 Continuum?  Individual variability in competence at juror reasoning (Kuhn et al., 1994) and evidential reasoning in general (Kuhn, 1991) SATISFICING Construct single story using evidence SATISFICING Construct single story using evidence THEORY-EVIDENCE CO-ORDINATION Construct multiple stories Evaluate against evidence and alternatives THEORY-EVIDENCE CO-ORDINATION Construct multiple stories Evaluate against evidence and alternatives Group deliberation helps shift ---> Requires ability to reflect on one’s own reasoning?

7 Stories: blessing or curse?  Story is concrete and categorical  Describes a singular causal process  Hard to simultaneously compare/evaluate multiple stories (cf Wigmore)  Danger of neglecting alternative accounts  Evidence often gathered/interpreted for a single story (confirmation bias)  The ‘truth’ might not make a good story  Economy of representation  Easy to communicate  Clear-cut basis for decision and action  Identify key variables to blame

8 Binocular rivalry Switch between two coherent percepts (green vs red) Even when inputs are mixed Switch between two coherent stories (prosecution vs defence) Even when evidence is mixed

9 Sally Clark case  Sally & Stephen Clark married, both solicitors  Son Christopher born in 1996 –Died suddenly at home aged 11 weeks –Sally alone with child; noticed he was unwell; ambulance called, but he could not be resuscitated  Postmortem (Dr Williams): –Death from natural causes - lung infection (and bruises consistent with resuscitation attempts) –Body was cremated

10 Sally Clark case  Harry born in 1997 –Died suddenly at home aged 8 weeks –Stephen at home with Sally; but Sally alone with child when discovered unwell; ambulance called, but he could not be resuscitated  Postmortem (Dr Williams): –Suspicious - death from shaking? –Re-examined death of Christopher –Concluded it too was unnatural, with evidence of smothering  Sally Clark charged with murder of both children

11 Prosecution case  Christopher & Harry were smothered –Nb change from Dr Williams’ initial claims of shaking for Harry (error in diagnosis of retinal haemorrhages)  Neither died from SIDS because there were unexplained injuries  Numerous similarities between the two deaths –‘which would make it an affront to commonsense to conclude that either death was natural, and it was beyond coincidence for history to so repeat itself’

12 Prosecution case  ‘Similarities’ –Babies died at similar ages –Both found unconscious in same room; at same time; shortly after feed –Mother alone with child when found unwell –Father either away or due to go away –(Medical evidence of previous abuse & deliberate injury)  How unlikely are these given that mother is innocent? (beyond coincidence?)

13 Prosecution case: Injuries to Christopher Blood in lungs Torn frenulum Bruises Smothering nosebleed Prior smothering Both fresh & older blood Between lip and jaw Small marks on arms and legs

14 Prosecution case: Injuries to Harry Old fracture & dislocation Hypoxic damage to brain Haemorrhage s in eyelids Haemorrhage s to eyes Smothering Shaking/ prior abuse Rib injuries Spinal injuries Spinal bleeding & swollen cord

15 Prosecution case: Credibility of witnesses Sally Clark states she found Harry slumped in bouncy chair Harry slumped in bouncy chair? Police surgeon says impossible for baby of 8 weeks to slump in bouncy chair Sally Clark reliability Sally’s testimony in doubt

16 Sally Clark alone with Harry Stephen Clark states he returned home at 5.30/5.45pm Taxi records show Stephen Clark returned home at 8.10pm Opportunity / Motive Sally Clark smothered Harry Stephen Clark reliability Stephen lying to protect wife Stephen’s testimony in doubt

17  Professor Sir Roy Meadow (Paediatrics)  Report – ‘Sudden unexpected deaths in infancy’  Risk factors – age of mother (<26), smoker in household, no wage earner  None applied to Clark family  Chance of one SIDS in family= 1 in 8,543  Chance of two SIDS = 1/8543 x 1/8543 = 1/73 million  ‘…by chance that happening will occur about once every hundred years’ Prosecution case: Statistical evidence

18 Defence case  Sally Clark did not kill her children –They died of natural but unexplained causes –Medical evidence amounts only to suspicion  Two of prosecution experts said cause of deaths ‘unascertained’  Case hinges upon Dr William’s reliability and competence

19 Blood in lungs Torn frenulum Bruises Resuscitation attempts Resuscitation attempts Reliability of Dr Williams Postmortem effects Report of Torn frenulum Report of Bruises Haemoderosis Report from police & hospital Change of opinion Poor conduct of postmortem Low quality photos etc Change of opinion Poor conduct of postmortem Low quality photos etc Defence case: Injuries to Christopher NB distinguish event from reports of event

20 Defence case: Injuries to Harry Hypoxic damage to brain Haemorrhage s to eyelids Haemorrhage s to eyes Natural causes post-death Natural causes post-death Postmortem Rib injuries Spinal injuries Reliability of Dr Williams Change of opinion Prior error with slides Change of opinion Prior error with slides

21 Stephen Clark states he returned home at 5.30/5.45pm Sally Clark alone with Harry Stephen Clark reliability Taxi records show Stephen Clark returned home at 8.10pm Opportunity / Motive Sally Clark smothered Harry Stephen very unlikely to lie to protect wife if she killed their children Stephen admitted lack of knowledge, and mentioned taxi records Explain Stephen testimony mistake

22 Defence case: Statistical evidence 2 SIDS death = significantly greater than 1/73 million Mother >26 No smokers Wage earner Mother >26 No smokers Wage earner SIDS death1 SIDS death2 genetic or environmental factors Calculation for two deaths ignores possible genetic & environmental factors Estimate for probability of one SIDS death questionable Known risk factors UNKNOWN risk factors Deaths are not independent (so cannot simply square)

23 Verdict  Sally Clark found guilty by 10-2 majority  Imprisoned for life

24 Statistical evidence misleading First Appeal: Statistical evidence misleading  Non independence –1/73 million figure flawed –Probabilities are not independent  Relevance –Probability of two SIDS deaths insufficient –needs to be compared against probability that mother murders both her children –Estimated incidence of this is much lower than of two SIDS deaths ‘it is clearly inadequate to concentrate on a single cause of death. If we make an assessment of the probability of two babies in one family both dying from SIDS, we must equally make a similar assessment of the probability of two babies in one family both being murdered (and so on, for any other causes that may be under consideration)…’ Dawid (2002)

25 Evidence SIDS Murder  Two alternative causes of the deaths (exclusive but not exhaustive – other causes possible, also possible that one SIDS, one murdered etc) Prior probability of SIDS is low Prior probability of murder is even lower Prior to other/medical evidence, probability of double SIDS greater than probability of double murder Evidence of 2 deaths

26 Appeal dismissed  Court of appeal judgment –No need for expert statisticians to give oral testimony –“it was hardly rocket science” –Defence already pointed out flaws in statistics –What matters is that probability of two SIDS deaths is very low, not exact figure –Statistic might have had larger impact on jury than it should have, but case against Sally Clark was nevertheless overwhelming  "In the context of the trial as a whole, the point on statistics was of minimal significance and there is no possibility of the jury having been misled so as to reach verdicts that they might not otherwise have reached."

27 Second appeal  Discovery of new evidence –Harry had bacterial infection –Known by Dr Williams but not disclosed at trial! –(When jury asked about blood tests for Harry, Williams said no relevant test results)  Plausible cause of Harry’s death –according to 11 independent experts –Also casts doubt on Christopher’s death due to unreliability of Dr Williams

28 Harry’s death Hypoxic damage to brain Hemorrhages to eyelids Hemorrhages to eyes Natural causes post-death Natural causes post-death Postmortem Rib injuries Spinal injuries Reliability of Dr Williams Failure to disclose etc Failure to disclose etc Bacterial infection Micro- biological tests Conclusions about Christopher

29 Second appeal  Conviction declared unsafe –Sally Clark released 2003  Postscript –Several other similar convictions involving Meadow subsequently overturned –Meadow struck off medical register 2005; reinstated on appeal 2006 –Williams guilty of serious misconduct 2005  Sally dies 2007

30 Lessons  Various repercussions for legal domain –Expert witnesses –(expert in child health not an expert in statistics) –Interpretation and presentation of statistical evidence  For evidential reasoning –Understanding statistical evidence –Role of causal networks –Reliability of evidence (and experts) –Stories and blame

31 Statistical evidence  Well-documented problems when people reason with probabilities (Kahneman, 2012) –Base rate neglect; prosecutor's fallacy; conjunction errors  In contrast people are good at qualitative causal reasoning  One approach that reconciles these findings –People need suitable causal models for appropriate probabilistic reasoning (Krynski & Tenenbaum, 2007; Sloman, 2005; Lagnado, 2011)  Classic probability problems facilitated with causal models

32 Medical diagnosis problem (Krynski & Tenenbaum, 2007)  Given +test people grossly overestimate probability of cancer  (Neglect low base rate)  Mistaken use of false positive probability  Low false +  high probability of cancer + TEST Cancer Cyst + TEST Cancer  Alternative cause of +test made explicit  People give better estimates of probability of cancer  Improved probabilistic reasoning given suitable causal model  shown for several classic problems

33 Statistical evidence  To avoid errors in Sally Clark case –Need suitable (causal) model to understand probabilities –Need to consider (probability) of alternative causes –Need to combine via Bayes rule

34 Misleading categories  Case framed as murder vs SIDS  Exclusive but not exhaustive  Tempting to reason: not-SIDS -> murder  But other natural explanations possible (eg infections etc)  Key to represent alternative causes … Evidence Natural Unnatural Evidence other natural smother Other … SIDS

35 Non-independence  Main focus on flawed assumption of independence of SIDS deaths –Judges, lawyers, media, etc  People understand independence/non-independence when framed causally –Possible unobserved common causes of SIDS deaths –Eg genetic or environmental factors SIDS1 Genetic or environmental SIDS2

36 Understanding/using probability  Second error – –How is probability of SIDS relevant to probability that Sally is guilty of murder? –Need to use Bayes rule –Requires comparison with prior probability of child murder  Danger of prosecutor's fallacy –Assume that 1 in 73 million figure applies to probability that Sally Clark is innocent –Eg P(2deaths|not guilty) = P(not guilty|2deaths)

37 Statistical evidence  Probabilistic reasoning improved by explicit causal models (Krynski &Tenenbaum,2007)  Avoid Meadow’s second error by explicitly representing probability of double murder? Evidence SIDS Murder Prior of double SIDS is low Prior for double murder is even lower Ongoing empirical work on improving Bayesian reasoning using causal models Representing alternative cause and its prior probability should improve probabilistic judgments

38 Causal networks  Key role of causal reasoning borne out by Sally Clark case  But story model needs to be developed  Formal means for representing causal models and inference  Include representation of evidence and reliability (and their interrelations)  Move closer to theory-evidence co-ordination  Even if people don’t always do this- they can!

39 Legal idioms  Evidential reasoning in terms of causal building blocks –Capture generic inference patterns –Reusable and combinable –Qualitative causal structure –Based on Bayesian networks –Akin to schema/scripts Fenton, Lagnado & Neil, 2012

40 Legal idioms  Evidence idiom  Evidence depends on Hypothesis  Evidence is more likely if hypothesis is true  Observed evidence raises the probability of hypothesis Bruises Smother Evidence Hypothesis Smothering causes bruises (probabilistically)

41 Explaining away Explaining away –Evidence is often rebutted Legal idioms Bruises Smother Resuscitation Stephen report Evidence for alternative cause of bruises

42 Legal idioms  Distinguish event from report Event Hypothesis Report Bruises Christopher smothered Williams report of bruises Police / hospital report of NO bruises

43 Legal idioms  Evidence – Reliability idiom Evidence E Hypothesis H Reliability Williams report Bruises Reliability of Williams Impact of evidence on hypothesis is modulated by its reliability Williams slide errors

44 Legal idioms  Reliability of witness reports –Separate factors for reliability Williams report Bruises Reliability of Williams Veracity Is Williams honest? Objectivity Is Williams biased? Competence Is Williams mistaken? From Schum (2001)

45 Legal idioms  Opportunity idiom Sally smothers Harry Sally alone with Harry Stephen report Reliability Opportunity is often a pre-condition of guilt

46 Legal idioms  Motive idiom Motive is typically a pre-condition of guilt Sally murders baby Sally career driven Sally resentful Letters to parents evidence Use of irony rebuttal

47 Combining idioms – alibi evidence Stephen lying to protect wife? Stephen memory error? Reliability Sally smothered Harry Sally alone with Harry Opportunity Taxi record 8.10 Stephen report 5.30 Conflicting Evidence reports

48 Status of framework  Normative –Formal model to capture appropriate probabilistic inference (and support theory-evidence co-ordination)  Descriptive –Do people’s inferences conform to the model? –Qualitatively? Quantitatively? –Empirical studies suggest good fit to qualitative patterns  Prescriptive –Guide to interpreting complex evidence and improving inference (shift towards TEC)

49 The big picture  Combining network fragments into a large-scale model

50 Key factors at trial

51 Prosecution case

52 Defence case

53 Cognitive Economy?  How do people do this? –Lab-based studies support the claim that they use idioms for small-scale problems (Lagnado, 2011; Lagnado et al., 2012) –But how does this scale-up?  Story-telling –Use of narrative to simplify? –Reasoning from but not about evidence

54 Stories and Blame  Stories constructed from causal networks  Cohesive narrative to explain events  To attribute blame for negative outcomes  But focus of stories can compromise proper theory-evidence co-ordination

55 Stories and Blame  At trial  Prosecution presented one cohesive story – Sally smothered both babies  Explains most of the medical evidence  Explains unreliability of Stephen & Sally testimony  ‘Supported’ by statistical evidence  Defence did not present one single story, but numerous disconnected pieces to explain the different injuries etc

56 Possible line of juror reasoning?  Jurors reject SIDS due to extreme rarity  Neglect low base-rate of smothering because this was never raised at trial  Accept smothering because: – it gives ‘simple’ explanation of injuries – (and explains inconsistent testimonies) –Assigns blame to someone  A ‘plausible’ story?

57  Importance of causal story that assigns blame?  At second appeal –New ‘story’ in which Harry died from infection and Dr Williams & Meadow were blamed  Aftermath & Media –Professor Meadow’s statistical errors are highlighted Stories and Blame

58  Importance of clarity in evidential reasoning –For jurors, lawyers, judges, experts, media …  How can this be improved? –Shift from single casual story to theory-evidence co- ordination –Use people’s capacity for causal reasoning to support better probabilistic inference? –Introduce formal methods eg Bayesian networks etc to help model and evaluate evidence? –Ongoing research! Lessons for evidential reasoning

59 Thank you!  Collaborators –Norman Fenton (QMUL) –Martin Neil (QMUL)  Evidence project –Philip Dawid –William Twining

60 BAYES RULE (odds version) P(2deaths|guilty) = 1 P(2deaths|~guilty) =1/73million (ignoring error of non-independence) P(guilty) = 1/84million (based on stats for double child murders – but perhaps should just consider guilty = at least one murder) P(guilty|2deaths) = Nb p/1-p = odds P = odds/(1+odds)


Download ppt "Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language."

Similar presentations


Ads by Google