Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language.

Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language Sciences University College London

Evidential reasoning How do people assess and combine evidence to make decisions? Legal, Medical, Financial, Social … Cognitive science approach What kinds of representations? What kinds of inference patterns? How do these compare with normative or formal methods of evidential reasoning? Bayesian networks now used to model complex forensic evidence (Taroni et al, 2006)

Reasoning with legal evidence
Legal domain E.g. juror, judge, investigator, media Complex bodies of interrelated evidence Forensic evidence; witness testimony; alibis; confessions etc Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

Story model (Pennington & Hastie, 1986, 1991, 1992)
Evidence evaluated through story construction ‘Stories involve human action sequences in which relationships of physical causality and intentional causality between events are central’ Jurors use prior causal knowledge and expectations about story structure to fill in gaps in evidence Active ‘sense-making’ process to construct an account of what happened

Evidential Reasoning Reasoning from evidence Reasoning about evidence
Use the evidence to construct ‘most plausible’ account of what happened Generate a causal story based on the evidence Reasoning about evidence Assessing the strength/reliability/validity of the evidence How well does the evidence support the putative hypotheses/stories? think-aloud protocols from jurors in simulated trials suggest that they predominantly engage in former

Continuum? Individual variability in competence at juror reasoning (Kuhn et al., 1994) and evidential reasoning in general (Kuhn, 1991) SATISFICING Construct single story using evidence THEORY-EVIDENCE CO-ORDINATION Construct multiple stories Evaluate against evidence and alternatives Requires ability to reflect on one’s own reasoning? Group deliberation helps shift --->

Stories: blessing or curse?
Story is concrete and categorical Describes a singular causal process Economy of representation Easy to communicate Clear-cut basis for decision and action Identify key variables to blame Hard to simultaneously compare/evaluate multiple stories (cf Wigmore) Danger of neglecting alternative accounts Evidence often gathered/interpreted for a single story (confirmation bias) The ‘truth’ might not make a good story

Binocular rivalry Switch between two coherent percepts (green vs red)
Switch between two coherent stories (prosecution vs defence) Even when inputs are mixed Even when evidence is mixed

Sally Clark case Sally & Stephen Clark married, both solicitors
Son Christopher born in 1996 Died suddenly at home aged 11 weeks Sally alone with child; noticed he was unwell; ambulance called, but he could not be resuscitated Postmortem (Dr Williams): Death from natural causes - lung infection (and bruises consistent with resuscitation attempts) Body was cremated

Sally Clark case Harry born in 1997 Postmortem (Dr Williams):
Died suddenly at home aged 8 weeks Stephen at home with Sally; but Sally alone with child when discovered unwell; ambulance called, but he could not be resuscitated Postmortem (Dr Williams): Suspicious - death from shaking? Re-examined death of Christopher Concluded it too was unnatural, with evidence of smothering Sally Clark charged with murder of both children

Prosecution case Christopher & Harry were smothered
Nb change from Dr Williams’ initial claims of shaking for Harry (error in diagnosis of retinal haemorrhages) Neither died from SIDS because there were unexplained injuries Numerous similarities between the two deaths ‘which would make it an affront to commonsense to conclude that either death was natural, and it was beyond coincidence for history to so repeat itself’

Prosecution case ‘Similarities’
Babies died at similar ages Both found unconscious in same room; at same time; shortly after feed Mother alone with child when found unwell Father either away or due to go away (Medical evidence of previous abuse & deliberate injury) How unlikely are these given that mother is innocent? (beyond coincidence?)

Prosecution case: Injuries to Christopher
Blood in lungs Torn frenulum Bruises Smothering nosebleed Prior smothering Between lip and jaw Both fresh & older blood Small marks on arms and legs

Prosecution case: Injuries to Harry
Hypoxic damage to brain Haemorrhages in eyelids Haemorrhages to eyes Smothering Shaking/ prior abuse Rib injuries Spinal injuries Old fracture & dislocation Spinal bleeding & swollen cord

Prosecution case: Credibility of witnesses
Sally’s testimony in doubt Sally Clark reliability Harry slumped in bouncy chair? Sally Clark states she found Harry slumped in bouncy chair Police surgeon says impossible for baby of 8 weeks to slump in bouncy chair

Stephen’s testimony in doubt
Sally Clark smothered Harry Stephen lying to protect wife Opportunity/ Motive Stephen Clark reliability Sally Clark alone with Harry Stephen Clark states he returned home at 5.30/5.45pm Taxi records show Stephen Clark returned home at 8.10pm

Prosecution case: Statistical evidence
Professor Sir Roy Meadow (Paediatrics) Report – ‘Sudden unexpected deaths in infancy’ Risk factors – age of mother (<26), smoker in household, no wage earner None applied to Clark family Chance of one SIDS in family= 1 in 8,543 Chance of two SIDS = 1/8543 x 1/8543 = 1/73 million ‘…by chance that happening will occur about once every hundred years’

Defence case Sally Clark did not kill her children
They died of natural but unexplained causes Medical evidence amounts only to suspicion Two of prosecution experts said cause of deaths ‘unascertained’ Case hinges upon Dr William’s reliability and competence

Defence case: Injuries to Christopher
Haemoderosis Resuscitation attempts Postmortem effects NB distinguish event from reports of event Blood in lungs Torn frenulum Bruises Report from police & hospital Report of Torn frenulum Report of Bruises Change of opinion Poor conduct of postmortem Low quality photos etc Reliability of Dr Williams

Defence case: Injuries to Harry
Hypoxic damage to brain Haemorrhages to eyelids Haemorrhages to eyes Natural causes post-death Postmortem Rib injuries Spinal injuries Reliability of Dr Williams Change of opinion Prior error with slides

Explain Stephen testimony mistake
Sally Clark smothered Harry Stephen very unlikely to lie to protect wife if she killed their children Opportunity/ Motive Stephen Clark reliability Sally Clark alone with Harry Stephen admitted lack of knowledge, and mentioned taxi records Stephen Clark states he returned home at 5.30/5.45pm Taxi records show Stephen Clark returned home at 8.10pm

genetic or environmental factors
Defence case: Statistical evidence Mother >26 No smokers Wage earner SIDS death1 SIDS death2 Known risk factors Calculation for two deaths ignores possible genetic & environmental factors Estimate for probability of one SIDS death questionable genetic or environmental factors UNKNOWN risk factors Deaths are not independent (so cannot simply square) 2 SIDS death = significantly greater than 1/73 million

Verdict Sally Clark found guilty by 10-2 majority Imprisoned for life

First Appeal: Statistical evidence misleading
Non independence 1/73 million figure flawed Probabilities are not independent Relevance Probability of two SIDS deaths insufficient needs to be compared against probability that mother murders both her children Estimated incidence of this is much lower than of two SIDS deaths ‘it is clearly inadequate to concentrate on a single cause of death. If we make an assessment of the probability of two babies in one family both dying from SIDS, we must equally make a similar assessment of the probability of two babies in one family both being murdered (and so on, for any other causes that may be under consideration)…’ Dawid (2002)

Two alternative causes of the deaths
(exclusive but not exhaustive – other causes possible, also possible that one SIDS, one murdered etc) Prior probability of SIDS is low Prior probability of murder is even lower Evidence SIDS Murder Evidence of 2 deaths Prior to other/medical evidence, probability of double SIDS greater than probability of double murder

Appeal dismissed Court of appeal judgment
No need for expert statisticians to give oral testimony –“it was hardly rocket science” Defence already pointed out flaws in statistics What matters is that probability of two SIDS deaths is very low, not exact figure Statistic might have had larger impact on jury than it should have, but case against Sally Clark was nevertheless overwhelming "In the context of the trial as a whole, the point on statistics was of minimal significance and there is no possibility of the jury having been misled so as to reach verdicts that they might not otherwise have reached."

Second appeal Discovery of new evidence
Harry had bacterial infection Known by Dr Williams but not disclosed at trial! (When jury asked about blood tests for Harry, Williams said no relevant test results) Plausible cause of Harry’s death according to 11 independent experts Also casts doubt on Christopher’s death due to unreliability of Dr Williams

Harry’s death Bacterial infection Failure to disclose etc
Hypoxic damage to brain Hemorrhages to eyelids Hemorrhages to eyes Natural causes post-death Postmortem Rib injuries Spinal injuries Reliability of Dr Williams Bacterial infection Micro-biological tests Failure to disclose etc Conclusions about Christopher

Second appeal Conviction declared unsafe Postscript Sally dies 2007
Sally Clark released 2003 Postscript Several other similar convictions involving Meadow subsequently overturned Meadow struck off medical register 2005; reinstated on appeal 2006 Williams guilty of serious misconduct 2005 Sally dies 2007

Lessons Various repercussions for legal domain
Expert witnesses (expert in child health not an expert in statistics) Interpretation and presentation of statistical evidence For evidential reasoning Understanding statistical evidence Role of causal networks Reliability of evidence (and experts) Stories and blame

Statistical evidence Well-documented problems when people reason with probabilities (Kahneman, 2012) Base rate neglect; prosecutor's fallacy; conjunction errors In contrast people are good at qualitative causal reasoning One approach that reconciles these findings People need suitable causal models for appropriate probabilistic reasoning (Krynski & Tenenbaum, 2007; Sloman, 2005; Lagnado, 2011) Classic probability problems facilitated with causal models

Medical diagnosis problem (Krynski & Tenenbaum, 2007)
+ TEST Cancer Cyst + TEST Cancer Alternative cause of +test made explicit People give better estimates of probability of cancer Improved probabilistic reasoning given suitable causal model shown for several classic problems Given +test people grossly overestimate probability of cancer (Neglect low base rate) Mistaken use of false positive probability Low false +  high probability of cancer

Statistical evidence To avoid errors in Sally Clark case
Need suitable (causal) model to understand probabilities Need to consider (probability) of alternative causes Need to combine via Bayes rule

Misleading categories
Case framed as murder vs SIDS Exclusive but not exhaustive Tempting to reason: not-SIDS -> murder But other natural explanations possible (eg infections etc) Key to represent alternative causes … Evidence other natural smother Other … SIDS Evidence Natural Unnatural

Genetic or environmental
Non-independence Main focus on flawed assumption of independence of SIDS deaths Judges, lawyers, media, etc People understand independence/non-independence when framed causally Possible unobserved common causes of SIDS deaths Eg genetic or environmental factors SIDS1 Genetic or environmental SIDS2

Understanding/using probability
Second error – How is probability of SIDS relevant to probability that Sally is guilty of murder? Need to use Bayes rule Requires comparison with prior probability of child murder Danger of prosecutor's fallacy Assume that 1 in 73 million figure applies to probability that Sally Clark is innocent Eg P(2deaths|not guilty) = P(not guilty|2deaths)

Statistical evidence Probabilistic reasoning improved by explicit causal models (Krynski &Tenenbaum,2007) Avoid Meadow’s second error by explicitly representing probability of double murder? Prior of double SIDS is low Evidence SIDS Murder Prior for double murder is even lower Representing alternative cause and its prior probability should improve probabilistic judgments Ongoing empirical work on improving Bayesian reasoning using causal models

Causal networks Key role of causal reasoning borne out by Sally Clark case But story model needs to be developed Formal means for representing causal models and inference Include representation of evidence and reliability (and their interrelations) Move closer to theory-evidence co-ordination Even if people don’t always do this- they can!

Legal idioms Evidential reasoning in terms of causal building blocks
Capture generic inference patterns Reusable and combinable Qualitative causal structure Based on Bayesian networks Akin to schema/scripts Fenton, Lagnado & Neil, 2012

Legal idioms Evidence idiom Evidence depends on Hypothesis
Evidence is more likely if hypothesis is true Observed evidence raises the probability of hypothesis Evidence Hypothesis Bruises Smother Smothering causes bruises (probabilistically)

Legal idioms Explaining away Evidence is often rebutted
Bruises Smother Resuscitation Stephen report Evidence for alternative cause of bruises

Legal idioms Distinguish event from report Event Hypothesis Bruises
Christopher smothered Williams report of bruises Report Police / hospital report of NO bruises

Reliability of Williams
Legal idioms Evidence – Reliability idiom Impact of evidence on hypothesis is modulated by its reliability Evidence E Hypothesis H Reliability Williams report Bruises Reliability of Williams Williams slide errors

Reliability of Williams
Legal idioms Reliability of witness reports Separate factors for reliability Competence Is Williams mistaken? Objectivity Is Williams biased? Veracity Is Williams honest? Williams report Bruises Reliability of Williams From Schum (2001)

Legal idioms Opportunity idiom
Sally smothers Harry Sally alone with Harry Stephen report Reliability Opportunity is often a pre-condition of guilt

Legal idioms Motive idiom rebuttal evidence
Sally murders baby Sally career driven Sally resentful Use of irony rebuttal Letters to parents evidence Motive is typically a pre-condition of guilt

Combining idioms – alibi evidence
Sally smothered Harry Sally alone with Harry Opportunity Taxi record 8.10 Stephen report 5.30 Conflicting Evidence reports Stephen lying to protect wife? Reliability Stephen memory error?

Status of framework Normative Descriptive Prescriptive
Formal model to capture appropriate probabilistic inference (and support theory-evidence co-ordination) Descriptive Do people’s inferences conform to the model? Qualitatively? Quantitatively? Empirical studies suggest good fit to qualitative patterns Prescriptive Guide to interpreting complex evidence and improving inference (shift towards TEC)

The big picture Combining network fragments into a large-scale model

Key factors at trial

Prosecution case

Defence case

Cognitive Economy? How do people do this? Story-telling
Lab-based studies support the claim that they use idioms for small-scale problems (Lagnado, 2011; Lagnado et al., 2012) But how does this scale-up? Story-telling Use of narrative to simplify? Reasoning from but not about evidence

Stories and Blame Stories constructed from causal networks
Cohesive narrative to explain events To attribute blame for negative outcomes But focus of stories can compromise proper theory-evidence co-ordination

Stories and Blame At trial
Prosecution presented one cohesive story – Sally smothered both babies Explains most of the medical evidence Explains unreliability of Stephen & Sally testimony ‘Supported’ by statistical evidence Defence did not present one single story, but numerous disconnected pieces to explain the different injuries etc

Possible line of juror reasoning?
Jurors reject SIDS due to extreme rarity Neglect low base-rate of smothering because this was never raised at trial Accept smothering because: it gives ‘simple’ explanation of injuries (and explains inconsistent testimonies) Assigns blame to someone A ‘plausible’ story?

Stories and Blame Importance of causal story that assigns blame?
At second appeal New ‘story’ in which Harry died from infection and Dr Williams & Meadow were blamed Aftermath & Media Professor Meadow’s statistical errors are highlighted

Lessons for evidential reasoning
Importance of clarity in evidential reasoning For jurors, lawyers, judges, experts, media … How can this be improved? Shift from single casual story to theory-evidence co-ordination Use people’s capacity for causal reasoning to support better probabilistic inference? Introduce formal methods eg Bayesian networks etc to help model and evaluate evidence? Ongoing research!

Thank you! Collaborators Evidence project Norman Fenton (QMUL)
Martin Neil (QMUL) Evidence project Philip Dawid William Twining

P(2deaths|~guilty) =1/73million (ignoring error of non-independence)
BAYES RULE (odds version) P(2deaths|guilty) = 1 P(2deaths|~guilty) =1/73million (ignoring error of non-independence) P(guilty) = 1/84million (based on stats for double child murders – but perhaps should just consider guilty = at least one murder) Nb p/1-p = odds P = odds/(1+odds) P(guilty|2deaths) =

Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language.

Similar presentations

Presentation on theme: "Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language.

Similar presentations

Presentation on theme: "Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language."— Presentation transcript:

Similar presentations

About project

Feedback