Thinking about Evidence David Lagnado University College London

Leonard Vole accused of murdering a rich elderly lady Miss French Vole had befriended French and visited her regularly including night of murder Vole needed money French changed her will to include him; shortly after he enquired about luxury cruises Maid testified Vole was with French at time of death Blood on Voles jacket same type as French Romaine, Voles wife, was to testify that he was with her at time of murder But instead Romaine appears as witness for prosecution Testifies that Vole was not with her, returned later with blood on his jacket, and said Ive killed her Letters written by Romaine to lover – reveals her plan to lie and incriminate Vole Vole is acquitted!

Evidential reasoning How do people reason with uncertain evidence? How do they assess and combine different items of evidence? –What representations do they use? –What inference processes? How do these compare with normative theories?

Reasoning with legal evidence Legal domain –E.g. juror, judge, investigator, media Complex bodies of interrelated evidence –Forensic evidence; witness testimony; alibis; confessions etc Need to integrate wide variety of evidence to reach singular conclusion (e.g. guilt of suspect)

Descriptive models of juror reasoning Belief adjustment model (Hogarth & Einhorn, 1992) –Sequential weighted additive model –Over-weights later items –Ignores relations between items of evidence Story model (Pennington & Hastie, 1992) –Evidence evaluated through story construction –Holistic judgments based on causal models –No formal, computational or process model

Descriptive models of juror reasoning Coherence-based models (Simon & Holyoak, 2002) –Mind strives for coherent representations –Evidential elements cohere or compete –Judgments emerge through interactive process that maximizes coherence –Bidirectional reasoning (evidence can be re-evaluated to fit emerging conclusions)

How should people do it? Bayesian networks? Nodes represent evidence statements or hypotheses Directed links between nodes represent causal or evidential relations Permits inference from evidence to hypotheses (and vice-versa) GuiltMaid Vole is guilty Maid testifies that Vole was with Miss French Blood Blood on Voles cuffs Cut Vole cut wrist slicing ham

Partial BN of Witness for Prosecution

Partial Bayesian net for Sacco and Vanzetti trial

Applicable to human reasoning? Vast number of variables Numerous probability estimates required Complex computations

Applicable to human reasoning? Fully-fledged BNs unsuitable as model of limited- capacity human reasoning BUT – a key aspect is the qualitative relations between variables (what depends on what) Judgments of relevance & causal dependency critical in legal analyses And people seem quite good at this! –Blood match raises probability of guilt –Alibi lowers it (not much!) Guilt Blood Alibi + -

Qualitative causal networks (under construction!) People reason using small-scale qualitative networks Require comparative rather than precise probabilities Guided by causal knowledge More formalized & testable version of story model?

Empirical studies Discrediting Evidence Alibi Evidence

Discredited evidence How do people revise their beliefs once an item of evidence is discredited? –When testimony of one witness is shown to be fabricated, how does this affect beliefs about testimony of other witnesses, or even other forensic evidence? –E.g., Romaines discredited testimony Involves a distinctive pattern of inference

Explaining away Vole cut himself P(G|B&C) < P(G|B) Finding out C too lowers the probability of G Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, mental models, mental logic) P(G|B) > P(B) Finding out B raises probability of G Blood on Voles cuffs Vole is guilty of murder Guilt Blood Cut

Discrediting vs. direct evidence Guilt Blood Cut Bayesian network model Causal model CUT only becomes relevant to guilt given BLOOD Important to distinguish explaining away from simply adding (negative) evidence Weighted additive model Standard regression model Guilt Blood Cut

Experimental questions Do people use causal models to reason with evidence in online tasks? Do they model discrediting evidence by explaining away? How does the discredit of one item of evidence affect other items?

YES when same source EVIDENCE 1 Neighbour says that suspect has stolen previously NO when different source EVIDENCE 1 Footprints outside house match suspects HYPOTHESIS: Local man did it Scenario: House burglary, local man arrested EVIDENCE 2 Neighbour says he saw suspect outside house on night of crime Neighbour is lying because he dislikes suspect ? Does the discredit of item 2 affect item 1?

Extension of discredit When do people extend the discredit of one item to other items? SAME –E.g. two statements from same neighbour SIMILAR –E.g. two statements from two different neighbours DIFFERENT –E.g., one statement and one blood test Causal model approach would expect people to distinguish SAME from DIFFERENT cases

BN models GUILTWitness AWitness B Discredit Same/Similar GUILTBlood testWitness Discredit Different

Experiment 1 Mock jurors given simplified criminal cases Four probability judgments (of guilt) –Baseline –Stage 1 (Evidence 1) Footprint match –Stage 2 (Evidence 2) Neighbour sees suspect –Final (Discredit 2) Neighbour is lying Compare judgments at Final stage and Stage 1 Does discredit return judgments to Stage 1? Vary relations between items of evidence –SAME, SIMILAR, DIFFERENT source

Witness1Witness2Discredit2 Both items undermined Forensic1Witness2Discredit2 When discredit presented LAST, it is extended regardless of relations between items Results Final judgments significantly lower than at Stage 1 for all conditions Discredit does not simply remove item 2; also affects belief in item 1

Summary Discrediting information extended regardless of relation to other evidence This pattern is consistent with Belief Adjustment model –Recency effect leads to over-weighting of discrediting information –Neglect relations between items Further test of BAM: manipulate order of evidence presentation

Experiment 2 Vary order of presentation of evidence –LATE……E1 E2 D2 –EARLY….E2 D2 E1 –Both orders ought to lead to same conclusions Relatedness –SAME, DIFFERENT

Witness1Witness2Discredit2 Both items undermined Forensic1Witness2Discredit2 When discredit presented LAST, it is extended regardless of relations between items Results: Late condition Final judgments lower than at Stage 1 for both conditions Discredit does not simply remove item 2 Replicates EXP 1

When discredit presented EARLY, only extended to related items Results: Early condition Pattern of judgments differ for SAME and DIFF SAME Final = Stage 2 DIFF Final > Stage 2 Appropriate sensitivity to relation between items Witness1Discredit1 Both items underminedWitness2Witness1Forensic1 Discredit1Only 1st item undermined

Problematic for current models Why are people rational in early but not late condition? Belief Adjustment model –Cannot explain early condition because does not consider relations between evidence Story model –Cannot explain bias in late condition (and needs to be adapted to online processing)

Coherence-based/grouping account Mind strives for most coherent representation Evidence grouped as +ve or -ve relative to guilt +ve and -ve groups compete, but within-group items mutually cohere (irrespective of exact causal relations) When an item of one group is discredited, this affects other items that cohere with it

LATE condition Incriminating evidence grouped together (regardless of source) Discredit affects the group (not just individual item) GUILT A + B + D ++

EARLY condition First item of evidence discredited Second item only discredited if from related source No grouping effect GUILT B + D A + + +

Study 3 Grouping hypothesis predicts that coherent groupings only emerge with elements that share the same direction (cf. Heider, 1946) Therefore discredit extended when evidence items both +ve or both -ve, but not with mixed items

Design Four evidence conditions 1.A+, B+, discredit B+ 2.A-, B-, discredit B- 3.A+, B-, discredit B- 4.A-, B+, discredit B+ Two levels of relatedness: similar and different Predictions –1&2 non-mixed -> discredit affects both items –3&4 mixed -> discredit affects only second item

Examples: Condition different Neighbour says she was with suspect at time of crime Neighbour lying because in love with suspect Lab tests reveal no footprint match Evidence 1 Evidence 2 Discredit

Examples: Condition different Lab tests reveal footprint match Evidence 1 Evidence 2 Discredit Neighbour says she was with suspect at time of crime Neighbour lying because in love with suspect

Results

Summary Grouping hypothesis supported Discredit extended when items share common direction, not when mixed Mutually coherent elements stand or fall together (even when no clear causal relation between them) Romaine & Agatha Christie knew this!

Alibi evidence Often crucial evidence (if true, absolves suspect) Treated with suspicion Hard to generate (even if innocent) Very little formal or empirical work Ongoing psychological studies – what makes a good alibi? (e.g., how much detail is best) Also interesting from normative viewpoint

Witness vs. Alibi models H E E* Suspect committed crime Witness report of suspect at crime scene Suspect at crime scene H E A Suspect claims he was not at crime scene D Suspect motivated to lie In alibi case – if suspect says he wasnt there, but he was, this raises likelihood of guilt (beyond that if you just find out he was there) To understand alibi evidence – need to represent potential deception With impartial witness – knowing that suspect was at crime scene screens off witness report from guilt judgment P(H|E&E*)=P(H|E)P(H|E&A)>P(H|E) Even though P(H|A)<P(H)

Pilot study Compare discredit of witness vs. alibi evidence Manipulate reason for discredit –Deception (X was lying in his statement) –Error (X was mistaken in his statement) Mock jurors given crime scenarios 3 judgments of guilt –Baseline –After statement (alibi/witness) –After discredit of statement

Alibi – discredit returns belief to baseline in error condition, but greatly enhances guilt in deception condition Fits with causal network predictions Witness – discredit returns belief to baseline (j1 = j3) irrespective of reason Results

General alibi model H E A Suspect claims he was not at crime scene D Suspect motivated to lie Case 1: Suspect provides alibi Higher motivation to lie if guilty than if innocent (hence link from H to D) Given alibi, discovery of E incriminates via two routes E raises likelihood of H directly E raises likelihood of H indirectly (via its effect on D) No screening-off ie P(H|E&A) > P(H|E)

General alibi model H E A Friend claims suspect was not at crime scene D Friend motivated to lie Case 2: Close relative/friend provides alibi AND they know whether or not suspect is guilty Higher motivation to lie if guilty than if innocent (hence link from H to D) Given alibi, discovery of E incriminates via two routes No screening-off ie P(H|E&A) > P(H|E)

General alibi model H E A Friend claims suspect was not at crime scene D Friend motivated to lie Case 3: Close relative/friend provides alibi BUT they do NOT know whether suspect is guilty Motivation to lie irrespective of actual guilt or innocence of suspect (effectively no link from H to D) Given alibi, discovery of E incriminates only via direct route Screening-off ie P(H|E&A) = P(H|E)

General alibi model H E A Stranger claims that suspect was not at crime scene D Stranger motivated to lie Case 4: Impartial stranger provides alibi AND they do NOT know whether suspect is guilty Low Motivation to lie AND this is unrelated to actual guilt or innocence of suspect (effectively no link from H to D) Given alibi, discovery of E incriminates only via direct route Screening-off ie P(H|E&A) = P(H|E)

Experimental study Do people conform to these models? Background info: – eg Victim is attacked on her way home … suspect is arrested Alibi: suspect was elsewhere at time of crime Manipulate who provides the alibi Discredit Alibi –e.g., suspect seen on CCTV near crime scene at time of crime

Alibi provider Motivated to lie? Knows H?Prediction SuspectYES P(H|E&A) > P(H|E) Close friendYES P(H|E&A) > P(H|E) Work colleague MAYBENOP(H|E&A) = P(H|E) StrangerNO P(H|E&A) = P(H|E) Results so far > = Scenarios dont clarify that close friend knows H (as shown by subjects judgments about this) Strong order effects --- ALIBI, CCTV >> CCTV, ALIBI

Conclusions so far People construct and use causal models Explaining-away inferences Grouping of variables can lead to biases Sensitive to Alibi model Puzzling order effect with Alibis Judgment involves both causality and coherence?

Thank you! Leverhulme/ESRC Evidence project –Nigel Harvey –Phil Dawid –Amanda Hepler –Gianluca Baio Students –Miral Patel –Nusrat Uddin –Charlotte Forrest