1. What they are and how they fit into
Bayesian Methods
What they are and how they fit into
Forensic Science

2. Outline Bayes’ Rule Bayesian Statistics (Briefly!)
Conjugates
General parametric using BUGS/MC software
Bayesian Networks
Some software: GeNIe, SamIam, Hugin, gR R-packages
Bayesian Hypothesis Testing
The “Bayesian Framework” in Forensic Science
Likelihood Ratios with Bayesian Network software

3. A little about conditional probability
Bayes’ Rule:

4. Probability
Frequency: ratio of the number of observations of interest (ni) to the total number of observations (N)
It is EMPIRICAL!
Probability (frequentist): frequency of observation i in the limit of a very large number of observations

5. Probability Belief: A “Bayesian’s” interpretation of probability.
An observation (outcome, event) is a “measure of the state of knowlege”Jaynes.
Bayesian-probabilities reflect degree of belief and can be assigned to any statement
Beliefs (probabilities) can be updated in light of new evidence (data) via Bayes theorem.

6. A better understanding of the world
Bayesian Statistics
The basic Bayesian philosophy:
Prior Knowledge × Data =
Updated Knowledge
A better understanding of the world
Prior × Data = Posterior

7. Bayesian Statistics Bayesian-ism can be a lot like a religion
Different “sects” of (dogmatic) Bayesians don’t believe other “sects” are “true Bayesians”
Bayes Nets
Graphical Models
Steffan Lauritzen (Oxford)
Judea Pearl (UCLA)
Parametric
BUGS (Bayesian Using Gibbs Sampling)
MCMC (Markov-Chain Monte Carlo)
Andrew Gelman (Columbia)
David Speigelhalter (Cambridge)
The major Bayesian “churches”
Empirical Bayes
Data-driven
Brad Efron (Stanford)

8. Bayesian Statistics What’s a Bayesian…??
Someone who adheres ONLY to belief interpretation of probability?
Someone who uses Bayesian methods?
Only uses Bayesian methods?
Usually likes to beat-up on frequentist methodology…

9. Bayesian Statistics Why? Actually DOING Bayesian statistics is hard!
We will up-date this prior belief later
Why?
Parametric Bayesian Methods
All probability functions are “parameterized”

10. YUK! Bayesian Statistics We need Bayes’ rule:
We have a prior “belief” for the value of the mean
We observe some data
What can we say about the mean now?
We need Bayes’ rule:
YUK!
And this is for an “easy” problem

11. Bayesian Statistics So what can we do???? Until ~ 1990, get lucky…..
Sometimes we can work out the integrals by hand
Sometimes you get posteriors that are the same form as the priors (Conjugacy)
Now there is software to evaluate the integrals.
Some free stuff:
MCMC: WinBUGS, OpenBUGS, JAGS
HMC: Stan

12. Bayesian Networks
A “scenario” is represented by a joint probability function
Contains variables relevant to a situation which represent uncertain information
Contain “dependencies” between variables that describe how they influence each other.
A graphical way to represent the joint probability function is with nodes and directed lines
Called a Bayesian NetworkPearl

13. Bayesian Networks (A Very!!) Simple exampleWiki:
What is the probability the Grass is Wet?
Influenced by the possibility of Rain
Influenced by the possibility of Sprinkler action
Sprinkler action influenced by possibility of Rain
Construct joint probability function to answer questions about this scenario:
Pr(Grass Wet, Rain, Sprinkler)

14. Bayesian Networks Pr(Sprinkler | Rain) Pr(Rain)
Rain:
yes no
Sprinkler:
was on
40% 1%
was off
60%
99%
Pr(Rain)
Rain:
yes
20% no
80%
Pr(Grass Wet | Rain, Sprinkler)
Sprinkler:
was on
was off
Rain:
yes no
Grass Wet:
99%
90%
80% 0% 1%
10%
100%

15. Bayesian Networks Pr(Sprinkler) Pr(Rain) Pr(Grass Wet)
Other probabilities
are adjusted given
the observation
You observe
grass is wet.
Pr(Grass Wet)

16. Bayesian Networks Areas where Bayesian Networks are used
Medical recommendation/diagnosis
IBM/Watson, Massachusetts General Hospital/DXplain
Image processing
Business decision support
Boeing, Intel, United Technologies, Oracle, Philips
Information search algorithms and on-line recommendation engines
Space vehicle diagnostics
NASA
Search and rescue planning
US Military
Requires software. Some free stuff:
GeNIe (University of Pittsburgh)G,
SamIam (UCLA)S
Hugin (Free only for a few nodes)H
gR R-packagesgR

17. Bayesian Statistics
Bayesian network for the provenance of a painting given trace evidence found on that painting

18. Bayesian Statistics Frequentist hypothesis testing:
Assume/derive a “null” probability model for a statistic
E.g.: Sample averages follow a Gaussian curve
Say sample statistic falls here
“Wow”! That’s an unlikely
value under the null hypothesis
(small p-value)

19. Bayesian Statistics Bayesian hypothesis testing:
Assume/derive a “null” probability model for a statistic
Assume an “alternative” probability model
p(x|null)
p(x|alt)
Say sample statistic falls here

20. The “Bayesian Framework”
Bayes’ RuleAitken, Taroni:
Hp = the prosecution’s hypothesis
Hd = the defences’ hypothesis
E = any evidence
I = any background information

21. { { { The “Bayesian Framework” Odd’s form of Bayes’ Rule:
Posterior odds in favour of
prosecution’s hypothesis
Likelihood Ratio
Prior odds in favour of
prosecution’s hypothesis
Posterior Odds = Likelihood Ratio × Prior Odds

22. The “Bayesian Framework”
The likelihood ratio has largely come to be the main quantity of interest in their literature:
A measure of how much “weight” or “support” the “evidence” gives to one hypothesis relative to the other
Here, Hp relative to Hd
Major Players: Evett, Aitken, Taroni, Champod
Influenced by Dennis Lindley

23. The “Bayesian Framework”
Likelihood ratio ranges from 0 to infinity
Points of interest on the LR scale:
LR = 0 means evidence TOTALLY DOES NOT SUPPORT Hp in favour of Hd
LR = 1 means evidence does not support either hypothesis more strongly
LR = ∞ means evidence TOTALLY SUPPORTS Hp in favour of Hd

24. The “Bayesian Framework”
A standard verbal scale of LR “weight of evidence” IS IN NO WAY, SHAPE OR FORM, SETTLED IN THE STATISTICS LITERATURE!
A popular verbal scale is due to Jefferys but there are others
READ British R v. T footwear case!

25. Bayesian Networks
Likelihood Ratio can be obtained from the BN once evidence is entered
Use the odd’s form of Bayes’ Theorem:
Probabilities of the theories after
we entered the evidence
Probabilities of the theories before
we entered the evidence

26. The “Bayesian Framework”
Computing the LR from our painting provenance example:

27. How good of a “match” is it?
Efron Empirical Bayes’
An I.D. is output for each questioned toolmark
This is a computer “match”
What’s the probability the tool is truly the source of the toolmark?
Similar problem in genomics for detecting disease from microarray data
They use data and Bayes’ theorem to get an estimate
No diseasegenomics = Not a true “match”toolmarks

28. Empirical Bayes’
We use Efron’s machinery for “empirical Bayes’ two-groups model”Efron
Surprisingly simple!
Use binned data to do a Poisson regression
Some notation:
S-, truly no association, Null hypothesis
S+, truly an association, Non-null hypothesis
z, a score derived from a machine learning task to I.D. an unknown pattern with a group
z is a Gaussian random variate for the Null

29. Empirical Bayes’ From Bayes’ Theorem we can getEfron:
Estimated probability of not a true “match” given the algorithms' output z-score associated with its “match”
Names: Posterior error probability (PEP)Kall
Local false discovery rate (lfdr)Efron
Suggested interpretation for casework:
We agree with Gelaman and ShaliziGelman:
“…posterior model probabilities …[are]… useful as tools for prediction and for understanding structure in data, as long as these probabilities are not taken too seriously.”
= Estimated “believability” of machine made association

30. Empirical Bayes’
Bootstrap procedure to get estimate of the KNM distribution of “Platt-scores”Platt,e1071
Use a “Training” set
Use this to get p-values/z-values on a “Validation” set
Inspired by Storey and Tibshirani’s Null estimation methodStorey
Use SVM to get KM and KNM “Platt-score” distributions
Use a “Validation” set
From fit histogram by Efron’s method get:
“mixture” density
z-density given KNM => Should be Gaussian
Estimate of prior for KNM
What’s the point??
We can test the fits to
and !
z-score

31. Posterior Association Probability: Believability Curve
12D PCA-SVM locfdr fit for
Glock primer shear patterns
+/- 2 standard errors

32. Bayes Factors/Likelihood Ratios
In the “Forensic Bayesian Framework”, the Likelihood Ratio is the measure of the weight of evidence.
LRs are called Bayes Factors by most statistician
LRs give the measure of support the “evidence” lends to the “prosecution hypothesis” vs. the “defense hypothesis”
From Bayes Theorem:

33. Bayes Factors/Likelihood Ratios
Once the “fits” for the Empirical Bayes method are obtained, it is easy to compute the corresponding likelihood ratios.
Using the identity:
the likelihood ratio can be computed as:

34. Bayes Factors/Likelihood Ratios
Using the fit posteriors and priors we can obtain the likelihood ratiosTippett, Ramos
Known match LR values
Known non-match LR values

35. Acknowledgements Professor Chris Saunders (SDSU)
Professor Christophe Champod (Lausanne)
Alan Zheng (NIST)
Research Team:
Dr. Martin Baiker
Ms. Helen Chan
Ms. Julie Cohen
Mr. Peter Diaczuk
Dr. Peter De Forest
Mr. Antonio Del Valle
Ms. Carol Gambino
Dr. James Hamby
Ms. Alison Hartwell, Esq.
Dr. Thomas Kubic, Esq.
Ms. Loretta Kuo
Ms. Frani Kammerman
Dr. Brooke Kammrath
Mr. Chris Luckie
Off. Patrick McLaughlin
Dr. Linton Mohammed
Mr. Nicholas Petraco
Dr. Dale Purcel
Ms. Stephanie Pollut
Dr. Peter Pizzola
Dr. Graham Rankin
Dr. Jacqueline Speir
Dr. Peter Shenkin
Ms. Rebecca Smith
Mr. Chris Singh
Mr. Peter Tytell
Ms. Elizabeth Willie
Ms. Melodie Yu
Dr. Peter Zoon