2. Medical Screening The government has a screening program for a potentially fatal medical condition which is thought to affect 1 in 1000 of the population. 100 000 people are tested. The test gives the correct result in 98 out of 100 patients. People will naturally worry if they have a result which suggests they have the condition, but how much of this concern is misplaced?

3. Where do we start? How worried should someone be if they have a positive result? –Extremely worried, –Very worried, –A little worried, –Not at all worried. Why? Do you think many will get a FALSE positive result? Explain why.

4. Some calculations that might help (i)How many of the 100 000 people tested would you expect to have the condition? (ii)How many of the 100 000 people tested would you expect to NOT have the condition? (iii)How many people would be expected to get a positive test result?

5. False Positives A ‘false positive’ is a result which suggests a person has a condition when they actually don’t have it at all Fill in the ‘tree diagram’ on the following slide, with numbers, to see how many people will receive genuine positive results and how many will receive false positive results Remember that the test result is correct 98% of the time: –98% of the time it says you do have it when you do –98% of the time it says you don’t have it when you don’t

6. Number tested 100000 Have the medical condition ??? Test Positive ??? Test Negative ??? Don't have the medical condition ??? Test Positive ??? Test Negative ??? 1 in 1000 999 in 1000 98% 2% False Positives

7. Number tested 100000 Have the medical condition 100 Test Positive 98 Test Negative 2 Don't have the medical condition 99900 Test Positive 1998 Test Negative 97902 1 in 1000 999 in 1000 98% 2% False Positives

8. What percentage of the people who have received positive results actually have the condition? Does this surprise you? Should doctors use this test? Is it better to worry people who don’t really have the condition, or to miss people who do have it? False Positives

9. Hit and Run The police receive a report of a hit and run accident involving a taxi. Although it is dark, an eye witness is 90% sure it was a green cab. There are 1000 taxis in the city, 10 are green and the rest are red. Is it more likely the accident involved a red or green taxi?

10. ??% Taxis ???Green ??Involved ?? Not involved ?? Red ???Involved ?? Not involved ?? Fill in the blanks: Eye witness is 90% sure There are 1000 taxi’s in the city 10 are green and the rest are red ??% Hit and Run

11. 90% Taxis 1000Green 10Involved 9 Not involved 1 Red 990Involved 99 Not involved 891 Is it more likely the accident involved a red or green taxi? 10% Hit and Run

12. Drug Testing In the USA, 20 000 air traffic controllers undergo random drug testing. The test is good but not perfect: 96% of those who use drugs test positive 93% of those that do not use drugs test negative Based on previous figures the Federal Airline Authority believe that 99% of air traffic controllers are drug free. Do you think it’s likely that people identified as positive by the test are guilty of taking drugs?

13. How is this problem different to the previous ones? Assuming that the Federal Aviation Authority are correct – that 99% of pilots are drug free – what percentage of those testing positive are actually drug free? Complete the tree diagram on the next slide and discuss your results. Drug Testing

14. Number tested 20000 Number expected to be innocent of taking drugs ??? Number expected to test Positive ??? Number expected to test Negative ??? Number expected to be guilty of taking drugs ??? Number expected to test Positive ??? Number expected to test Negative ??? ??% Drug Testing

15. Number tested 20000 Number expected to be innocent of taking drugs 19800 Number expected to test Positive 1386 Number expected to test Negative 18414 Number expected to be guilty of taking drugs 200 Number expected to test Positive 192 Number expected to test Negative 8 99% 4% 96% 93% 7% 1% Drug Testing

16. Lie Detector Test A TV show uses a lie detector test to try to establish which one of two people is telling the truth. Assume that just one of the two people in the dispute is not being honest, and it is equally likely to be either guest. The American Polygraph Association claim the tests are 89% accurate for a single issue response. Out of 1000 shows, how many people will be falsely accused of lying? How many guests will receive the wrong result?

17. Draw out a diagram to help you ascertain how many inaccurate results there are likely to be. What will happen on the shows if one or other or both of the results are inaccurate? Lie Detector Test

18. Number tested 2000 Truthful 1000 Pass Lie Detector 890 Fail Lie Detector 110 Not Truthful 1000 Pass Lie Detector 110 Fail Lie Detector 890 Lie Detector Test

19. Down’s Syndrome Occurrence In an article in 2009, a “top doctor” called for changes to the pre-natal screening for Down’s Syndrome. To consider the concerns expressed by the doctor here are the most recent figures from the Office for National Statistics which show that there were approximately 730,000 babies born in the UK in 2012 and gives the approximate age of their mothers. As stated in the article, screening for Down’s Syndrome is currently offered to all prospective mothers. 2012 Live Births In England and Wales All agesUnder 2020-2425-2930-3435-3940-4445 and over 729,67433,815132,456202,370216,242114,79728,0191,975

20. The current test has a false positive rate of about 3% according to information from the NHS. To complicate matters further the chances of having a baby with Down’s Syndrome increases with the age of the mother. (Figures from the NHS) 25 years of age has a risk of 1 in 1,250 30 years of age has a risk of 1 in 1,000 35 years of age has a risk of 1 in 400 40 years of age has a risk of 1 in 100 45 years of age has a risk of 1 in 30 How many false positives are there likely to be for the different age groups? Use this evidence to decide whether you think pre-test counselling is a good idea, explaining your response. Down’s Syndrome Occurrence

21. Births: The current test has a false positive rate of about 3% Chances of having a baby with Down’s Syndrome increases with the age of the mother. 25 years of age has a risk of 1 in 1,250 30 years of age has a risk of 1 in 1,000 35 years of age has a risk of 1 in 400 40 years of age has a risk of 1 in 100 45 years of age has a risk of 1 in 30 Down’s Syndrome Occurrence

23. Teacher notes: It’s a Risk Pupils will be familiar with situations in which ‘risk’ is used, such as the use of ‘Lie Detector tests’ on day-time TV shows and athletes testing positive for the use of banned substances, but may be less familiar with the impact of test accuracy in medical tests. Through these activities, pupils will gain a better understanding of false positives and how to interpret ‘risk'.

24. Teacher notes: It’s a Risk There are 5 activities with a decreasing amount of support and structure. It is important to give students time to think, discuss and absorb. Many of the results will challenge what they believe or suspect to be true. Attaining an understanding of why there are so many false negatives is helpful. It is suggested that students work in pairs or small groups in order to discuss their initial thoughts and then to make sense of the outcomes. Activities could all be tackled within a lesson or each could be used as a starter activity in a series of lessons.

25. Teacher notes: Medical Screening Where do we start Most students in the trials said very or extremely worried. The cause of this worry was generally stated as the 98% accuracy of the test. Some calculations that might help Leave a little time between each question for the students to think and respond. Students should attempt to produce a figure by calculation… the answers will be confirmed in the next section ‘False Positives’ False Positives Of the 2096 positive results, only 98 are genuine – that’s less than 5% Why, with such an accurate test, are there so many false positives? Because 98% of small amount (in this case 100) < 2% of much larger amount (in this case 99900). This is a key point so take some time to ensure that the students fully appreciate it.

26. Teacher notes: Hit and Run Eye witness testimony is notoriously inaccurate. In this case, the eye-witness is 90% sure it was a green taxi, which means that there’s a 90% chance of it being one of the green taxis and a 10% chance of it being one of the red taxis. This is similar in structure to the ‘Medical Screening’ question, where there are a lot more items in the ‘non-target’ group. In this case there’s a lot more red taxis than green ones. Out of the 108 taxis that could have been involved, 9 are green and 99 are red, so it’s far more likely that the taxi involved was actually red.

27. Teacher notes: Drug Testing What is different in this problem? The probabilities are conditional this time. 96% for correctly identifying a drug user, but only 93% chance of correctly identifying a non-user. Ask the students to fill in the blanks in the diagram and discuss the outcomes. Pose the question: “If an air traffic controller has a positive result, how likely are they to actually have taken drugs?” Of the 1578 likely to test positive, only 192 are likely to have done. That means approximately 88% of those testing positive in this test are actually innocent.

28. Teacher notes: Lie Detector Test This is a relatively straight forward problem. If students have tried the previous problems with some structure, then this would be a good one to let them try without teacher input and without being given a blank tree diagram. In the 1000 shows, there are likely to be 110 false negatives and 110 false positives. This would mean that on some shows the following could happen: If one result is accurate and one is erroneous, then either both people are found to be telling the truth or both people are found to be lying Occasionally, the results are completely the wrong way round – the liar is found to be telling the truth and the honest person is found to be lying

29. Teacher notes: Down’s Syndrome Sensitivity must be shown with this content; this activity is simply about identification of a condition and allowing prospective mothers to be prepared. Students should be encouraged to explore the information for themselves and justify their responses to the questions. It might be helpful to print out copies of the information slide for students to refer to more easily. Working with a partner or in a small group would be helpful for students to share the workload and to interpret their answers. If students struggle to get started, encourage them to think about the following: Number of Down’s Syndrome babies expected for each age group Number of non-Down’s Syndrome babies expected for each age group Then consider how many babies would fall into each of the following categories for each age group: Correct test resultIncorrect test result Down’s baby Non-Downs baby

30. Teacher notes: Down’s Syndrome

31. Looking at the results it’s possible to conclude that mothers below 35 years of age are much more likely to get a false positive result. However, about half the children with Down’s Syndrome are born to mothers aged under 35; this is because they make up the largest proportion of mothers. Perhaps mothers should be made aware of the probability of a false positive, for their age group, and they can then make an informed choice of whether they want the test.

32. References American Polygraph Association accessed 3-2-2014 http://www.polygraph.org/section/resources/polygraph-validity-research Lie Detector Test accessed 3-2-2014 http://www.dailymail.co.uk/femail/article-1203070/Jeremy-Kyle-nearly-killed-The- horrifying-story-womens-decision-daytime-TVs-notorious-show.html Down’s Syndrome article accessed 3-2-2014 http://www.dailymail.co.uk/health/article-1223406/Doctor-calls-mothers-opt-Downs- Syndrome-test.html Data from the ONS accessed 3-2-2014 http://www.ons.gov.uk/ons/taxonomy/index.html?nscl=Births+by+Mother%27s+Area+of+ Residence#tab-data-tables NHS information accessed 3-2-2014 http://www.nhs.uk/news/2013/06June/Pages/New-Downs-syndrome-blood-test-more- reliable.aspx http://www.nhs.uk/Conditions/Downs-syndrome/Pages/Causes.aspx