1. What works and what is worth it; Science versus the rest Paul Marchant 11 July 2007

2. Science is the belief in the ignorance/unimportance of experts. Science is a way to teach how something gets to be known, what is not known, to what extent things are known (for nothing is known absolutely), how to handle doubt and uncertainty, what the rules of evidence are, how to think about things so that judgements can be made, how to distinguish truth from fraud, and from show. Views of Richard Feynman

3. Science is much more than a collection of facts. Science is about truth and honesty...as in ……If you're doing an experiment you should report everything that you think might make it invalid - not only what you think is right about it …from ‘Cargo cult science’.

4. We have a desire to improve matters through ‘technology’, ( in a very general sense, e.g. medicine.) Need to tell if the technology works….Does CCTV reduce crime?...is it worth it? Best to do a trial…a fair test; rather than ‘ask an expert’. Note the James Lind library www.jameslindlibrary.org www.jameslindlibrary.org

5. James Lind (1747), HMS Salisbury, Scurvy trial, Oranges and Lemons found to be superior to other treatments…..helped develop the might of the British Navy! A fair test is comparing like with like. If not ‘fair’, the apparent effect might be artefactual. Beware of ‘science’ from sales persons.

6. Regression towards the mean First discovered by Francis Galton (1880s) who found that: Tall parents tend to have tall offspring but not as tall as themselves (on average); i.e. offspring tend to be less tall than their parents. Short parents tend to have short offspring but not as short as themselves (on average); i.e. offspring tend to be less short than their parents. This tendency for both to become more average is ‘regression towards the mean’ RTM.

7. Regression towards the mean X The before measurement Y The after measurement Cloud of Data Points Line of Equality 010050 0 100 Line of mean of Y for a given X (The conditional Mean)

8. RTM needs to be accounted for in making comparisons Imagine a thought experiment for a treatment to reduce the height of the next generation! Problematic if the treatment is given only to tall parents and the results (height of offspring) are compared with those of short parents, who receive placebo, (because of RTM as in Galton’s discovery).

9. The Randomised Controlled Trial (A truly marvellous scientific invention) Note to avoid bias: Register trial / protocol. Allocation is best made tamper-proof. (e.g. use ‘concealment’) Use multiple blinding of: –patients, –physicians, –assessors, –analysts … Population Take Sample Randomise to 2 groups Old Treatment Compare outcomes (averages) recognising that these are sample results and subject to sampling variation when applying back to the population New Treatment

10. Randomisation ensures that in long run one compares like with like, so RTM is not a problem. (The effect of baseline imbalance in a trial can be handled in the analysis.) Avoid human interference.

11. The claim that street lighting reduces crime is questionable. The claim is used in part to justify spending billions on street lighting PFIs. (See Marchant, 2006) The research at the basis of the claim is ‘not of the highest quality’. Can judge effectiveness through a ‘stepped wedge’ roll out. (Use MMMC models estimated with MCMC, talk at RSS2007 Statistics and Public Policy) Good research is relatively cheap.

12. Of course not everything can be subject to RCTs…. but a lot of things can. (There is resistance…perhaps because science is ‘stark’ ….but ethics dictates that we should get best evidence. ) We ought to see after implementation of a policy what has been achieved, in a scientific way, e.g. define the criteria at the outset.

13. Even as a die hard scientist, I accept that there are some propositions which can’t be determined, scientifically! Rely on human values …. Because experts are unreliable, gives more weight to an egalitarian approach where all peoples views are valued.

14. P Marchant (2005) Evaluating area-wide crime reduction measures, Significance 2, pp63-65 www.blackwell-synergy.com/links/toc/sign/2/2 P Marchant (2006) Investigating whether a crime reduction measure works, Radical Statistics 91, pp46-62 www.radstats.org.uk/no091/Marchant91.pdf P Marchant (2008?) Regression towards the mean, in; The International Encyclopaedia of Social Sciences