The uptake of value of information methods Solutions found and challenges to come Alan Brennan Director of Operational Research ScHARR

Aim of this session Discuss the roles of VoI analysis Demonstrate recent technical progress Growth in interest in and use of VoI methods Discuss challenges for further uptake Some recommendations Personal Views !

What is Value of Information? Given …. a choice between strategies, a decision rule for adoption of strategies, some uncertainty VoI analysis tells us… ›How valuable more information would be to reduce uncertainty, to help us choose ›Which uncertain parameters are most crucial ›How valuable a sample of size n=10 would be compared to n=100, or n=1000 ….

Roles of VoI

Probabilistic Sensitivity Analysis Without VoI C-E planeandCEAC i.e. describes how uncertain we are

Probabilistic Sensitivity Analysis With VoI adds (1) Which parameters or groups of parameters are important i.e. the causes - why we are uncertain

Probabilistic Sensitivity Analysis With VoI adds (2) Value of different samples and research design combinations i.e. how best to resolve the uncertainty

Probabilistic Sensitivity Analysis - VoI difficulty myth “It is complex” “It takes a long time” “Methods are not developed” “People can’t understand it”

Probabilistic Sensitivity Analysis - VoI truth If you have done a probabilistic sensitivity analysis i.e. Then you are 90% to 95% there

Prioritising and Planning HTA Recommendations from systematic review of the use of modelling in planning and prioritising trials (HTA 2003; Vol 7: number 23) Results of Pilot Study show.. Beneficial in refining research design and quantifying uncertainty choice of timing and topic are vital who should do the analysis is important

Industry Roles – Societal Perspective Undertaking probabilistic sensitivity analysis and VoI to be ahead of re-imbursement authorities VoI to identify likelihood of cost-effectiveness and priorities for further data collection ›Very early Discovery phase work ›Phase II results available – now design phase III ›Phase III results available – what else do we need to make the economic case?

Industry Roles – Commercial Perspective Approaches still developing (and commercial in confidence) Define Net benefit not as λ * QALY – cost but rather as commercial sales / profit Or model linking sales to likelihood of re- imbursement or extent of cost-effectiveness

Growth in Uptake A personal view 1996 – Claxton and Posnett – “what are all these squiggles, it will never catch on” 1999 – doing our review, realising the conceptual validity and practicability of VoI 2000 – some applications but people are unsure on methods 2001 – IHEA York / MDM San Diego 2 or 3 speeches (all UK)

Growth in Uptake A personal view 2002 – CHEBS Focus fortnight with ourselves Karl Claxton and Tony Ades – methods sorted Nice Appraisals beginning to use VoI 2002 – MDM Baltimore -7 people UK and Canada – lots of interest 2003 – MDM Chicago -15 to 20 people (US, Canada, Netherlands, UK) even more interest

Technical Problems Recently Solved

Technical Problems Recently Solved:EVPI Correct method for EVPI = 2 level simulation EVPI = 1 level simulation EVPI works if …. (a) the net benefit functions are linear functions of the  -i for all of the decisions d and all of the possible values of the parameter set of interest  i, and (b) if  i and  -i are independent. “

Technical Problems Recently Solved:EVSI Correct Method for EVSI = 2 level simulation (Bayesian update given simulated collected data) EVSI easy with conjugate distributions (i.e. retain same functional form when additional data is collected and synthesised) ›Normal, Beta, Gamma, Lognormal EVSI more complex without conjugacy

Technical Problems Recently Solved: Shortcuts Complex models can be ‘emulated’ e.g. Gaussian Processes, which also offer quicker EVPI and EVSI for 1 variable calculation functionality Laplace approximation can help shortcut to a 1 level simulation for EVSI calculations (poster) The EVSI curve has a common but not universal shape – exponential of square root of n Valuing additonal data in survival trials (poster)

Technical Problems Recently Solved: Software EXCEL for models R / Splus advantages ›More sampling functionality ›More optimisation functionality ›Faster running times ›Easier code WinBUGS for complex posterior distributions – interlinkage with R and Splus

Key Challenges: Uptake for Sensitivity Analysis - AIDA Awareness  Interest  Desire  Action Publications / Conferences / Seminars Training  York / Oxford Advanced Modelling Course  CHEBS characterising uncertainty and analysing outputs courses (2004)  Bristol WinBUGS course (2004) Network of experts Web resources

Key Challenges: Uptake in trial design Engaging with traditional trial designers Clinically significant difference is a proxy for the decision rule Need to work together to compare results and approaches on pilot trials particularly ones with Economic and Quality of life data to be collected

Key Challenges: Meeting Criticisms 1.How do you know uncertainty is properly characterised i.e. you might be uncertain about the uncertainty 2.You need to multiply by the number of people affected by the decision - How long does the technology / decision last – 2 years, 5 years, 10 years? An issue even without doing EVI 3.Information has value beyond the jurisdiction

Key Challenges: Technical Correlation, Correlation, Correlation Integrated VoI and evidence synthesis quickly Reversible decisions Deciding to wait – options pricing

Conclusions Significant recent progress, interest and growth Key challenges ›Speeding up and teaching standard routines to do the calculations ›Working through when exactly VoI is likely to be most valuable ›Engaging sceptics to collaborate Now is the time ….

Conclusions