1. The relative importance of clinical, economic, social and organizational criteria in cancer drug reimbursement in Canada:
A revealed preferences analysis of recommendations of the pan Canadian Oncology Drug Review
Dr. Christopher Skedgel (University of East Anglia)
Dr. W. Dominika Wranik (Dalhousie University)
Min Hu (Dalhousie University)

2. Wranik, Katz, Levy, Korchagina, Edwards
Canadian Health System
11/28/2018
CANADA HEALTH ACT - FEDERAL
Publicly administered;
Comprehensive;
Accessible;
Transferrable;
Universal;
PUBLIC HEALTH CARE BUDGET
Provincial/ Territorial
Federal transfer (ca. 15%)
Provincial taxation (ca. 85%)
PRIVATE SPENDING
30% of total spending
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

3. Health Expenditures in Canada
Health Expenditures in Canada in 2015 (‘000,000 CAD)
Category
Amount
Federal Health Transfer
34,026
Total Health Expenditure (All Sectors)
219,144
Total Health Expenditure (Public)
155,000
Total Drug Expenditure (All Sectors)
34,452
Total Drug Expenditure (Public)
12,589
Public to Total Drug Expenditure (%) 37
Per Capita Drug Expenditure (Public) ($)
350
1 Canadian Institute for Health Information. National Health Expenditure Database. October 28,
2 Statistics Canada, CANSIM Tables, 2015 Estimates
3 Federal Support to Provinces. 2015/16 Budget Estimates.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences

4. Wranik, Katz, Levy, Korchagina, Edwards
Canadian Formulary Review Process
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Canadian Agency for Drugs
and Technologies in Health
pan-Canadian Oncology Drug Review
Review and formulary recommendation for non-cancer drugs (CDR)
Review and formulary recommendation for cancer drugs (pCODR)
Provincial formulary decision for all drugs
Rationale for Centralized Review
Support consistency across Provinces and Territories
Speed up review process
Cancer drugs have unique features
Review Criteria
Clinical Benefit
Economic Evaluation
Patient-based Values
Adoption Feasibility
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

5. Wranik, Katz, Levy, Korchagina, Edwards
pan-Canadian Oncology Drug Review (pCODR) Process
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Step 1 – Screen request for review and initiate review process
Step 2 – Collect patient advocacy group information
Step 3 – Collect registered clinician information
Step 4 – Conduct clinical review
Step 5 – Conduct economic review *
Step 6 – Clarify information with the submitter of drug review *
Step 7 – Summarize and review with pERC *
Step 8 – Post initial recommendation, receive feedback, and review with pERC
Step 9 – Post final recommendation including all feedback (REPORT)
RECOMMENDATION
Approve
Conditionally approve
Reject
FUNDING DECISION
Provincial/ Territorial Ministry of Health
Possible for manufacturer to offer different prices to different Provinces/ Territories
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

6. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
GOAL
To characterize challenges with the use of several types of evidence (with focus on economics) in recommendations made by the pCODR.
RATIONALE
Ongoing challenges with the production and use of economic evidence in the context of formulary committees.
REVEALED PREFERENCES
What do pCODR decisions reveal about the relative importance of decision criteria assigned by the committee?
STATED PREFERENCES
What do individuals involved in the pCODR process say is the relative importance of decision criteria?
ROLE OF PATIENT VALUES
How are patient values defined and used to contribute to the decision process?
QUALITATIVE INTERVIEWS
How to individuals involved in the pCODR process describe their approach to balancing competing criteria?
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

8. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
REVEALED PREFERENCES
How are the criteria (clinical benefit, economic evaluation, patient-based values, and adoption feasibility weighted by the pCODR expert review committee, as revealed by the recommendations made.
Variable
Values (% frequencies)
Recommendation
Approve (16)
Conditional (58)
Reject (26)
Clinical Benefit
Yes (66)
Maybe (13)
No (21)
Relative Survival Gain
Yes (78)
No (22)
OS Flag
Yes (54)
No (46)
Quality of Clinical Evidence
High (77)
Low (23)
ICER ($150,000)
High (62)
Low (38)
ICER reported (flag)
Yes (91)
No (9)
ICER uncertainty
High (83)
Low (17)
Severity of side effects
High (17)
Low (83)
Unmet need (no alternatives)
Yes (26)
No (74)
Type of drug
Oral (50)
IV (50)
Infrastructure
High (56)
Low (44)
Budget impact
High (79)
Low (21)
VARIABLE CODING
94 publically available pCODR Expert Review Committee reports, each including the recommendation and description of deliberation about each criterion.
Independently coded by two reviewers, conflicts discussed, third reviewer with pivotal vote when necessary.
Initial sample of 10 used to validate code.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

9. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
RESEARCH QUESTIONS
Model 1 – What are the factors associated with a rejection as compared to an approval (full or conditional)?
Model 2 – What is the ICER threshold between full and conditional approval?
STATISTICAL APPROACH
Model 1 (n=94)
Binary dependent variable: Rejection and Not Rejection.
Tested all possible combinations of variables and plausible interaction terms.
Selected preferred specification based on the highest predictive power and lowest Aikake’s Information Criterion (AIC).
Calculated marginal effects for each variable included in the preferred specification to estimate its impact on the odds of rejection.
Model 2 (n=70)
Binary dependent variable: Approval and Conditional Approval.
Preferred model selection same as in Model 1.
Derived predicted probability of full approval by ICER.
Identified ICER inflection points and threshold for full approval greater than 50%.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

10. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Model 1 Results
Rejection versus Approval (F/C)
Variable
(reference level)
Coefficient
Standard Error
p-value
Marginal Effects
Intercept
0.6028
0.82 --
Quality of clinical evidence (high)
0.6585
0.02
-24.5%
Budget impact (low)
0.7219
0.16
-16.1%
Relative survival X adverse events (low)
0.3965
0.04
-13.0%
AIC
83.346
Predictive power (all decisions)
81 %
Drugs with higher quality clinical evidence and low budget impact are less likely to be rejected.
Drugs with greater survival advantage combined with low side effects are less likely to be rejected.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

11. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Model 2 Results
Full Approval versus Conditional Approval
Variable
(reference level)
Coefficient
Standard Error
p-value
Marginal Effects
Intercept
1.4240
2.2689
0.53 --
ICER x 10K
0.0169
0.00
-3.0%
OS Flag (yes)
5.5410
3.8028
0.15
33.3%
Relative survival X OS Flag
2.3300
0.12
-22.0%
Relative survival X AE (low)
2.0529
1.1599
0.08
12.3%
AIC
26.39
Predictive power (all decisions)
70 %
Drugs with higher relative survival combined with low side effects are more likely to be fully approved.
Drugs with higher ICER are less likely to gain full approval.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

12. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Model 2 Results
ICER threshold
ICER ($/QALY)
Full Approval
Conditional
Rejection
Total
0 – 50,000 5 1 6
50,001 – 100,000 4 2 11
100,001 – 150,000 3 19
150,001 – 200,000 14 8 22
200,001 and up 28
Unreported 15 55 24 94
Above $150,000 there are no full approvals.
Above $100,000, the probability of conditional approval (versus full) increases.
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.

13. Wranik, Katz, Levy, Korchagina, Edwards
AFFORD – Affordable eFfective eFficient Oncology Reimbursement Decisions
Wranik, Katz, Levy, Korchagina, Edwards
11/28/2018
Discussion
First to estimate relative weights of decision criteria assigned in oncology drug funding recommendations in Canada.
Decision to reject a submission is driven primarily by its clinical profile. This is consistent with studies from Belgium, Canada (CADTH), Poland, the United Kingdom and Wales that highlight importance of clinical superiority and clinical uncertainty.
Economic criteria play lesser role in the decision to reject, but play a key role in the decision to approve fully or conditionally.
Feasibility (budget impact) plays a small role in the rejection decision.
Patient based values do not appear to play a role (subject of a separate analysis).
Skedgel, Wranik, Hu (2017) – pCODR Revealed preferences
Working draft - Please DO NOT CITE without permission of the authors.