Download presentation

Presentation is loading. Please wait.

Published byAlejandro Moore Modified over 3 years ago

1
Methods for Cost Estimation in CEA: the GLM Approach Henry Glick University of Pennsylvania AcademyHealth Issues in Cost-Effectiveness Analysis Washington, DC 06/10/2008

2
Outline Policy-relevant parameter for cost-effectiveness Problems posed by nonnormality of cost data Generalized linear models as a response to the problems –Identifying links and families (gets a little technical) General comments My objective is to provide practical advice for ways implement GLM models. Slides available at: http://www.uphs.upenn.edu/dgimhsr/presentations.htm

3
Policy Relevant Parameter for CEA Policy relevant parameter: differences in the arithmetic, or sample, mean –In welfare economics, a project is cost-beneficial if the winners from any policy gain enough to be able to compensate the losers and still be better off themselves Thus, we need a parameter that allows us to determine how much the losers lose, or cost, and how much the winners win, or benefit –From a budgetary perspective, decision makers can use the arithmetic mean to determine how much they will spend on a program

4
Policy Relevant Parameter for CEA (2) In both cases, substitution of some other parameter for the sample mean can be justified only if it provides a better estimate of gains and losses or spending

5
Are Sample Means Always the Best Estimator? In simulation, when cost data are sufficiently nonnormal, the relative bias (truth - observed) 2 for other parameters such as the median or adjusted geometric mean can sometimes be lower than the relative bias observed for the arithmetic mean HOWEVER, –Distribution required to be sufficiently nonnormality that ln(cost) is also substantially nonnormally distributed –In actual data, since we never know truth, it is difficult to determine whether other parameters will have lower relative bias than sample mean

6
The Problem Common feature of cost data is right-skewness (i.e., long, heavy, right tails)

7
The Problem (cont.) Distributions with long, heavy, right tails will have a mean that differs from the median, independent of outliers Cost data also cant be negative, and can have large fractions of observations with 0 cost Nonnormality of cost data can pose problems for common parametric tests such as t-test, ANOVA, and OLS regression

8
Common (Relatively Bad) Responses To Violation Of Normality Adopt nonparametric tests of other characteristics of the distribution that are not as affected by the nonnormality of the distribution (biostatistical approach) Transform the data so they approximate a normal distribution (classic econometric approach)

9
Recommended Response: Adopt More Flexible Models Generalized Linear Models (GLM) –Have the advantages of the log models, but dont require normality or homoscedasticity and evaluate a direct of the difference in cost and dont raise problems related to retransformation from the scale of estimation to the raw scale –To build them, one must identify a "link function" and a "family (based on the data)

10
Stata and SAS Code STATA code: glm y x, link(linkname) family (familyname) General SAS code (not appropriate for gamma family / log link): proc genmod; model y=x/ link=linkname dist=familyname; run;

11
When running gamma/log models, the general SAS code drops observations with an outcome of 0 If you want to maintain these observations and are predicting y as a function of x (M Buntin): proc genmod; a = _mean_; b = _resp_; d = b/a + log(a) variance var = a 2 deviance dev =d; model y = x / link = log; run; SAS Code for a Gamma Family / Log Link

12
The Link Function Link function directly characterizes how the linear combination of the predictors is related to the prediction on the original scale –e.g., predictions from the identity link -- which is used in OLS -- equal:

13
The Log Link Log link is most commonly used in literature When we adopt the log link, we are assuming: ln(E(y/x))=Xβ GLM with a log link differs from log OLS in part because in log OLS, one is assuming: E(ln(y)/x)=Xβ ln(E(y/x) E(ln(y)/x) i.e. log of the mean mean of the log costs

14
ln(E(y/x) E(ln(y)/x) VariableGroup 1Group 2 Observations 11535 245 37555 Arithmetic mean45 Log, arith mean cost3.8066623.806662 * Natural log 12.708053.555348 23.806662 34.3174884.007333 Arith mean, log cost3.6107343.789781 * Difference = 0; Difference = 0.179047

15
Comparison of Results of GLM Gamma/Log and log OLS Regression VariableCoefficientSEz/Tp value GLM, gamma family, log link Group 20.0000000.4057300.001.000 Constant3.8066620.28689413.270.000 Log OLS Group 20.1790480.4924940.360.74 Constant3.6107340.34824610.320.000

16
The Power Link Function Statas power link provides a flexible link function It allows generation of a wide variety of named and unnamed links, e.g., –power 1 = Identity link; = B i X i –power.5 = Square root link; = (B i X i ) 2 –power.25: = (B i X i ) 4 –power 0 = log link; = exp(BiXi) –power -1 = reciprocal link; = 1/(B i X i ) –power -2 = inverse quadratic; = 1/(B i X i ) 0.5

17
Negative Power Links Retranslation of negative power links to the raw scale: When using a negative power link, negative coefficients yield larger estimates on the raw scale; positive coefficients yield smaller estimates

18
Selecting a Link Function There is no single test that identifies the appropriate link Instead can employ multiple tests of fit –:Pregibon link test checks linearity of response on scale of estimation –Modified Hosmer and Lemeshow test checks for systematic bias in fit on raw scale –Pearsons correlation test checks for systematic bias in fit on raw scale –Ideally, all 3 tests will yield nonsignificant p-values Others (e.g., Hardin and Hilbe) have proposed use of (larger) log likelihood, (smaller) deviance, AIC and BIC statistics

19
The Family Specifies the distribution that reflects the mean-variance relationship –Gaussian: Constant variance –Poisson: Variance is proportional to mean –Gamma: Variance is proportional to square of mean –Inverse Gaussian or Wald: Variance is proportional to cube of mean Use of the poisson, gamma, and inverse Gausian families relax the assumption of homoscedasticity

20
A constructive test that recommends a family given a particular link function Implemented after GLM regression that uses the particular link The test predicts the square of the residuals (res 2 ) as a function of the log of the predictions (lnyhat) by use of a GLM with a log link and gamma family to –Stata code glm res 2 lnyhat,link(log) family(gamma), robust If weights or clustering are used in the original GLM, same weights and clustering should be used for modified Park test Modified Park Test

21
Recommended family derived from the coefficient for lnyhat: –If coefficient ~=0, Gaussian –If coefficient ~=1, Poisson –If coefficient ~=2, Gamma –If coefficient ~=3, Inverse Gaussian or Wald Given the absence of families for negative coefficients: –If coefficient < -0.5, consider subtracting all observations from maximum-valued observation and rerunning analysis Recommended Family, Modified Park Test

22
Variance function: V(u) = u^2 Link function: g(u) = ln(u) [Gamma] [Log] Deviance = 214.8893 Log likelihood = -20782.51 AIC 20.7895 BIC -14933.71 cost CoefStd ErrzP>|t|95% CI treat -.0361495.015123 -2.390.017-.065790 -.006510 dissev1.1250814.101609 1.230.218-.074696.324232 dissev2 -.6837741.076062 -8.990.000-.832853 -.534696 blcost.0069198.023707 0.290.770-.039545.053385 blqscore -.1993532.046650 -4.270.000-.290786 -.107921 bledvis.0158549.005082 3.120.002.005894.025816 _cons 9.756198.090169108.200.000 9.57947 9.93293 Example, GLM gamma/log miscand1.dta glm cost treat dis* bl*,link(log) family(gamma)

23
FITTED MODEL: Link = Log ; Family = Gamma Results, Modified Park Test (for Family) Coefficient: 1.1680 Family, Chi2, and p-value in descending order of likelihood FamilyChi2P-value Poisson:0.06750.7951 Gamma:1.65390.1984 Gaussian NLLS:3.25990.0710 Inverse Gaussian or Wald:8.01930.0046 Results of tests of GLM Identity link Pearson Correlation Test:0.9688 Pregibon Link Test:0.8529 Modified Hosmer and Lemeshow:0.8818 GLM Diagnostics, log/gamma miscand1.dta

24
Variance function: V(u) = u Link function: g(u) = ln(u) [Poisson] [Log] Deviance = 25171259 Log likelihood = -1296805 AIC = 1296.812 BIC = 2556110 cost CoefStd ErrzP>|t|95% CI treat -.0342075.00041-83.640.000-.035010 -.033406 dissev1.1209046.00274 44.120.000.115533.126276 dissev2 -.6857563.00206-333.40.000-.689788 -.681725 blcost.0072812.00064 11.410.000.006031.008532 blqscore -.1970875.00125-157.60.000-.199538 -.194637 bledvis.0159274.00014 116.70.000.015660.016195 _cons 9.756579.0024340140.000 9.75182 9.76134 Change Family to Poisson and Rerun Model miscand1.dta

25
FITTED MODEL: Link = Log ; Family = Poisson Results, Modified Park Test (for Family) Coefficient: 1.1627 Family, Chi2, and p-value in descending order of likelihood FamilyChi2P-value Poisson:0.06210.8032 Gamma:1.64600.1995 Gaussian NLLS:3.17340.0748 Inverse Gaussian or Wald:7.92490.0049 Results of tests of GLM Identity link Pearson Correlation Test:0.9882 Pregibon Link Test:0.8136 Modified Hosmer and Lemeshow:0.8928 GLM Diagnostics, log/poisson miscand1.dta

26
Passes Tests, But Can We Improve the Link? Iteratively evaluate power links (in 0.1 intervals) between -2 and 2 –Use the modified Park test to select a family –Evaluate the fit statistics –Dont show you the results here, but I then fine tune the power link in 0.01 intervals within candidate regions of the power link

27
Results for Selected Power Links PowerFamilyPearsonPregibonmH&M -0.2Poisson0.98350.70890.8929 -0.1Poisson0.98650.7610.8943 0.0Poisson0.98820.81360.8928 0.1Poisson0.99050.86690.9063 0.2Poisson0.99340.92090.9559 0.3Poisson0.99690.97370.9461 0.4Poisson0.99910.97300.9359 0.5Poisson0.99460.92010.9369 0.6Poisson0.98950.86780.9008 0.7Poisson0.98390.81640.8125 0.8Poisson0.97780.76610.7444 miscand1.dta

28
Individual Criteria Do Not Uniquely Identify Link, But… PowerFamilyPearsonPregibonmH&M -0.2Poisson0.98350.70890.8929 -0.1Poisson0.98650.7610.8943 0.0Poisson0.98820.81360.8928 0.1Poisson0.99050.86690.9063 0.2Poisson0.99340.92090.9559 0.3Poisson0.99690.97370.9461 0.4Poisson0.99910.97300.9359 0.5Poisson0.99460.92010.9369 0.6Poisson0.98950.86780.9008 0.7Poisson0.98390.81640.8125 0.8Poisson0.97780.76610.7444 miscand1.dta

29
Can We Develop a Summary Measure? PowerFamilyPearsonPregibonmH&MSummary -0.2Poisson0.98350.70890.8929.096426 -0.1Poisson0.98650.7610.8943.068476 0.0Poisson0.98820.81360.8928.046376 0.1Poisson0.99050.86690.9063.026586 0.2Poisson0.99340.92090.9559.008356 0.3Poisson0.99690.97370.9461.003607 0.4Poisson0.99910.97300.9359.004839 0.5Poisson0.99460.92010.9369.010395 0.6Poisson0.98950.86780.9008.027428 0.7Poisson0.98390.81640.8125.069124 0.8Poisson0.97780.76610.7444.120533 miscand1.dta

30
LL, AIC, BIC, and Deviance PowerFamilyLLAICBICDeviance -0.2Poisson-1296851.71296.858725562042571353 -0.1Poisson-1296825.51296.832525561522571300 0.0Poisson-1296804.81296.811825561102571259 0.1Poisson-1296789.71296.796725560802571229 0.2Poisson-1296779.91296.786925560602571209 0.3Poisson-1296775.51296.782525560522571200 0.4Poisson-1296776.31296.783325560532571202 0.5Poisson-1296782.31296.789325560652571214 0.6Poisson-1296793.51296.800525560882571236 0.7Poisson-1296809.81296.816825561202571269 0.8Poisson-1296831.11296.838125561632571311 miscand1.dta

31
Why Not Simply Use AIC and BIC? In the current example: –AIC, BIC, log likelihood, and deviance all agreed –They yielded an answer that was similar answer to that from the Pearson correlation test, Pregibon link test, and modified Hosmer and Lemeshow tests Power link 0.3 AIC, BIC, log likelihood, (and deviance?) already commonly used for decisions about model fit Why do we need the new tests?

32
AIC / BIC There are at least 3 reasons why in the long run log likelihood, AIC, BIC, and deviance are unlikely to be the recommended tests for identifying the appropriate link function First, when there are a large fraction of observations with zero cost: –The recommendations from log likelihood / AIC agree with each other –The recommendations from BIC / deviance agree with each other –But the log likelihood/AIC recommendations differ from the BIC/deviation recommendations

33
AIC / BIC (2) Second, the 4 statistics arent stable across families, and the shifts in their magnitude across families do not provide information about which family/link is best For example, in a dataset where the modified Park test recommends a gamma family for power links 0.5, the magnitude of the AIC statistic shifts from ~18 for the gamma family to ~454 for the poisson family –Although the smaller AIC values associated with power links 0.5 are actually superior

34
AIC / BIC (3) Third, while this instability across families is less of a problem when our statistical packages offer 4 continuous families only, it will eliminate comparability across links when statistical packages begin to offer more flexible GLM power families –i.e., when we dont have to choose between Gaussian (0) and poisson (1) families, but instead can use a family of 0.7 In this case, given that each power will be associated with a slightly different family, it will be impossible to compare the resulting AIC/BIC statistics

35
Basu and Rathouz (2005) have proposed use of extended estimating equations (EEE) which estimate the link function and family along with the coefficients and standard errors Tends to need a large number of observations (thousands not hundreds) to converge Currently cant take the results and use them with a simple GLM command (makes bootstrapping of resulting models cumbersome) Extended Estimating Equations

36
GLM Disadvantages Disadvantages –Can suffer substantial precision losses If heavy-tailed (log) error term, i.e., log-scale residuals have high kurtosis (>3) If family is misspecified

37
The distribution of cost can pose problems for common parametric tests of cost Responses in the literature that suggest that we should evaluate something other than the difference in the sample mean (or a direct transformation of this difference) – e.g., nonparametric tests of other characteristics of the distribution or transformations of cost – generally create more problems than they solve Use of more flexible models that evaluate a direct transformation of the difference in cost generally pose fewer problems –Does require we identify functional forms for the relationship between the predictors and the mean and for the variance structure Review

38
EXTRA SLIDES

39
Fine Tuning (1) PowerFamilyPearsonPregibonmH&MSummary 0.31Poisson0.99730.97900.9399.00406 0.32Poisson0.97600.98440.9394.00392 0.33Poisson0.99800.98970.9390.00383 0.34Poisson0.99840.99510.9386.00379 0.35Poisson0.99880.99960.9382.00382 0.36Poisson0.99920.99430.9378.00390 0.37Poisson0.99960.98890.9373.00405 0.38Poisson1.00000.98360.9367.00427 0.39Poisson0.99950.97830.9363.00452 miscand1.dta

40
Fine Tuning (2) PowerFamilyLLAICBICDeviance 0.31Poisson-1296775.3241296.78232556051.22571199.801 0.32Poisson-1296775.2241296.78222556051.02571199.600 0.33Poisson-1296775.1761296.78222556050.92571199.505 0.34Poisson-1296775.1811296.78222556050.92571199.514 0.35Poisson-1296775.2381296.78222556051.02571199.628 0.36Poisson-1296775.3371296.78232556051.22571199.847 0.37Poisson-1296775.5091296.78252556051.62571200.170 0.38Poisson-1296775.7231296.78272556052.02571200.598 0.39Poisson-1296776.9891296.78302556052.52571201.130 miscand1.dta

41
AIC / BIC (When there are a large fraction of 0s) log likelihood / AIC recommendations differ from BIC / deviance recommendations 4 statistics arent stable across families, and shifts in their magnitude across families do not provide information about which family/link is best

42
Different Recommendations PowerFamilyLLAICBICDeviance -0.5Gamma-6408.418.767-3144.61094.8 -0.4Gamma-6399.118.740-3127.71111.8 -0.3Gamma-6390.318.714-3103.91135.6 -0.2Gamma-6382.318.691-3073.61165.9 -0.1Gamma-6374.118.667-3035.81203.7 0.0Gamma-6359.118.623-2979.51259.9 heroin2.dta

43
4 Statistics Arent Stable Across Families PowerFamilyLLAICBICDeviance -1.2Gamma-4495.5218.0061-2985.7884.2 -1.1Gamma-4495.3518.0054-2986.1283.9 Gamma-4495.1918.0048-2986.4483.6...... 0.4Gamma-4493.9918.0000-2988.8581.2 0.5Gamma-4493.9717.9999-2988.8981.1 0.6Poisson-113533454.16219123222193 0.7Poisson-113504454.04219066222136 0.8Poisson-113502454.03219061222131 0.9Poisson-113526454.13219109222178 1.0Poisson-113576454.32219210222280 rchapter5.dta

44
4 Statistics Arent Stable Across Families PowerFamilyPearsonPregibonmH&M -1.2Gamma.0102.0004.0275 -1.1Gamma.0115.0007.0411 Gamma.0144.0010.0673..... 0.4Gamma.5259.4715.5783 0.5Gamma.6181.6064.6607 0.6Poisson.8783.6736.5934 0.7Poisson.9286.8231.4050 0.8Poisson.9853.9812.3191 0.9Poisson.9514.8582.4205 1.0Poisson.8818.7021.5134 rchapter5.dta

45
Limitations If Power Family Becomes Available Generally not a big problem when we are limited to the 4 named continuous families –Because, as in the example, within families we can look for the power at which the statistics reach a maximum (ll) or maximum (AIC, BIC, deviance) When a more flexible GLM family is added to our statistical packages that allows a family of 0.7 or 1.3, rather than forcing us to round to a poisson family (1.0), the change in the scale of the AIC, etc., will make these statistics difficult, if not impossible, to use

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google