1. The measurement and comparison of health system responsiveness
The measurement and comparison of
health system responsiveness
Nigel Rice, Silvana Robone, Peter Smith
(Centre for Health Economics, University of York)
XX Riunione Scientifica Siep settembre 2008

2. Introduction (1)
Patients’ views and opinions are an essential means for assessing the provision of health services
Traditionally, patients’ views were sought on the quality of care provided
Recently the concept of responsiveness has been promoted as a more desirable measure to judge health systems.
Responsiveness = the way in which individuals are treated and the environment in which they are treated, encompassing the notion of patient experience with the health care system (Valentine et al., 2003)

3. Introduction (2) ISSUE: data on Responsiveness are self-reported.
ISSUE: data on Responsiveness are self-reported.
Ex: “For your [child’s] last visit, how would you rate the experience of being involved in making decisions about your health care or treatment?”
Response categories: “Very good”, “Good”, “Moderate”, “Bad”, and “Very bad”.
Individuals may systematically interpret the meaning of the available response categories differentially across population sub-groups (Sadana et al., 2002).
REPORTING HETEROGENEITY: Responses are influenced by individuals' preferences and expectations. EX

4. Introduction (3)
Use of anchoring vignettes to address the issue of reporting heterogeneity.
Vignettes = descriptions of fixed levels of a latent construct, such as respons.
EX: “When the clinic is not busy, [Mamadou] can choose which doctor he sees. But most often it is busy and then he gets sent to whoever is free”. How would you rate [Mamadou’s] freedom to choose his health care provider? 1. Very good 2. Good 3. Moderate 4. Bad 5. Very bad
Any systematic variation across individuals in the rating of the vignettes can be attributed to reporting heterogeneity
OUR AIM: Evaluate the presence of reporting bias across socio-economic groups within countries, and how it is related to the characteristics of the individuals.

5. DATA (1) The World Health Survey
The World Health Survey
Launched by the World Health Organisation (WHO) in 2001, 70 countries
Survey modes: face to face interview and computer assisted telephone interviews
Samples: randomly selected (+ 18 years), sizes between 600 and 10,000
The Responsiveness Module
Sections: Needing Health Care and General Evaluation of the Health System, Seeing Health Care Providers, Outpatients and Care at Home, Inpatient Hospital, Vignettes
Responsiveness Domains: Autonomy, Choice, Clarity of communication,
Confidentiality, Dignity, Prompt attention, Quality of basic amenities, Social support
Our sample: Mexico, India and Philippines, inpatient care

6. DATA (2) Dependent Variables Independent Variables
Dependent Variables
2 out of the 8 domains, Dignity and Clear Communication (considered as most important by the respondents in the countries selected)
For each domain, 2 questions about respondents` experiences of contact with health systems.
Response categories: “very good”, “good”, “moderate”, “bad” and “very bad”.
Independent Variables
Education: categorical variable (7 categories) or continuous variable (number of years in education).
Gender: dummy variable, 1 if woman, 0 if man.
Income: categorical variable (quintiles of the distribution of household permanent income, 1 if in the lowest income quintile, 5 if in the highest one). Permanent income measured with the HOPIT model (Ferguson et al., 2003).
Age: continuous variable (years)

7. Econometric model The Hierarchical Ordered Probit Model (HOPIT)
The Hierarchical Ordered Probit Model (HOPIT)
Terza (1985), Tandon et al. (2003)
Two parts model:
Reporting behaviour equation + Responsiveness equation
Assumptions:
a) Response consistency
Individuals classify the vignettes in a way that is consistent with the rating of their own experiences of the service provided.
b) Vignette equivalence
The level of the variable represented by any one vignette is perceived by all respondents in the same way and on the same unidimensional scale (King et al., 2004, p.194).

8. Descrip. Stat.
Summary frequencies for the reporting of vignettes Mexico. .1 .2 .3 .4 .5 .6 %
own
vig1
vig2
vig3
vig4
vig5
i_clarity_communic
mean of verygood
mean of good
mean of moderate
mean of bad
mean of verybad

9. Results (1) Tests of homogenous reporting and parallel cut-point shift
Tests of homogenous reporting and parallel cut-point shift
(p-value for Wald Test under H0 )
Mexico
Homo
geneity
Parallel cut -
point shift
All
Inc.
Women
Age
Educ.
Dignity
Greeted and talked to respectfully
Privacy respected
Communication
Provides explained thing s
Enough time for questions
.000 .
000
.210
.012
.988
.851
.010
.083
.119
.005
.978
.738
.053
India
Educ
Provides explained things
.002
.00
.209
.007
.314
.557
.024
.001
.009
.160
.030
.003
.183
.044 5 3
.362
.199
392
.389
.094
.036
189
.862
.710
101
Philippines
.092
.619 .0 50
.716
.505
521
232
.011
.016
027 08
.250 .1 38
.004
.205
.714
.042
.560
.565
140
017
.069
.017 03
.159
.073

10. Results (2)
Estimated coefficients of permanent income in the reporting bias equation
MEXICO: positive for first and second cut-point, negative fourth cut-point
INDIA: mainly negative for all the cut points

11. Results (3)
Estimated coefficients of permanent income in the responsiveness equation
OPROBIT
For most items, positive and significant effect of income on responsiveness
HOPIT
Positive and significant effect of income on responsiveness
The magnitude of the coefficients is smaller than in the OPROBIT (up to 1/2 for India, 1/3 for Mexico and Philippines)
The influence of income on responsiveness is overestimated by the OPROBIT model

12. Conclusions
Preliminary results on the extent of reporting bias, using data from three countries contained within the WHS.
Heterogeneity in reporting behaviour exists, it is a function of income and education, but does not appear to be strongly related to age and gender.
Adjusting for reporting bias impacts on the estimated coefficients of the responsiveness equation when comparing the OPROBIT and HOPIT model
Future extensions: compare heath system responsiveness across countries
THANKS