1. The effects of panel conditioning on party support and political interest in Understanding Society
Nicole Martin
University of Essex

2. Motivation
“Each time the interview comes around, it gives me a chance to assess my progress since the last meeting. The interviewer’s questions about my happiness levels at home, at work, and with friends and family are really the only time at which I am made to stop and think about each area of my life. This way, it feels like an informal annual report on the business that is my life. Here’s to many more years of self-administered personal progress reports!”

3. Motivation Two specific biases to panel data:
(i) people leaving the study (attrition) in a non-random way.
(ii) people changing their answers as a result of being surveyed repeatedly (panel conditioning).
Panel conditioning isn’t always a negative thing.
Political attitudes are:
(i) an attitude not a behaviour.
(ii) conceivably affected simply by asking questions (don’t think of an elephant).
(iii) potentially sensitive information.

4. Motivation
An Innovation Panel experiment asked people questions about their body weight and height different amounts of time. Both are known to be misreported in line with cultural norms. Those who were asked the questions more times gave answers that appear less biased. Uhrig (2014) interprets this as the survey process gaining the respondent’s trust.
Fisher (2016) noticed differences in income reporting behaviour in Understanding Society, which appear to be panel conditioning where respondents mention more income components after they have been interviewed the first time.

5. Identity(fication) crisis
The easiest way to identify panel conditioning is through an experimental design.
But not normally feasible, so panel conditioning is usually subsumed into the effect of time and/or age, or into differences between different samples drawn at different points (with different patterns of attrition).
Luckily, there is a quirk of Understanding Society fieldwork that permits the identification of panel conditioning.

6. Identification strategy
Understanding Society has a new wave going into the field each January, but each wave lasts two years.
The sample of each month of the (General Population Sample) were stratified in such a way that each month is a representative sample of Great Britain.
Some questions repeated each year.
….So we can look at comparable samples being asked the same question at the same point in time, with their number of previous interviews the only difference between them.
Deal with attrition by keeping a balanced panel i.e. people interviewed (not proxy) at each wave 1-5.
The one exception is age.

7. Fieldwork Schedule 2008 (Sep-Dec) 2009 2010 2011 2012 2013 BHPS w18
GPS – GB w1
EMBS w1
GPS – NI w1
GPS – GB w2
EMBS w2
GPS – NI w2
BHPS w2
GPS – GB w3
EMBS w3
GPS – NI w3
BHPS w3
GPS – GB w4
EMBS w4
GPS – NI w4
BHPS w4

8. Respondent co-operation

9. Measuring party preference in Understanding Society
Now I have a few questions about your views on politics. Generally speaking do you think of yourself as a supporter of any one political party? (Which one?)
Do you think of yourself as a little closer to one political party than to the others? (Which one?)
If there were to be a general election tomorrow, which political party do you think you would be most likely to support?

10. Combined measure: original sample

11. Combined measure: original sample

12. Combined measure: new entrants to the panel

13. Combined measure: new entrants to the panel
Data: Pooled new entrants to UKHLS over waves 2-4 in their first two waves, who remained in the panel at least for the subsequent year.
For a real 2 percentage point gap, sample size needed c6494

14. Which party benefits? Data: 2011 year, UKHLS GPS sample
Wald tests show no difference in the effect for the destination party
Data: 2011 year, UKHLS GPS sample

15. Highly-engaged or unengaged with politics?
The effect of having one fewer interview is stronger for respondents with higher political interest
Data: 2011 year, UKHLS GPS sample

16. Political interest No effect on political interest
Data: UKHLS year 2011

17. Robustness checks
Interaction with whether respondent received post-general election questions.
Respondents in a shorter length balanced panel.

18. Conclusions
In Understanding Society, there appears to be a positive effect of panel conditioning on the probability of expressing a party preference – despite this generally declining over the time period being studied.
Short-lived? By the time the fourth wave had entered its first year, there was no difference.
And out-weighed by other changes in this case.
However, no effect seen on political interest.
Poses a (small) threat to external validity of long panels.
Poses a (small) threat to using panels as representative cross-sections.

19. What to do about it?
In this case, it would be possible to use wave year as an instrument for number of interviews, and adjust accordingly. But (by default) this would assume that each additional interview has the same effect.
Where surprising time trend noticed in panel data, should try to exclude panel conditioning as an alternative e.g. by looking at new entrants to a panel and evolution of responses (NB assumption about attrition over time).

20. Limitations
Can’t measure the effect on actual behaviour e.g voting, due to rotating question modules.
Maybe the two samples aren’t as balanced as they should be.
No information on the mechanism – is this a change in reporting, or a change in attitudes? (Or both).
Maybe treatment varied over waves. Fieldwork agency learning over the course of the wave.