Introduction to the key large-scale government surveys Vanessa Higgins, Jo Wathan and Reza Afkhami ESDS Government Centre for Census and Survey Research (CCSR) University of Manchester

Our mission… Intro to Economic and Social Data Service (ESDS) and how we can help you. What data is available, how it can be used in research, accompanying documentation Hierarchical nature of the data Registration, access, support services/resources STATA demo

ESDS Government One of four specialist services of ESDS. ESDS is a new national data service (since Jan 03) –ESDS Government –ESDS Longitudinal –ESDS Qualidata –ESDS International ESDS Government provides access and user support for key large-scale government surveys such as Labour Force Survey and General Household Survey Access remains via the UKDA

ESDS Government - some of the things we do! Helpdesk Survey pages incl. how to get started Online guides – SPSS, STATA, Weighting, Employment Research, Health Research User Group seminars (data users and data creators) Publications Database Derived variables - consistent over time - consistent with Census Teaching datasets Training

Which surveys? UK or GB surveys General Household Survey Labour Force Survey Family Resources Survey Expenditure and Food Survey (previously the National Food Survey and Family Expenditure Survey) ONS Omnibus Survey National Travel Survey Time Use Survey British Crime Survey/Scottish Crime Survey British Social Attitudes/Scottish Social Attitudes/Northern Ireland Life & Times/Young Peoples Social Attitudes Health Survey for England/Wales/Scotland Survey of English Housing (England only)

Microdata

QUALITY OF DATA (1) Two main data collectors: –Office for National Statistics (ONS) –NatCen Both have considerable experience –ONS Social Surveys started in 1941 –Natcen founded in 1969 (as SCPR) Permanent panels of highly trained field interviewers Management and Quality Checking (Relatively) high response rates – but falling Widespread use by secondary analysts

QUALITY OF DATA (2) Example of GHS data collection

What would you use the data for? Straightforward secondary analysis –To assess theoretical accounts –To quantify characteristics or behaviours –To challenge official views –To apply alternative definitions Context to your own primary research –Your research could be quantitative or qualitative –To assess the national context of an area study –To assess whether your sample is typical –To assess the scale of behaviours

Practical research uses of the data Looking at change over time Look at sub-populations Using the flexibility of the data to look at alternative definitions Looking within households

Change over time

Secondary analysis: change over time among sub-populations Marmot, M (2003)

Using successive cross- sectional data over time Pros… Reasonable amount of comparability Can pool years/quarters Data is representative at each time point Good at looking at impacts on groups Cons… Limits to continuity in the data (e.g. ethnic) Cannot establish individual change

Looking at small populations Only the Samples of Anonymised Records have larger sample sizes Many surveys with 10+k respondents –Permits minority groups to be represented –Rare subpopulations sample size may be too small… can consider combining years if appropriate

Survey data is subject to sampling error! Example: Pregnancy and Employment Using General Household Survey data alone there are only 168 pregnant women aged % Confidence interval for % pregnant women economically inactive 34.2 – 49.1% Combined 3 years data to obtain sample of 465 pregnant women Confidence interval using 3 years data: 34.9 – 43.9% Combining datasets to increase sample size

Using the flexibility of the data to look at alternative definitions What are hours worked? Is it just paid work? Or unpaid as well? Hours usually worked, or actually worked last week? In main job, or in any job? What about students? Overtime – paid? Overtime – unpaid? Lunch hours? Do non-workers work zero hours or should they be excluded?

Choosing a survey for research Which surveys cover your main topic? Which other topics are you interested in? Measurement over time Geography Respondents – whole household, children? Sample size

Using the data in teaching Methods courses –Using the data in a hands on manner –Using substantive exemplars to demonstrate a methodological point –Using the surveys as methodological exemplars Substantive courses –Making your point using data –Integrating methods into substantive courses Teaching datasets –General Household Survey –Labour Force Survey –British Crime Survey –Health Survey for England

QUALITY OF DOCUMENTATION Questionnaire Code book of Variables Description of Derived Variables Definitions Methodology including Sampling method Response achieved Population base Published reports

Health Survey for England documentation

Documentation - GHS Questionnaire

Continuous Population Survey Objectives Survey Aspects Benefits Risks Development programme

Objectives Develop a world class modular survey system Provide more coherent, better quality information Increase the precision of most existing statistical outputs Create a range of new outputs survey system with the flexibility to accommodate other surveys Maintaining continuity Delivering further efficiency savings

Development programme October 2004 – First formal consultation Spring 2005 First feasibility trial Summer 2005 Second feasibility trial Late 2005 Pilot Mid 2006 Parallel run January to March 2007 second consultation April 2007 Decision January 2008 Start date

Survey Aspects Integrates LFS, GHS, EFS, OMN, APS into single survey –Annual achieved sample size = 265,000 households (over half a million adults) GB –Common core module of questions to whole sample Topic modules to portions of sample Core and topic modules combined to construct a number of different viable interview combinations

Benefits a very large annual sample for core module variables; an improved, unclustered sample design; better representation at local authority district level, and improved weighting methodology Greater coherence in official statistics, fewer competing estimates between surveys

Risks Should the CORD or SCMS projects suffer significant delays in delivery, the CPS timetable would be affected. To a cost-effective CPS, interviewers would need to cover a larger range of topics. New e-based learning methodologies and techniques must be devised and validated in field trials. Equally, the proposed new fieldwork design and modular questionnaire would need to be tested in practice to ensure they are acceptable to respondents and interviewers while also demonstrating that outputs could be delivered within the timescales necessary for required quality. any decision to implement the CPS rests on a demonstrable ability to deliver the benefits anticipated while maintaining the integrity of key time series.