Local Surveys Tools 29th October 2009 The Acute Survey Co-ordination Centre: Jason Boyd

Improving local use of surveys by…  Adapting existing acute survey questionnaires  Expand question bank content  Remove or revise existing questions  Developing a “Day Case Surgery” survey  Provision of methodology documents comparable with national surveys  Provide advice to trusts on conducting surveys

Questionnaire content: methods Analysis of previous survey data Examining patient free-text comments Feedback survey of trusts/contractors (Maternity) Online survey of trusts/contractors to prioritise question development (Inpatients) Focus groups (Inpatients, Day case & Outpatients) Stakeholder consultation Cognitively tested new/revised questions

Local survey tools available Accident & Emergency Department Adult Inpatient Day Case Surgery Maternity Services Outpatient Department

Tools available for each survey Guidance manuals – “Guide to conducting a local … survey” – “Guide to mailing questionnaires, data entry and cleaning” – “Guide to analysis, reporting and dissemination of results” Online question bank tool Covering letters and multi-language sheet Sample construction and data entry spreadsheets Model service contracts Survey development report

Ask the right questions To evaluate service improvement initiatives Monitoring progress against national targets or trust-wide initiatives Comparison of results over time Analysis by different groups Online questionnaire tool

Expanded question bank tools Survey ‘Core’ questions in last national survey (n) Questions in online bank (n) Demographic questions in bank (n) A&E Department Maternity 96 (132)12015 Day case surgery Outpatients Inpatients

Questions designed by the trust…  Issues uniquely important to a trust may require questions not in the question bank Develop new survey questions Co-ordination Centre can advise Survey Contractors can advise Consider:  Effect on response rate  Placement of questions Pre-test questions

Ethics and R&D approval Local surveys all derived from Research Ethics Committee approved national surveys (except Day Case Surgery) Patient information material REC approved format Local surveys are “service evaluation” Ethics not required Trust R&D notified

Data protection Use unique patient reference numbers Use contracts with any external organisation Never link patient name and address information to their responses Be aware: patient identity can be inferred from data Anonymise the data of small groups Encrypt datasets between use

Sample size Consider the organisational level to be surveyed, i.e. trust, department, ward Estimate confidence intervals  Recommend minimum 100 respondents per group for any sub-group analysis Adjust for expected Response Rate  Consult RR from previous surveys  Understand demographic variations in RR

Maximise response rates A higher response rate results in more representative and reliable findings:  Publicise the survey to staff and the public  Questionnaire length  Cohesive questionnaire design and relevant questions  Reminder letters for postal surveys  Appropriate fieldwork period  Complementary collection methods  Real-time and frequent feedback  Telephone surveys  Qualitative methods such as interviews, focus groups, forums  Telephone help-line and translation services

Sampling methodology Sampling methods vary between surveys – random sampling: outpatients, A&E – consecutive sampling: inpatients, day case surgery, maternity – different exclusion and inclusion criteria Check for sampling errors Check for deceased patients Consider additional sample information

When to conduct a local survey Avoid duplication with the national patient survey programme: – Survey samples should not overlap (remember many patients will access more than one service in their care pathway) Avoid fieldwork during major holiday periods Consider staff workload

Using a survey contractor 12 CQC approved survey contractors Issues to consider – Staff time – Cost – Expertise – Data protection – Reporting – Perceptions of objectivity – Service contracts

Using results: access to previous data CQC publishes national survey results: – Trust-based data: benchmark reports, comparative scores workbook (benchmark data) – National data: historical tables, briefing note and press releases Individual trust spreadsheets provided to trusts Previous trust data held by Co-ordination Centre – available on request

Standardising data and duplicating benchmark data Changes in survey results can be due to differences in the sample composition! Results should be “standardised” when compared to the national surveys and over time All national programme benchmark tables standardised for age and sex (and for ‘route of admission’ for inpatients) Refer to the data cleaning and analysis guides to duplicate national programme results

Standardising data and duplicating benchmark data The Co-ordination Centre can supply: – All previous national benchmark data for the trust (‘core’ questions only) – Demographic proportion tables for the most recent national survey to allow standardisation of data to these – Advice on standardisation and data weighting

Getting help from the Co-ordination Centre… Telephone and support Advice on: Sampling Questionnaire design Analysis and standardisation Updating and adding additional local surveys

Contacting the Co-ordination Centre Current staff:  Sally Donovan (Manager)  Esther Howell (Senior Research Associate)  Jason Boyd (Senior Research Associate)  Geraldine Cooney (Senior Research Associate)  Elisabeth Garratt (Research Associate)  Harriet Hay (Research Associate) Telephone: