1. Local Surveys Tools 29th October 2009 The Acute Survey Co-ordination Centre: Jason Boyd

2. Improving local use of surveys by…  Adapting existing acute survey questionnaires  Expand question bank content  Remove or revise existing questions  Developing a “Day Case Surgery” survey  Provision of methodology documents comparable with national surveys  Provide advice to trusts on conducting surveys

3. Questionnaire content: methods Analysis of previous survey data Examining patient free-text comments Feedback survey of trusts/contractors (Maternity) Online survey of trusts/contractors to prioritise question development (Inpatients) Focus groups (Inpatients, Day case & Outpatients) Stakeholder consultation Cognitively tested new/revised questions

4. Local survey tools available Accident & Emergency Department Adult Inpatient Day Case Surgery Maternity Services Outpatient Department

5. Tools available for each survey Guidance manuals – “Guide to conducting a local … survey” – “Guide to mailing questionnaires, data entry and cleaning” – “Guide to analysis, reporting and dissemination of results” Online question bank tool Covering letters and multi-language sheet Sample construction and data entry spreadsheets Model service contracts Survey development report

6. Ask the right questions To evaluate service improvement initiatives Monitoring progress against national targets or trust-wide initiatives Comparison of results over time Analysis by different groups Online questionnaire tool

8. Expanded question bank tools Survey ‘Core’ questions in last national survey (n) Questions in online bank (n) Demographic questions in bank (n) A&E Department 5010012 Maternity 96 (132)12015 Day case surgery -12215 Outpatients 5812614 Inpatients 8720312

9. Questions designed by the trust…  Issues uniquely important to a trust may require questions not in the question bank Develop new survey questions Co-ordination Centre can advise Survey Contractors can advise Consider:  Effect on response rate  Placement of questions Pre-test questions

10. Ethics and R&D approval Local surveys all derived from Research Ethics Committee approved national surveys (except Day Case Surgery) Patient information material REC approved format Local surveys are “service evaluation” Ethics not required Trust R&D notified

11. Data protection Use unique patient reference numbers Use contracts with any external organisation Never link patient name and address information to their responses Be aware: patient identity can be inferred from data Anonymise the data of small groups Encrypt datasets between use

12. Sample size Consider the organisational level to be surveyed, i.e. trust, department, ward Estimate confidence intervals  Recommend minimum 100 respondents per group for any sub-group analysis Adjust for expected Response Rate  Consult RR from previous surveys  Understand demographic variations in RR

13. Maximise response rates A higher response rate results in more representative and reliable findings:  Publicise the survey to staff and the public  Questionnaire length  Cohesive questionnaire design and relevant questions  Reminder letters for postal surveys  Appropriate fieldwork period  Complementary collection methods  Real-time and frequent feedback  Telephone surveys  Qualitative methods such as interviews, focus groups, forums  Telephone help-line and translation services

14. Sampling methodology Sampling methods vary between surveys – random sampling: outpatients, A&E – consecutive sampling: inpatients, day case surgery, maternity – different exclusion and inclusion criteria Check for sampling errors Check for deceased patients Consider additional sample information

15. When to conduct a local survey Avoid duplication with the national patient survey programme: – Survey samples should not overlap (remember many patients will access more than one service in their care pathway) Avoid fieldwork during major holiday periods Consider staff workload

16. Using a survey contractor 12 CQC approved survey contractors Issues to consider – Staff time – Cost – Expertise – Data protection – Reporting – Perceptions of objectivity – Service contracts

17. Using results: access to previous data CQC publishes national survey results: – Trust-based data: benchmark reports, comparative scores workbook (benchmark data) – National data: historical tables, briefing note and press releases Individual trust spreadsheets provided to trusts Previous trust data held by Co-ordination Centre – available on request

18. Standardising data and duplicating benchmark data Changes in survey results can be due to differences in the sample composition! Results should be “standardised” when compared to the national surveys and over time All national programme benchmark tables standardised for age and sex (and for ‘route of admission’ for inpatients) Refer to the data cleaning and analysis guides to duplicate national programme results

19. Standardising data and duplicating benchmark data The Co-ordination Centre can supply: – All previous national benchmark data for the trust (‘core’ questions only) – Demographic proportion tables for the most recent national survey to allow standardisation of data to these – Advice on standardisation and data weighting

20. Getting help from the Co-ordination Centre… Telephone and e-mail support Advice on: Sampling Questionnaire design Analysis and standardisation Updating and adding additional local surveys

21. Contacting the Co-ordination Centre Current staff:  Sally Donovan (Manager)  Esther Howell (Senior Research Associate)  Jason Boyd (Senior Research Associate)  Geraldine Cooney (Senior Research Associate)  Elisabeth Garratt (Research Associate)  Harriet Hay (Research Associate) Telephone:01865 208127 Email:advice@pickereurope.ac.uk