1. Real Time Patient Feedback – Best Practice in the West Midlands Mary Parkes – NHS West Midlands Hilary Brown, Health Services Management Centre

2. Why did NHS West Midlands commission this Study Patient experience feedback centre stage Use of patient experience feedback growing Concern that much work is ad hoc Not sure how widely used - methods Find out how systematic use was – is it influencing commissioning PCTs & Trusts need to develop competencies No clear guidance on what is best practice Clarify place of real time

3. Research methodology Initial data gathering via internet – overview of users, methodologies, suppliers etc. to provide additional examples and case studies Health Care Commission, Patient Information Forum, DH, IDeA, LGA, National Social Marketing Centre, International Journal of Market Research Two academic literature searches – health management database and combined business and law database Questionnaires Semi-structured interviews

4. Findings of Research Users value being asked – little feedback action Culture - patient engagement led from top No one method suits all – need to choose Use of feedback increasing but not systematic Lack of comparable data for benchmarking Great emphasis on data – analysis capacity? Little evidence feedback influencing services Real time useful for hot spots – follow up? Once collected, not shared, little feedback

5. Engaging people and soliciting feedback – the practitioners perspective Range of methods required to maximise response rates, increase representation and avoid bias Patient/service user enthusiasm for feedback related to power to change or improve things for others Exercise only as good as resulting action Survey design should be informed by patient/service user – what are the right evaluation questions?

6. Research findings – methodologies Face-to-face - most inclusive method, takes longer and produces fewer responses, requires an investment in training and administration, qualitative and therefore rich source of data, interviewer can have moderating effect on responses Paper-based - cheap, convenient, user-friendly, obtains large quantities of quantitative data, preferred by certain demographics, self-administered – higher reporting of undesirable behaviour, comment or feedback cards can result in mostly extreme responses Hand-held devices – potential issues re: sample size and representativeness, generally user-friendly (devices can be adapted), great variety in number of questions (quantitative with some provision for free text), produce high number of responses in short timescales

7. Research findings – methodologies Kiosks – anonymous, self-selected sample, generally user- friendly, quantitative data but poor use - not visible enough, purpose not explained and no assistance or encouragement, time pressures etc, can be vulnerable to misuse Bedside terminals – self-selected, quantitative data, incentives for completion, use of mobile phones affect future use of bedside units Telephone interviewing – random consumer research, opportunistic follow-up, prearranged – different response rates and either intrusive or convenient, moderating results with interviewer, CATI – quick, cost-effective, less detailed responses, lack of universal coverage as land lines declining

8. Research findings – methodologies Online questionnaires - internet coverage not universal, self-administered results in higher reporting of sensitive information, more complex structures, respondent fatigue more evident, ability to gather a lot of data in short time Other online methods - use of organisations own web-site; online communities – large volumes of qualitative data, quick and convenient, ongoing feedback over longer periods of time but require significant amount of management and maintenance; social media – health-related chatter

9. Benefits of using real-time – the practitioner perspective Ease of use Staff attitudes – real data, increased ownership, motivation Freshness of data - cause and effect easier to determine Perception of greater responsiveness by service users More likely to capture detail as not reliant on recall Prevent issues becoming problems and improve experience for others

10. Implications of using real-time – the practitioner perspective For what purpose is it being used? Unsystematic management of information Lack of expertise to interpret and analyse data Value of quantitative versus qualitative feedback How to turn feedback into actionable data Whose responsibility? Who is accountable?

11. Implications of using real-time – the practitioner perspective Query ease of use for all groups Potential positivity of response bias Reflection is an important element of giving feedback for experience rather than fact Not reaching people who are not using services Not closing the feedback loop – for staff, patients and service users

12. Implementation – the practitioner perspective Requires strategic and systematic approach Creating a structured process for quality improvement Building a system of leadership for improvement Developing a person centred approach Feedback to, and involvement of, staff Working with human responses to change Real time or right time Prioritising and decision making Managing expectations

13. Technology versus the human touch Real-time patient feedback through technological solutions has limitations, not least because it is largely limited to gathering responses to what questions There is a danger that technology will become a proxy for human contact Technology has an important part to play, but its introduction needs to be thought through carefully, to ensure that those providing direct care for patients, dont just regard themselves as transactional suppliers.

14. What is NHS West Midlands going to do with results of research Develop and publish How To Guide Run 2 workshops to disseminate research Begin dialogue with PCTs re what would help Commission development programme - WCC Agree how PCTs can be more systematic Define small no of areas to benchmark Model data capture and sharing Review practice Contribute to/influence national policy