Presentation on theme: "Don’t Stop Me Now Preparing for an ageing population"— Presentation transcript:
1 Don’t Stop Me Now Preparing for an ageing population Mystery Shopper SurveyFindings and Methodology
2 ContentsPart 1Why a mystery shopper survey is valuablePart 2Don’t Stop Me Now methodology – What we didPart 3Don’t Stop Me Now findings – What we foundPart 4Running your own mystery shopper survey - checklists
3 Mystery shopper survey Part 1 : Why is a mystery shopper survey valuable?
4 Why is a mystery shopper survey valuable? A Mystery Shopper Survey will enable you to:Capture an accurate and realistic picture of service user experiencesAssess whether service users are able to easily obtain information about their local area
5 Mystery shopper surveys and age-proofing Working with older mystery shoppers:Is a quick and cost effective method of assessing whether local services are age-proofed and accessible to older residentsWill enable councils to benchmark their performance against the findings in Don’t Stop Me NowWill enable councils to measure their own direction of travel by repeating the survey on a regular basis
6 Mystery shopper survey Part 2: Methodology from Don’t Stop Me Now – What we did
7 Part 2: Methodology from Don’t Stop Me Now Part 2 covers:The purpose and remit of the Don’t Stop Me Now mystery shopper surveyHow the Don’t Stop Me Now survey was conducted
8 Purpose and remit of the survey The purpose of the survey was to:Evaluate how well councils are able to direct older people to a range of local services designed to maintain their well-being and independence; andAscertain if councils are enabling older people to play a full and active role in the communityThe focus was on:Services and initiatives outside of social care; andLocal opportunities to keep fit, well, active and engaged
9 How the survey was conducted The mystery shoppersAged 50 – 702 male, 2 femaleAll lived actively and independently with no social care needsAll made the calls from their own homesCalls were spread over Monday – Friday and were made at different times of the dayEach used a common scenario and tested 10 – 12 councils each (49 councils in total)
10 How the survey was conducted: Making the calls The mystery shoppers:used a local phone directory to find the number(s) they felt were most appropriateadditionally used the internet to find numbers and information if they were comfortable doing soasked to be transferred to the relevant person or department if they could only locate a call centre number for the councilused their own personal details (name, age, address)prompted and probed if information was not forthcoming
11 How the survey was conducted: Scenario and script The common scenario: “My name is x, I currently live in # and I am moving to your area in the next few months. I am # years old / recently retired and I am looking for information on #”.The mystery shoppers were briefed to prompt and probe in all calls to assess the “best case” scenario. For example, if the council was struggling to provide information on opportunities to engage in voluntary work, the mystery shopper would provide some suggestions .e.g. “Is there a local Age Concern that you could put me in touch with?”If the council asked for the mystery shopper’s new address or postcode the response was “I don’t have a new address yet, I am just seeking general local information at the moment”.
12 How the survey was conducted: Areas tested In each call the shoppers asked for local information on one of the following areas:Opportunities to keep fit and activeOpportunities to socialiseTransportLearningVoluntary work and employmentIf a council was struggling to refer the call to the appropriate person or department then the mystery shopper would ask “Do you have a team or a person that specifically deals with issues for older people?”
13 Mystery shopper survey Part 3: Findings from Don’t Stop Me Now – What we found in 2007
14 Survey findings – November 2007 In 82% of calls Mystery Shoppers needed to prompt and probeIt was easiest to obtain information on leisure services for older people and most difficult to obtain information on employment and volunteering opportunitiesIn 69% of calls shoppers were referred to a website and no information was sent in the post56% of calls were dealt with on first contact, 35% were transferred once and the remainder transferred more than onceMost transferred calls were to adult social care
15 Mystery shoppers needed to prompt and probe in 82% of calls overall
16 Was information sent in the post? 69% were referred to the internet for more information.31% were sent information in the post* 71%of people over 65 have never used the internet
17 Good information provided on leisure, socialising and transport They told me about a leisure card based on ability to pay – it gives various discounts. They produced a directory for adults over 50 and will send it out. Plenty of sports facilities for the older person – Mystery shopperTry U3A – they do classes, nights out, pub visits, meals out - CouncilGo to George Lawson hall – it’s the time of your life for the over 50s! - CouncilWe have a very good transport system – buses and trains - all you need to get round. We have a transport map and it will contain all the information that you need. Freedom passes are available to all over sixty - councilI’ve actually learned a lot from doing this project and speaking to all the different councils. I never realised how many possible services are out there for people my age – mystery shopper
18 Reliance on the internet and transferred calls I felt shunted around and that there was no single place where I could get all the information – Mystery shopperIf you haven’t access to a computer you will spend hours on the phone! I asked the switchboard if they had leaflets they could send me and had no reply – mystery shopperThe lady had to ask a colleague. I felt sure I had been put through to the wrong department. I was kept on hold for several minutes….she really couldn't help me with anything else and didn't put me through to anyone else who might have been able to answer my questions – mystery shopperThe council said “Services for older people? I've never heard of this. I have been here for 2 years but never heard of that. We have social care and social welfare - go onto our website.” – Mystery Shopper
19 Mystery shopper survey Part 4: Running your own mystery shopper survey: Methodology checklists
20 Part 4: Methodology checklists Checklists cover:Project managementRemit, scenarios and scriptsAreas to testSelecting and training the mystery shoppersData captureAnalysis and PresentationUsing the results
21 Checklist 1 – Project Management Appoint a project managerProject manager will need awareness and expertise about local older people and services availableProject manager should consult other local experts to shape survey and analyse data (e.g. local analysts / local voluntary and community / older communityAgree timeframe for surveyAgree overall cost of survey
22 Checklist 2 – Remit, scenario and scripts Agree the remit of the survey at the outset (e.g. ease of finding appropriate contact in council / ease with which information was provided / type and quality of information provided)Agree a common scenario (e.g. I am about to move to your area / I live in this area and have recently retired)Decide on method(s) of contact (e.g. phone calls / face to face visits / )Decide on number and nature of contacts needed and over what periodEnsure contacts are made across a range of days and times (e.g. spread across all working days / opening hours)
23 Checklist 3 - Selecting the areas to test Possible options are:Opportunities to keep fit and active / leisure servicesOpportunities to meet new people / socialiseTransport options in the areaOpportunities to study / learn new skillsOpportunities to get involved in volunteeringEmployment options in the areaThis is not an exhaustive list. The range of area tested can be expanded or contracted depending on local issues.
24 Checklist 4 - selection and training of mystery shoppers Select shoppers with a range of ages from 50 upwardsEnsure a gender spiltEnsure diverse representation to reflect local communityProvide a training session to brief mystery shoppers on the purpose and remit of the workTake on board suggestions from the mystery shoppers, and adapt methods and script accordinglyWork with local voluntary and community sector partners to help select / train / brief mystery shoppers
25 Checklist 5 - Data Capture Consult the mystery shoppers on their preferred method of data capture (e.g. written pro-forma / IT solution)Consider recording calls to capture full conversation and to provide accurate quotes. Consult mystery shoppers on preferred methods of data recordingEnsure the method of data capture reflects each aspect of the remit of the survey (e.g. number of times call transferred / department / person transferred to / number of referrals to the intranet / offers to send written information / quality of information provided)Allow mystery shoppers to record overall experiences / any other information on each call
26 Part 6 – Analysis and presentation of the data Use expertise from in-house analystsProvide quantitative data (e.g. on number of calls / number of transferred calls / number of calls that provided spontaneous information / number of calls that needed prompting and probing)Separate data into areas tested e.g leisure / transport etcUse quotes from calls and comments from mystery shoppers to illustrate the content and usefulness of callsUse Audit Commission report and findings as a guide on presentation (part 3)
27 Checklist 6 - Using the results Compare results against Audit Commission findings from November 2007 to benchmark performance (part 3)Repeat survey on a regular basis (quarterly / bi-annual / annual) to benchmark the direction of travel of councilReport on results as part of scrutiny and performance process
Your consent to our cookies if you continue to use this website.