1. AIRS Conference, 2015
Dallas, TX
May 29, 2015
The 211 Quality Assurance Mystery Caller Program – The Design, Implementation and Performance Results to Date

2. Agenda The Design Why do a mystery caller program.
A call to action from supervisors.
Is quality from the outside looking in important?
How was the 211 Quality Assurance Mystery Caller Program Developed
Performance to Date
November 17, 2018

3. Agenda The Design Why do a mystery caller program
A call to action from supervisors
Is quality from the outside looking in important
November 17, 2018

4. Recent Research: Based on research conducted by Dynamic Markets, 2014
Customer Retention
(based on caller responses)
66% of callers that have been in contact with a provider’s engagement center choose to spend ($ and time and effort) with companies that treat them as individuals not just another call.
88% would rather spend with an organization that is easy to do business with

5. Recent Research: Based on research conducted by Dynamic Markets, 2014
How to manage customer engagement (based on organizational responses)
83% of supervisors feel they cannot deliver on behaviors that drive customer/caller perception of the CSR.
#1 reason, (a call to action)
THEY DON’T KNOW WHAT THE BEHAVIORS ARE

6. Agenda The Design Why do a mystery caller program
A call to action from supervisors
Is quality from the outside in important
How was the 211 Quality Assurance Mystery Caller Program Developed?
November 17, 2018

7. What are the behaviors? Referral /Loyalty Perceived Value
Overall
Quality of
Products &
Services
Cost of
Acquisition
& Usage
Market
Presence
& Reputation
Exxon Valdez – March 1989
Remember starting at 10
Who born after 1979
Enron 12/2001
Arthur Anderson
The Contact Center
November 17, 2018

8. The Filter What are the behaviors? The Caller The Organization
Understanding the filter.
Referral
/Loyalty
Perceived Value
Overall
Quality of
Products &
Services
The Caller
Cost of
Acquisition
& Usage
Market
Presence
& Reputation
The Contact Center
The Filter
The Contact Center
Strategy
The Organization
Goals/Objectives
Mission
Vision

9. The Hawthorne Effect What are the behaviors? The Caller
Understanding the filter.
Referral
/Loyalty
Caller
Perceived Value
Overall
Quality of
Products &
Services
The Caller
Cost of
Acquisition
& Usage
Market
Presence
& Reputation
The Contact Center
The Hawthorne Effect
The Contact Center
Strategy
The Organization
Goals/Objectives
Mission
Vision

10. Frederick Taylor What are the behaviors? The Contact Center
Referral
/Loyalty
Caller
Perceived Value
Overall
Quality of
Products &
Services
Cost of
Acquisition
& Usage
Market
Presence
& Reputation
The Contact Center
Frederick Taylor
The Contact Center
Strategy
Goals/Objectives
Mission
Vision

11. Key Performance Indicators – Set 1
What are the behaviors?
Referral
/Loyalty
Caller
Perceived Value
ASA
Key Performance Indicators – Set 1
Overall
Quality of
Products &
Services
Talk time
Cost of
Acquisition
& Usage
Market
Presence
& Reputation
The Contact Center
Frederick Taylor
The Contact Center
Calls handled per hour
Strategy
Goals/Objectives
Etc.
Abandon rate
Mission
Vision

12. Key Performance Indicators – Set 2
What are the behaviors?
Key Performance Indicators – Set 2
What percent of calls contain behaviors that make the caller feel that their question/issue is of secondary importance?
What percent of calls contained unspoken needs that were not identified by the specialist?
What percent of calls contain misunderstood directions or phone numbers causing call backs?
What percent of calls would cause the caller to call back hoping for a different specialist that would be empathetic to their plight?
What percent of calls contain irrelevant conversation thereby increasing talk time?
What percent of calls contain tragic phrases?

13. Difference between #1 and #2 – CONSENSUS REALITY “If we believe something long enough, whether or not it’s true, it becomes reality.”
# 1 From the inside out, drives consensus reality # 2 From the outside in – the callers perception! (the specifics of what the caller encounters and the criteria the caller uses to evaluate the center's performance)

14. Agenda The Design Why do a mystery caller program
A call from supervision
Is quality from the outside in important
How was the 211 Quality Assurance Mystery Caller Program Developed?
November 17, 2018

15. How was the 211 Quality Assurance Mystery Caller Program Developed?
* Built on Fraser’s SETS Program (Service Excellence Tracking )
* Fraser’s model based on multicollinearity and regression modeling of over 10,000 call center observations

16. How was the 211 Quality Assurance Mystery Caller Program Developed?
Fraser’s SETS Research Results:
5 major categories
The Greeting (The Initial Contact)
Listening, Confirmation, Questioning (Assessment and Clarification)
Appropriate Solution (Information and Referral Giving)
Professionalism (Communication Techniques)
Closing (Closure)

17. How was the 211 Quality Assurance Mystery Caller Program Developed?
March 2014
Qualitative research w/UW Worldwide senior management
Literature search including AIRS
Augmented criteria developed
April 2014
Roundtable discussions with UW Worldwide and 211 US Steering Committee Quality Council
April/May 2014
15 Center Benchmark
June 2014
Findings reported AIRS 2014 National Convention
November 17, 2018

18. Specialist Monitoring Using the Voice of the Customer Through Mystery Calling – 41, 211 specific criteria
Overall Quality of Specialist Performance
Contact
The Initial Greeting and Building Rapport
Closure
Assessment and Clarification
Communication Techniques
Build the Relationship with Professionalism 7 5
Information and Referral Giving 5 7 17

19. Agenda The Design Why do a mystery caller program
A call to action from supervisors
Is quality from the outside looking in important
How was the 211 Quality Assurance Mystery Caller Program Developed
Performance to Date
November 17, 2018

20. Participant deliverables
Documentation on how program was developed
In depth description of each criteria for training
What the auditor is listening for
Why is it important to the caller
Why is it important to the 211 center
Calibration forms
Definition of each criteria
Area to score
Kick off conference call with Fraser senior staff

21. Participant deliverables, cont.
Monthly mystery calls
23 scenarios
Random dates and times
Real time reporting, (agency specific and program overall)
Agency specific reporting design
Quarterly rollup and executive review
Direct access to Fraser senior staff
Quarterly calibration session with Fraser

22. Agenda The Design Why do a mystery caller program
A call to action from supervisors
Is quality from the outside in important
How was the 211 Quality Assurance Mystery Caller Program Developed
Performance to Date
November 17, 2018

23. COMMUNICATION TECHNIQUES
Overall Calls Score
Overall
Quality
CONTACT
ASSESSMENT & CLARIFICATION
INFORMATION GIVING
COMMUNICATION TECHNIQUES
CLOSURE

24. Managing Director BManers@The FraserGroup.org
Thank you
Barry Maners
Managing Director FraserGroup.org
The Fraser Group
Voice of the Customer Measurements