1. How to use public resources in such a way that government agencies can provide better service at lower cost to a growing number of citizens
Professor Claes Fornell
University of Michigan
Improving Public Services Group
Expert Meeting on Measuring Customer Satisfaction
The Office of Public Services Reform
London, June 27th 2005

2. Government System Requirements
Key requirements:
Accurate measurement
Operational information for improvement
Benchmarking to private sector

3. Introducing Two Best in Class Systems
Professor Jan Eklöf
Stockholm School of Economics
Dr Forrest Morgeson
University of Michigan
Share powerful, proven methodology
Breadth of application
Quality of guidance

4. A Brief History
► 1989 – Sweden starts the first national index of customer satisfaction
► 1993 – U.S. Congress passes Government Performance and Results Act (GPRA)
► 1993 – President Clinton signs Executive Order 12862, “Setting Customer Service Standards”
► 1994 – ACSI, based on the Swedish approach, is launched
► 1999 – President Clinton’s President’s Management Council (PMC) selects ACSI to measure 30 “high impact” agencies
2005 – Mexico pre-approve & Canada evaluating ACSI e-Government measurement

5. Number of Participating Agencies
U.S. Expansion since 1999
Number of Participating Agencies

6. Development cost $1.5 million
System Costs in the U.S.
Development cost $1.5 million
Agency cost $35,000 per annum

7. Jan Eklöf, IFCF and EPSI Rating
A System for Pan European Measurement of Customer Satisfaction with Public Services
Jan Eklöf, IFCF and EPSI Rating

8. EPSI Rating – History
1989: The first systematic customer satisfaction studies using similar principles as EPSI Rating were started in Sweden in 1989
1995 – 98: Development took place on a Pan European basis, with the European quality organisations (EFQM and EOQ)
1997 – 98: Feasibility study supported by the European commisison
1999: Pilot study with 11 countries and 5 common sectors
2000 Onwards: Annual studies conducted in 8 – 20 countries

9. EPSI Rating – Governance
EPSI Rating governed through IFCF (International Foundation for Customer Focus);
Established in 1998 by representatives from half a dozen academic institutions;
Current membership exceeds 20 academic institutions in 15 countries. Methodological development channeled through its expert committees;
Agreements for EPSI national studies are signed with national platforms (often including EOQ/EFQM partners);
Coordination, data analysis, Pan European reporting, etc. hosted by Stockholm School of Economics (SSE).

10. Common Sectors and Industries
EPSI Rating started with a focus on:
Retail banking
Insurance
Supermarkets
Telecoms (especially mobile phone operators)
Public services have been added step-by-step, including:
Public transport
Health
Public administration

11. EPSI Rating Core Output
A measurement of customer satisfaction as well as key success factors…
of individual agencies, companies and organisations in a specific market…
via periodical analyses by a respected, neutral institution…
using quality standards based on state-of-the art methodology

12. The Generic Structure Used
Loyalty
Trust

13. This Means
A causal model approach that is not merely descriptive but gives cause – effect relations
An understanding why users have the experienced “feelings” they express
A precise decision making approach
The relationships and strength of links (impact) are estimated
Qualitative measures to further validate the model

14. The Analytical Tool
The model is estimated using structural equation techniques (based on Partial Least Squares approaches);
Results are obtained for:
customer perceived index levels
impact
strength/importance of all aspects (manifests)
Model quality (accuracy, explanation power, robustness)
Can ‘by preference’ be used also for goods and services not paid directly at purchase/use

15. Design Aspects (the Pan European approach)
Survey design properties
Questionnaire design
Sample selection (250 per domain)
Field operation (experience required)

16. Questionnaire Design
A set of master questionnaires is developed and tested;
A common scale is used (1 - 10);
The wording is calibrated on the Pan-European level (with adoption to national “peculiarities”);
Each latent variable is covered by at least 3 manifest questions.

17. Some Contemporary Results - Public services
EPSI Rating and benchmarks
Health (a few country results)
Public transport
The Police

18. Public Service EPSI Sweden 1998 - 200450 55 60 65 70 75 80
Year 1998
Year 1999
Year 2000
Year 2001
Year 2002
Year 2003
Year 2004
Index value
Sector
Primary Health
Supplied by Govern
Supplied privately
Nursery homes
Education
Primary education
Secondary education
Academic education
Homes for elderly
The Police
The Tax Authority

19. EPSI Results Matter! Some examples:
Police: results play an important role in Swedish public sector policy, and caused substantive re-organisation, especially on county level (what crimes to prioritise?)
Health Sector: results used extensively for priority setting and re-structuring in the hospital sector (for example, Greece)
Public Transport: results used for benchmarking on Pan European level

20. Subscription Arrangement
The national platform offers subscription at standard charge, including national analysis and reporting;
Geographic dis-aggregation special arrangement is offered (with extended number of intreviews) at marginal cost;
Studies are conducted as ”syndicated research” keeping the costs down;
Reports with priority matrices, and presentations, as well as ’what-if” analyses are included

21. Critics and Criticism At institutional level:
We already have a system for this, what does the EPSI approach add?
Why should we benchmark? Our ”business” is unique;
We need the results more frequently, or at other intervals, or other levels of detail.
At national level:
We are not involved in ’strategic’ development;
The politicians do not understand it...

22. European Performance Satisfaction Index

23. ACSI and Government June 27, 2005
A Proven System Providing Valued Guidance To Public & Private Sector Managers
ACSI and Government June 27, 2005
Forrest Morgeson, University of Michigan

24. Participating Agencies
Department of Education, Grants Administration Payments
Department of Housing and Urban Development
Employment Standards Administration
Federal Aviation Administration
General Services Administration
Health Resources and Services Administration
Immigration and Naturalization Service
Internal Revenue Service
Office of Personnel Management
Pension and Welfare Benefits Administration
Small Business Administration
Social Security Administration
Bureau of Labor Statistics
Education Information Service
Environmental Protection Agency
Food and Drug Administration
Food Safety and Inspection Service
National Aeronautics and Space Administration
National Science Foundation
Occupational Safety and Health Administration
United States Customs Service
United States Mint

25. Participating Agencies
Administration for Children and Families
Alcohol, Tobacco and Firearms
Animal and Plant Health Inspection Service
Board of Veterans Appeals
Bureau of Consular Affairs
Bureau of Land Management
Census Bureau
Centers for Disease Control and Prevention
Centers for Medicare and Medicaid Services
Corps of Army Engineers
Employment and Training Administration
Export-Import Bank
Energy Information Agency
Farm Service Agency
Federal Emergency Management Agency
Federal Highway Administration
Federal Housing Administration
Fish and Wildlife Service
Administration for Children Food and Nutrition Service
Forest Service
National Agricultural Statistical Services
National Cemetery Administration
National Park Service
Natural Resources Conservation Service
Office of Justice Programs
Patent and Trademark Office
Pension Benefit Guaranty Corporation
Railroad Retirement Board
Student Financial Assistance
Veterans Benefits Administration
Veterans Health Administration

26. The Cause-and-Effect Model
Perceived
Quality
Customer
Complaints
Customer
Satisfaction
(ACSI)
Customer
Expectations
Citizen Trust

27. Enhanced Model for Government (Generic)
Process
Ease/Accessibility
Timeliness
Information
Perceived
Quality
Customer
Complaints
Clarity
Usefulness
Customer
Satisfaction
(ACSI)
Customer
Service
Courtesy
Professionalism
Customer
Expectations
Citizen Trust
Confidence
Recommend

28. What Agencies Receive 78 72
Process
Ease/Accessibility
Timeliness 65 73 78
15% 70
Information
Perceived
Quality
Customer
Complaints
Clarity
Usefulness 75 72
Customer
Satisfaction
(ACSI) 80 80
Customer
Service
Courtesy
Professionalism 70 76 80
Customer
Expectations
Citizen Trust 77
Confidence
Recommend
Scores on scale – How is the agency performing? 74

29. What Agencies Receive 78 72
Process
Ease/Accessibility
Timeliness 65
2.1 73 78
15% 70
Information
Perceived
Quality
Customer
Complaints
Clarity
Usefulness 75
0.5
3.5 72
-2.1
1.3
Customer
Satisfaction
(ACSI) 80
Customer
Service
0.4
0.6 80
Courtesy
Professionalism
4.0 70 76
0.1 80
Customer
Expectations
Citizen Trust 77
Confidence
Recommend
Scores on scale – How is the agency performing?
Impacts – What is the best leverage? 74

30. What difference has it made?

31. Case Study: Internal Revenue Service Poor delivery potentially compromising revenue collection
Before leveraging customer satisfaction …
Dissatisfied taxpayers
Disgruntled employees
“As only one taxpayer representative out of thousands across the country,
I have seen dozens of taxpayers severely damaged and even made
homeless by the IRS collection division.”
Anonymous Witness #1, IRS Employee Senate IRS Hearings 1997
“The long list of IRS horribles included arbitrary collection decisions, sale
of taxpayer lien property far below value, and the cavalier mistreatment
of taxpayers.”
Bob Zelnick, ABC Good Morning America September 26, 1997

32. Case Study: Internal Revenue Service A Key Discovery: ACSI Study 1999
►e-Filers vastly more satisfied
far fewer errors
quicker problem resolution
status tracking
faster refunds
►e-Filing was the key. IRS decides to:
● devote more resources to develop e-filing
● persuade more taxpayers to e-file

33. Number of e-filers nearly triples in 6 years
Case Study: Internal Revenue Service A performance measure…
Number of e-filers nearly triples in 6 years
“We realize we have more work to do, but the survey is just one more indication that the IRS reorganization and its emphasis on customer service are paying off. The satisfaction with IRS e-file won´t surprise any taxpayer who has used it. When they try it, they like it. It is fast, accurate and dependable.” - Charles O. Rossotti, IRS commissioner, December 17, 2001

34. Case Study: Internal Revenue Service A systematic improvement
Tax filer satisfaction increases
Increased awareness and usage of e-Filing
+25% from 1999

35. Government Improving more than Private Sector
National ACSI
+2% from 1999
+5% from 1999
Federal Government

36. How to use public resources in such a way that government agencies can provide better service at lower cost to a growing number of citizens
Professor Claes Fornell
University of Michigan
Information?
Jan Eklöf, EPSI Rating: ,
Guy Whitaker, CFI Group: ,