2005 Customer Satisfaction Study September 2005 NASA Earth Observing System Data and Information Systems

2 What is the ACSI? The only standardized measure of customer satisfaction covering more than 200 companies in 40 industries; about 66% of the U.S. economy –Including over 70 U.S. Federal Government agencies A quarterly snapshot of the national economy’s health Results published quarterly –For more information, see

3 Who is FCG?  A franchise of the Department of the Treasury  Assists agencies in managing change  Has three centers of excellence: Management consulting Executive coaching Customer/employee satisfaction metrics Executive Agent for American Customer Satisfaction Index Generic Office of Management and Budget clearance Streamlined contracting process

4 Who is CFI Group? Founded in 1988; headquartered in Ann Arbor, Michigan 13 offices, 150 full-time consultants and researchers worldwide CFI system is the source for University of Michigan’s American Customer Satisfaction Index (ACSI)

5  Which NASA EOSDIS customer segment was surveyed? This customer segment includes individuals who have accessed NASA EOSDIS data and/or products by means of a NASA EOSDIS Data Center (DAAC).  How were the NASA EOSDIS customers identified? NASA EOSDIS provided a list of 21,871 addresses for people who have used NASA EOSDIS data and/or products. The data center was identified within the customer list. After cleaning the sample list CFI Group sent out 17,424 invitations asking customers to participate in the online survey. During the initial mailing over 3,800 s were bounced back to CFI Group due to an invalid address. The survey was available online June 28, 2005 – July 20, A total of 1,395 surveys were completed. CFI Group used 1,263 surveys for analysis. Project Background

6 78* NASA EOSDIS Results  The Customer Satisfaction Index for NASA EOSDIS is… NASA EOSDIS Aggregate Segment  The Customer Satisfaction Index score is derived from customer responses to three questions in the survey: –How satisfied are you overall with the products and services provided by the Data Center (82)? –To what extent have the data, products and services provided by the Data Center fallen short of or exceeded your expectations (73)? –How well does the Data Center compare with an ideal provider of scientific data, products and services (76)?  This score is 6 points higher than the 2004 American Customer Satisfaction Index for the Federal Government overall (72.1).

7 Comparison Across Government Agencies Customer Satisfaction Index +3

8 NASA EOSDIS Customer Satisfaction Model

9 NASA EOSDIS 2004 vs Comparisons Significant increases from 2004 in Product Selection and Product Quality

10 NASA EOSDIS Customer Satisfaction Model

11 x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x 1 x 3 x 4 x 5 x 6 y1y1 y2y2 y3y3 y 3 y 2 y 1 11 22  x ixit i , for i=1,2,3 t=1,2 y jyjj  1, for j=1,2,3     x 2 The Math Behind the Numbers A discussion for a later date…or following this presentation for those who are interested.

12 Areas of Opportunity for NASA EOSDIS Focus on the quality and accessibility (71) (74) (76)

13 Product Quality Impact: 0.9 Only 28% had documentation delivered with the data… 70% of data products were HDF-EOS or HDF format “If possible send along the software to open HDF” “DAAC products are acceptable however their documentation could be a little better. I find the information provided regarding processing and things like data projections and units very confusing.” “If possible send along the software to open HDF” “DAAC products are acceptable however their documentation could be a little better. I find the information provided regarding processing and things like data projections and units very confusing.” Note: All score increases are statistically sig.

14 Product Search Impact: % used EOS Data Gateway to search for data and products which remains consistent with 2004 results (67% in 2004) Note: All score increases are statistically sig.

15 Product Search Score Comparison By method for most recent search “The search engine should produce only the data requested. For example, when I type MOD12Q1 in the search engine, I get every MODIS product generated.”

16 Product Selection and Order Impact: 0.8 Question wording changed from 2004 “For the different data formats, have summary of basic data use and interpretation, particularly for level 3 data, along with a test read file to make sure user is reading and converting the data correctly.” Over 50% are requesting results for Earth Science Research and 1/2 are from higher education

17 Kudos for Customer Support Personnel Score of 84 maintained vs  While only 21% used Customer Support during their most recent request/order, they are rating their experience well and this component has a substantial impact on overall Satisfaction with EOSDIS.  Overall Satisfaction with EOSDIS is higher for those who did use customer support when compared to those who did not (80 vs. 77)

18 Customer Support Impact: 1.4

19 Delivery Impact: % of customers’ data was delivered through FTP – which is consistent with 2004 survey results “Compress data before delivery - will cut download times drastically”

20 Desired Outcomes  NASA EOSDIS wants its customer to recommend and use EOSDIS services in the future. Desired customer behaviors were measured with a single question each. –“How likely are you to recommend the Data Center to a colleague?” scored an 89 43% state the way they heard about gathering data from EOSDIS is from a colleague. –“How likely are you to use the services provided by the Data Center in the future?” scored a 91  NASA EOSDIS customers were also asked if they had ever contacted the Data Center’s user services office to report a problem. –33% of of customer say they have contacted the Data Center’s user services office to report a problem.

21 Summary of Findings  NASA EOSDIS performs well compared to various Government benchmarks  NASA EOSDIS saw a significant increase in their overall Customer Satisfaction score when compared to 2004 results  Likelihood to Recommend increased in 2005 and almost half of respondents heard about NASA EOSDIS from a colleague (word of mouth really does matter!)  Higher priorities remain centered around the quality of products and the ability to access the products  Customer Support continues to score well, and remains important to those customers using the service

22 Score Comparison Data Centers show a 10-point variation in Satisfaction 2% n=29 5% n=65 8% n=96 28% n=359 11% n=138 33% n=419 7% n=92 3% n=35 2% n=30 % reflects percentage of aggregate score

23 Score Comparison Time of Last Data Product or Service Request Results consistent with 2004 where those whose last request was less than 3 months ago are rating EOSDIS higher across the board.

24 Score Comparison Higher satisfaction outside of the USA Respondents outside the USA continue to have a higher overall Satisfaction score with EOSDIS but as in 2004 their rating of Customer Support continues to be lower and both have approximately 20% who used Customer Support

25 Recommendations  Product Quality – Use terminology most users would understand and consider delivering documentation with the product  Product Search – Focus on lessons learned from data center search capability which can be applied to EOS Data Gateway  Product Selection and Order – Clearly describe data and give examples for how the particular data can be used  Customer Support – Praise staff for continuing to bring a high level of service to customers when needed oIdentify different practices used for customer support for USA users and outside of USA users  Use data centers strengths to create “best practices” which can be shared across the data centers

26 Additional Information Detailed survey results for all of the Federal services, including trends in performance and customer satisfaction, were updated in June 2005 and can be found on the website Many agencies share best practices through the interagency customer service forum Contact information –Sheri Teodoru CFI Group –Bernie Lubran Federal Consulting Group

27 Review of ACSI Results  A component score is a weighted average of the set of survey questions comprising a component or activity area. Responses to survey questions are given on a 1-10 scale, which is converted to a scale for score reporting.  An impact predicts the increase in satisfaction that would result from a 5-point increase in a component or input score.  Areas for improvement are those components or activities with a relatively low score and a relatively high impact on satisfaction. In the simplified example shown here, Product Selection would be an area of improvement due to its lower score and higher impact. EXAMPLE