1. American Community Survey (ACS) Briefing on Messaging and Mail Package Assessment Research f or the United Nations Economic Commission For Europe Tasha Boone, United States Bureau Of the Census April 29 – 30, 2015

2. Contents  American Community Survey Basics  Challenges  Research Goals  Overview of Research Projects  Focus Groups  Key Message Findings  Proposed Alternate Mail Designs  Key Mail Package Findings  Online Visual Testing & Key Findings  Refined Alternate Mail Designs  Key Cumulative Findings  Reports  Next Steps 2

3. American Community Survey Basics  Previously part of the 10-year Decennial Census  A large national mandatory survey that goes to 3.5 million addresses each year to produce over 11 billion estimates  Designed to produce critical information on small areas and small populations  Covers over 35 topics that cut cross demographics, economic, social and housing  Data Collection Modes:  Internet/Web  Mail  Telephone  Personal Visit 3 Month 1: Online Response Month 1: Mail Response Month 2: Computer-Assisted Telephone Interviewing (CATI) Month 3: Computer-Assisted Personal Interviewing (CAPI)

4. Challenges  Lack of general awareness of the ACS  Confusion over the difference between ACS and the Decennial Census  Concerns from respondents/public about perceived intrusiveness and burden of the survey  A re-examination around making the ACS voluntary  Marketing a survey 4 4

5. Research Goals  Develop and test messages and mail package refinements to increase ACS self-response rates  Apply insights from ACS message testing to support general outreach, data dissemination, materials development, and call center and field operations  Help inform the 2020 Census Integrated Communications Program 5

6. Overview of Research Projects  Messaging Research 1.Mental models interviews with internal stakeholders 2.Key informant interviews with stakeholders 3.Focus groups 4.Message testing - Benchmark survey 5.Message testing - Refinement survey  Mail Package Assessment 6. Creative design of mail package alternatives and messages to test 7. Focus groups and one-on-one interviews 8. Quantitative online visual testing  Experimental Testing - Conducted by Census in 2015  Independent reviews – Don Dillman, PhD (survey methodologist), United States Census Bureau Advisory Committees 6

7. Messaging Research and Mail Package Assessment Focus Groups  Focus groups were held for message testing and mail package testing  Focus groups were held in seven cities. Selected cities based on demographics, low response rates, and skepticism about the survey  The cities were diverse in their geographic and racial composition. Participants recruited from the following communities: Hispanic; African-American; Asian; American Indians living on tribal lands, and low-income  Each focus group had approximately 24 to 28 participants  The mail package focus group included 30-35 one-on-one interviews that spanned 45 minutes to 1 hour 7

8. Key Message Findings  America knows Census—but not American Community Survey  Why would the government need to know this?  The government has this information  Distrust of government is pervasive  Community is key  Language barriers  Good tools and training are key in the field  Local influencers are trusted messengers 8

9. Design of Alternative Mailing Packages Description  Use learnings from messaging research to inform language and designs Approach  Five initial themes presented, narrowed to three after input from Census 9 “Community”“Official”“Patriotic” “Personal”“Historical”

10. Current ACS Design vs “Community” Mailing Design Used messaging and mail package research, feedback from Dr. Don Dillman and Census advisory committee feedback to inform changes to language Current ACS Envelope “Community” Proposed initial design 10

11. Current ACS Design vs “Official” Mailing Design Current ACS Envelope Proposed initial design “Official” Proposed initial design 11

12. Current ACS Design vs “Patriotic” Mailing Design Current ACS Envelope Proposed initial design “Patriotic” 12

13. Current ACS Design vs “Personal” Mailing Design Current ACS Envelope Proposed initial design “Personal” 13

14. Current ACS Design vs “Historical” Mailing Design Current ACS Envelope Proposed initial design “Historical” 14

15. Key Mail Package Findings  The current package was seen as most effective  Consider a mix of effective design elements  The name “American Community Survey” does not register with the public  More direct messages that tell the respondent what action you want them to take 15

16. Current ACS Design vs Revised “Community” Designs Current ACS Envelope Design of “community mail design after message testing Design of “community mail design after mail package focus group testing 16

17. Current ACS Design vs Revised “Official” Designs Current ACS Envelope Design of “official” mail design after message testing Design of “official” mail design after mail package focus group testing 17

18. Current ACS Design vs Revised “Patriotic” Designs Current ACS Envelope Design of “patriotic” mail design after message testing Design of “patriotic” mail design after mail package focus group testing 18

19. Online Visual Testing Survey Design  Mail Sort Exercise: an online simulation selection of print mail to assess what designs were noticed, opened, and recalled  Image Click Analyzer: explored how participants interacted with mail designs by asking respondents to click on areas of designs which caught their attention  Message Highlighting Tool: instructed respondents to highlight the words and phrases in the text of ACS letters that they found most compelling 19

20. Key Findings from the Online Visual Testing  Visual design elements and deadline messaging can have a significant impact on how people read and remember mail items  The U.S. Census Bureau logo should be prominently featured on mail items  Key words and phrases in letters should be emphasized using callout boxes, line spacing, and bolded text  “Your response is required by law” attracts more attention than any other message  It is possible to overdo the commercial “marketing” look and feel 20

21. 21 Proposed Mail Designs for use in Field Testing “Official” mail package:  Builds on successful elements of existing ACS mail package  Introduces improvements to visual design and messaging suggested by our research “Official”“Control” Looks very similar

22. 22 Proposed Mail Designs for use in Field Testing “Blended” mail package:  Incorporates successful aspects of “Official” design with evocative visuals of “Patriotic” theme  Intersperses straightforward “governmental” pieces with more designed elements to catch recipients who may be drawn to more eye-catching pieces “Blended” “Control”

23. Cumulative Findings 1.Emphasize the highly favorable Census brand in ACS materials 2.Use visual design principles to draw attention to key messages and help respondents better navigate materials 3.Use “governmental” appearance over a “marketing” approach 4.Emphasize effective “mandatory” messaging 5.Demonstrate benefits of ACS participation to local communities 23 Twenty eight cumulative findings……here are five key ones:

24. Reports  The following reports are final:  Mental Models Interviews with Internal Stakeholders  Key Informant Interviews with Stakeholders  Deliberative Focus Groups with Stakeholders  Message Testing: Benchmark Survey  Message Testing: Refinement Study  Mail Package Research  Access reports at:  http://www.census.gov/acs/www/library/by_year/2014 http://www.census.gov/acs/www/library/by_year/2014 24

25. Next Steps  Continue to research and test changes to messaging and the mailing package materials  Results will be used to enhance the mailing package, the data collection instrument and materials, and the survey educational materials  Continue to measure trends over time 25

26. Continue the ACS Conversation Connect on Social Media: @uscensusbureau facebook.com/uscensusbureau Email: tasha.r.boone@census.gov acso.users.support@census.gov Receive email updates: https://public.govdelivery.com/accounts/USCENSUS/subscriber/new Find more information: census.gov/acs acsdatausers.org 26

27. 27 Questions