Incorporating Technology in CE’s New Design Laura Erhard (presenting, BLS) David Cantor (Westat) Carol House (NAS-CNSTAT) FedCASIC March 20, 2014

Outline Brief overview of current survey Overview of planned redesign Redesign guiding principles Major issues and decisions  Scope of new technology use  Role of the interviewer  Household Use of Technology  Resources and respondent help  Confidentiality and Technology 2

Current CE Design Interview Consumer Expenditure Survey Estimates Diary - 5 waves of personal interviews - 3 month recall - Most expenses collected - Estimates use: Large or recurring expenditures - 2 one-week household paper diaries - Contemporaneous recall -All expenses collected - Estimates use: Small, frequently purchased items

Redesign Motivation Evidence of measurement error Changes in technology and spending behaviors Need for greater operational flexibility High level of burden – linked to data quality 4

Redesign Guiding Principles Keep it simple Make sure the design works for all respondents (e.g., have technology options) Increase flexibility in content, modes, and technology Meet data requirements Reduce measurement error Keep costs neutral Have no negative effect on response rates 5

Design Overview: Visit 1 Advance mailing with token cash incentive Personal visit interview  Household roster  Demographics  Recall-based expenditures (3-month reference period) Diary training and placement Records training $20 interview incentive for household Goal: 45 minutes average 7

Design Overview: Diary Week All HH members 15+ maintain individual diary for 1 week Diaries are web based, accessible via computer or mobile device, or paper (on request) Centralized monitoring of diary reporting, interviewers following up with respondents as necessary Open-ended diary, designed to collect:  smaller, more frequently purchased items  Items more willing to report privately  Items an proxy respondent would not know $20 individual diarist incentives 8

Design Overview: Visit 2 Personal visit interview (goal: 45 minutes average)  Records-based expenditures (3-month reference period) – Items that respondents likely do not accurately know but could easily obtain from records – Items that respondents may know but may be more accurately reported using records  Diary review  Income and assets $20 or $30 interview incentive (dependent on record use) Goal: 45 minutes average 9

Design Overview: Wave 2 After each wave*: CE “experience package” as non-monetary incentive, for example:  Charts displaying HH expenditures vs. national average  Information sheet listing helpful government websites * Post wave 1 non-monetary incentive is pending further research regarding impact on wave 2 reporting. 12 months later: repeat Wave 1  Visit 1  Diary Week  Visit 2 10

Major Issues and Decisions 1. Background – what is the topic related to technology? 2. Recommendations from reviews – what types of ideas were recommended from outside sources? 3. Adopted approach – what choices were adopted CE’s redesign plan? 4. Considerations - what considerations were made when adopting the approach, particularly when differing from recommendations?

Scope of New Technology Use 1. Background/Issue: - Which components of the current data collection process could be replaced with newer technologies? 2. Recommendations from reviews: - Electronic diary (long term, short term, limited topics) - Recall module – Computer assisted self administered interview (CASI), Continued use of CAPI/telephone interviewing - Real-time recording of expenditures - Apps - Use of household records – receipt scanning (by interviewer/centralized repository); downloading household financial information 12

Scope of New Technology Use (cont’d) 3. Adopted Approach: - Web based electronic diary with paper back-up 4. Considerations: - Concern about respondent cooperation with self- administration of recall and with longer duration diaries - Concern about development resources and security with real- time apps and receipt scanning - Interviewer/centralized repository scanning of receipts viewed as impractical in large survey - Telephone interviews minimized 13

Role of the Interviewer 1. Background/Issue: - How does the field interviewer’s role change in presence of newer technology? 2. Recommendations from reviews: - Current interviewer role continues for CAPI recall modules - Interviewer becomes trainer/coach for newer technology adaptations (Reassurance, Training, Feedback, Continued encouragement) - Monitoring – centralized versus interviewer monitoring 14

Role of the Interviewer (cont’d) 3. Adopted Approach: - Interviewer becomes trainer / coach - Centralized monitoring with interviewer follow-up 4. Considerations: - Centralized monitoring limits the impact on interviewers - Centralized monitoring allows leverage of technology through automation 15

Household’s Use of Technology 1. Background/Issue: - Should the government supply the devices? -Should there be one device for all respondents? - Should the data collection application be web-based or local? 2. Recommendations from reviews: - Owner of device: Government vs. respondent vs. mix - Each person enters his/her own expenditures from own device vs. having a common tablet for everyone - Web-based application for all collection vs. mix of web and local app for mobile device vs. local application for tablet 16

Household’s Use of Technology (cont’d) 3. Adopted Approach: - Web-based application designed for all devices (pc; mobile) - R’s age 15+ use their own devices, use paper if necessary 4. Considerations: - Hardware ownership: Overall costs; Easier to keep up with technology if not owned; Users may not want more technology; FR safety concerns - Multiple devices maximizes self-reporting -Software: more familiarity with web-based applications; Respondent hesitancy downloading software 17

Confidentiality and Technology 1. Background/Issue: What are the constraints of the security requirements for: -Transmitting and storing data securely -Within household confidentiality 2. Recommendations from reviews: - Storage and transmitting data -Use Web-based application (no data transmission required) -Entering data on device and is encrypted/securely transmitted - Shared vs. individual diaries 18

Confidentiality and Technology (cont’d) 3. Adopted Approach: - Enter data directly to web - Use individual diaries 4. Considerations: - Minimize risk of transmission of data with a web application, investigating possibility of securely storing on device for later transmittal - Maximize within household confidentiality 19

Resources and respondent help 1. Background/Issue: - How to support respondents with differing levels of technological ability? - What can be used in lieu of the current paper diary FAQs and example pages? 2. Recommendations from reviewers: - Interactive design to assist in data entry - Allow direct questions to help desk - Monitoring to identify respondents having difficulties 20

Resources and respondent help (cont’d) 3. Adopted Approach: - FR to train respondents and leave user tips sheet - Online tutorials available - Provide system help menu - Monitor respondent activity, FR to follow-up - Provide respondent help desk - Allow paper diary for respondents uncomfortable with technology 4. Considerations: - Emphasis on user centered design, while still collecting required data - Resources limitations to support 24/7 help desk 21

Contact Information Laura Erhard Senior Economist Branch of Research and Development Consumer Expenditure Survey