1. Building Processes for Conducting and Managing Data Collection
Erin Geary
Tribal MIECHV Annual Grantee Meeting
May 2015

2. Objectives
To introduce tools for improving timeliness and quality of data collection**
To present best practices for institutionalizing data collection in your program
To hear from you what strategies or tools are helping you improve data collection
**Tools come from TEI’s Data Collection Toolkit– Coming Soon!

3. What data are collected? Who collects the data? When are data collected? How are data entered? How is data collection monitored? How are data used?

4. Why “systematize” data collection

5. Collecting data on time

6. Considerations
“On-time” window can look different for different measures Ticklers and alerts can be helpful but…. It is helpful to talk through scenarios with staff

7. Discussion
How have you addressed data collection scheduling? What’s worked (tips, tools, etc.)?

8. Data Collection Tracking Tool
Screenshot of Tool and Timepoint entry screen

9. Data Collection Tracking Tool
Screenshot of Scheduling Template

10. Monitoring Data Quality

11. In many programs, quality happens (or doesn’t) when…
1) data are collected and 2) when data are entered

12. Monitoring data collection
Training/role play
Shadow
Collection while being observed
Independent data collection
Monitoring data collection

13. Monitoring data entry- knowing what to look for
Data entry “red flags”
Lots of missing data
Values outside the normal range
Lack of variation
Score patterns that seem out of the ordinary
Is the error part of a larger issue or a one time mistake?
Are there patterns related to certain staff?
What happens when you sort by date?
Are certain measures or certain questions being skipped?

14. Discussion
How have you monitored data quality? What issues are you seeing show up regularly? What’s worked for monitoring quality (tips, tools, etc.)?

15. Quality Assurance Form

16. Institutionalizing Data Collection

17. Best Practices- Write it down… all of it
Policies and Procedures Manuals are your friend!
EXAMPLE- Quality checks
How frequently are data checked?
By who?
Sample of data or all of it?
What will they look for?
How does the review come back to staff?

18. Data Collection Protocol Outline

19. Best Practices- Consistent messaging
Training, in-services, weekly meetings, supervision, written memos are all good platforms for communicating data collection procedures BUT…. It is important to ensure that all staff are getting the same messages

20. Sample Training Schedule

21. Best Practices- Be supportive
Home visitors play a key role in whether data collection is institutionalized in a program
Acknowledge that data collection can be challenging AND emphasize why it matters
Highlight good work
Ask for Home visitor’s input and leadership (form redesign, issues with measures, training new staff, etc.)

22. Final thoughts?
Has this generated other ideas or potential strategies? Are there tools you might take back and use?

23. Thank you for coming!

24. For more information on TEI contact: Nicole Denmark Kate Lyon
The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP WC.
TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings.
TEI1 was awarded to MDRC; James Bell Associates, Inc.; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health.
For more information on TEI contact:
Nicole Denmark Kate Lyon
Federal Project Officer Project Director
Office of Planning Research and Evaluation James Bell Associates, Inc.
The Tribal Evaluation Institute is funded by the Office of Planning, research and Evaluation within the Administration for Children and Families. TEI was awarded to James Bell Associates in partnership with the University of Colorado’s Centers for American Indian and Alaska Native Health and Michigan Public Health Institute. For more information, contact the individuals on this slide.