1. Establishing or Enhancing a Process and Approach for Data-Driven Inquiry
Abby introduce Robb and Anne
Robb Geier, Abby Winer & Anne Lucas
Supporting Local Data Use for Program Improvement Meeting
Day 1, November 4, 2015

2. Aims of this Session
Provide an overview of the components of a high-quality process and approach for data-driven inquiry at the state and local level
Provide examples of different approaches and models for data-driven inquiry
Provide an opportunity to reflect on the strengths and areas for improvement in your state, your vision for data-driven inquiry and ideas for how to get there
Abby
During this session we aim to…

3. Data Use Theory of Action
Sustained | Systematic | Child Centered
Increased Child Outcomes
Focused
Results
Policy
Approach/
Model
Data-Driven
Actions
Practice
Settings
Robb
This is a ToA we use to describe the necessary conditions for data use (on the bottom in blue) to make data driven actions (in purple) that lead to outcomes (in green).
Quality
Capacity
Culture
Conditions for Data Use
Accurate
Timely
Relevant
Complete
Data Integration
Analysis Tools
Skills
Structures
Commitment
Beliefs
Collaboration
Leadership
© 2015 Public Consulting Group.

4. DaSy Framework for Data Use
Planning for data use
DU1. Plan for data analysis, product development, and dissemination to address the needs of the users
Analyzing and disseminating for data use
DU2. Conduct data analysis activities and implement procedures to ensure the integrity of the data
DU3. Prepare data products to promote understanding of the data and inform decision-making
DU4. Disseminate data products to users to meet their needs
Using and promoting capacity for data use
DU5. State and local staff use data to inform decisions
DU6. Support the use of data at state and local levels
Abby
Let’s revisit the DaSy framework for data use
3 Main sections/areas and 6 indicators

5. A Data Leadership Model
Create and articulate the vision for data use
Set and model expectations
Implement and uphold policies for data use in the district/region/program
Identify data to be collected
Manage data infrastructure and access
Design meaningful data displays
Select or develop models for inquiry and data use that will be used state-wide
Model the inquiry process publicly
Provide training and professional development to support district/regions, agencies, local administrators, local data teams, and providers/teachers in their use of data
Use data to identify professional development needs
Monitor the progress of the district/region toward achieving its vision for data use
Establish the lines of communication necessary to share results and best practices
Vision and Policy Management (DU1)
Data
Management (DU2, 3)
Inquiry, Analysis,
and Action (DU1, 4, 5 )
Professional Development (DU6)
Monitoring and Communication (DU1-6)
Functions of a District Data Team © Public Consulting Group
The components of this model (and the slides that follow) also come from our work with school districts and state DOEs. We may want to swap this out for the DaSY Framework. Many of the components are similar, and we may want to pare down to just focus on a few to shape the afternoon. In general, the slides that follow include a pair for each component that elaborate on the piece with questions and examples.
What matters is that people come into agreement in what the steps and the process is and what do we call those things.
For monitoring and communication, review and evaluation is the final element of each quality indicator– reviewing the effectiveness of the data displays, data use for accountability, and professional development

6. Vision and Policy Management
Setting the vision for how data are to be used.
Setting and modeling expectations.
Implementing and upholding the policies for data use across the organization.
Robb

7. Vision and Policy Management
Questions to Consider
Actions to Take
What is the current vision for data use in your organization?
How does the district/region/program specifically support, through structures and policies, the implementation of that vision?
Who needs to be part of the conversation?
Meet with local administrators, coaches, and other local leaders to communicate the vision for how data are to be used.
Review scheduling and common planning time to ensure that all local staff have the time to use data collaboratively.
Robb
Additional Questions:
Is it a powerful vision that will improve children’s progress or outcomes?
Where is improvement needed?
Additional Actions
Develop and communicate a organizational vision for data use

8. Data Management Identifying data to be collected.
Managing data infrastructure and access.
Designing meaningful data displays.
Robb

9. Data Management Questions to Consider Actions to Take
Do users have the data they need when they need it? How do we know?
What feedback channels exist to help improve our data quality and reports?
What specific steps can you take to improve how data is collected, managed, and reported?
Ensure teacher/provider access to existing data and data analysis tools.
Monitor data quality and provide documentation and ongoing training to data collection staff.
Identify reports needed for particular data use needs.
Robb
Some types of reports might include student child roster reports (by caseload, program/agency, district/region) with historical data; reports to monitor key performance indicators for principals.

10. Inquiry, Analysis, & Action
Selecting and implementing a model for data use.
Developing focusing questions.
Analyzing data to make decisions about practices/curriculum, staffing, resources, and professional development.
Robb

11. Inquiry, Analysis, & Action
Questions to Consider
Actions to Take
Does your state have a common approach to data-driven inquiry? What is it called?
Is use of the model widespread? Why or why not?
Who needs to be involved in a conversation about how your locals use inquiry?
Analyze data at the state or local level to identify essential questions across schools/programs; e.g., “What factors are associated with better child outcomes”
Coach local improvement teams to use data to set goals and develop action plans.
Robb
Other specific actions that can be taken:
Use data to assess the efficacy of the region’s coaching; e.g., are our coaches working with the right teachers/providers on the right strategies based on children’s needs?

12. Approaches to Data-Driven Inquiry to Consider
Add picture

13. Preparation – Plan to Succeed
Define purpose and the issue
Identify who needs to be involved
Timelines
Identify relevant data
Identify relevant questions
Generate hypotheses
Evidence – Dig Into Data
Analyze the data
Develop methods and materials for displaying the data
Inference – Interpret & Share With Others
Share data materials
Check support for hypotheses
Connect inferences with root causes
Action – Contribute to Success
Celebrate success
Develop & implement improvement plans
Evaluate progress
Abby
Key Components in a Process for Supporting Local Data Use – Using the Evidence Inference Action Framework (which we’ve used in ECO data workshops and trainings)
Preparation – Plan to succeed
Identify who needs to be involved = identify team members
Need someone who…
Evidence- Digging into the data
Inference – Interpreting and Sharing Data with Others
Action – Next Steps to Contribute to Success
Use group to make inference and determine the real actions – using stakeholder to finish that inference and decide what appropriate action should be taken
EIA gets repeated in cycle, but preparation occurs at beginning and gets revised occasionally (as needed, not part of regular cycle)
Depending on state infrastructure and system, locals may be ones completing process and development materials or state may be the one developing materials – completing preparation and evidence phases and then involving/sharing with locals for inference and action

14. Abby

15. Cycle of Inquiry and Action
Identify an Issue
And formulate questions to help define it
2. Understand the Issue
By analyzing data and refining questions
3. Diagnose the Cause
Through deeper analysis, observation, and best practice research
4. Plan Action
By setting measurable goals and articulating strategies
5. Take Action
While monitoring data to make adjustments
6. Evaluate
And reassess our situation 1 6 2
The Cycle of Inquiry and Action 5 3
Robb
This example comes from NJ 4
| NJ SMART

16. Cycle of Inquiry Cycle of Inquiry Focusing Questions Results
© 2008 Public Consulting Group. Used with permission.
If we ask the right questions
Results
Information
…Then we will see increased student learning and improved student achievement
…And collect and analyze data to help us answer those questions
Cycle of
Inquiry
Robb
This example comes from MA
Instructions:
Talk through the slide and the key messages.
Stress the “If … then” logic.
Give examples where appropriate. Especially needed in the “Action” phase.
Key Messages:
A disciplined application of this kind of data-driven approach builds a district and school environment that is focused on continuous improvement grounded in evidence.
Data collection and analysis is useless if appropriate action based on what the data say isn’t taken.
Action
Knowledge
…And apply that meaning to improve systems, programs, and classroom instruction
…And use current research and shared experience to make meaning of those data

17. Professional Development
Providing training and professional development to support data use by:
Districts/regions/programs
Local Administrators
Local Leadership Teams
Teachers/providers
Using data to differentiate professional development according to need.
Robb

18. Professional Development
Questions to Consider
Actions to Take
What kinds of professional development about data use have been provided?
How has this professional development been differentiated?
How is assistance deployed to support issues identified through using data?
Identify and train staff to serve as facilitators of professional learning communities in high need districts/programs.
Use child and family outcome data and information collected from coaching/supervision to target embedded professional development where it is most needed.
Robb
Other Questions to Consider
What structures and teams exist in your district/program/agency? How is professional development delivered through each of these?
Who needs to be involved in conversations about professional development in your district/program?
Other Actions to Take
Conduct targeted professional development for new providers and local administrators in local data use protocols to ensure effective use.

19. Communication & Monitoring
Communicating local-level focusing questions and findings.
Monitoring local-level use of data.
Collecting and sharing results.
Robb

20. Communication & Monitoring
Questions to Consider
Actions to Take
Establish quarterly roundtable meetings with leaders to review key performance indicators for program improvement and child and family outcomes.
Collect child and family challenges being addressed by teams to identify patterns across programs and ways to provide assistance.
How do your locals currently foster communication across districts/regions/programs about efforts to improve child and family outcomes?
How do local administrators currently monitor the effectiveness of the supports they provide?
Robb
What steps can you take to improve both of the first two questions?
How can you make use of available technology to help with these activities?
Who would need to be involved in conversations about policies, technology, time, and other resources to support data use?

21. Culture:
the set of shared attitudes, values, goals, and practices that characterizes an institution or organization
Robb
Let’s look at our definition again.
What does it take to align attitudes, values, and goals in an organization?
How do we get people to faithfully implement practices, every time, every day?

22. State Sharing
In relation to the components discussed, what is a strength in your state?
In relation to the components discussed, what is an area you are focusing on for improvement in your state?
Abby
Kyla from VA and Emily from PA will share a little about their states – an area that they believe is a strength and an area they are focusing on for improvement with regards to data leadership and data-driven inquiry

23. Tools and Resources
PCG White Paper –Theory of Action for data use culture
Critical questions for EI/ECSE
Guidance Table for Analyzing Child Outcomes for Program Improvement
Planning, Conducting & Documenting Data Analysis for Program Improvement
Abby
Just a few resources

24. State Examples
Maryland TAP-IT model:
NJ SMART Ed Data System website:
MA Dept. of Elem. & Secondary Ed. District Data Team Toolkit:
Abby
Links to state examples for those that are web-accessible

25. State Team Time 5-10 min. for personal reflection
Think about where your state and/or program is now in relation to having a process and approach for data-driven inquiry
10-15 min. for state team sharing and reflection
Then spend the next min.
Brainstorming new ideas and ways to improve process/exploration of alternative system processes
Identifying your vision and plan for where to go/what to do next
Anne
Personal and state-team reflection on where we are in relation to having a consistent process and vision for data use
Structure:
Personal reflection
State team reflection
Brainstorming vision and plan for what’s next
Share reflections with partner state
Transitioning from an effort to collect and report data to an inquiry and action effort
Came into meeting with high-level priority areas, has that shifted at all from what you’ve heard today?
As wrapping up, identify someone to share reflections and plans for where they want to go next with partner state

26. Paired State Reflection & Sharing
State Pairs: PA & NE, TX & NJ, KS & MA, VA & WI
For the next 30 min. - Each state take turns sharing reflections and plans for where you want to go next
What did you identify as strengths? Weaknesses?
What did you identify as your vision and approach for data-driven inquiry?
What do you need to do to get there?
States will then report out on each other
What did you hear was a strength in the other state?
What did you hear were key next steps?
Abby
Spend the next 30 min. taking turns sharing reflections and key next steps with another state
Each state should spend about 10 min. sharing, allowing time for the other state to respond and provide feedback

27. The contents of this presentation were developed under a grant from the U.S. Department of Education, # H373Z However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. Project Officers, Meredith Miceli and Richelle Davis.
Instructions to presenters: This slide is to be included as the last slide in your deck but you are not expected to show it to the audience. Please be sure to delete these instructions from this slide’s notes page in your presentation.