1. Eastern Maine Community College Achieving the Dream Kickoff August 25, 2011 Using Evidence to Improve Student Success David Buchdahl Ph.D, ATD Data Coach

2. Success is what counts. 2 A national initiative to increase student success ATD’s model for increasing student success is based on four key principles: 1.Committed leadership 2.Use of evidence to improve student success 3.Broad engagement 4.Systematic institutional improvement 5.Equity

3. Success is what counts. 3 With ATD, your college builds a culture of evidence and inquiry using three kinds of data: 1.Data that provide a comprehensive picture of student success -- “Indicator data” 2.Qualitative data and other kinds of evidence that help explain levels and performance gaps in the indicator data -- “Why data” 3.Data that measure the results of improvement strategies that have been implemented to improve student success -- “Performance measure data”

4. Success is what counts. 4 Building a culture of evidence and inquiry: Indicator Data To track changing patterns in the indicators of student success “Why?” Data To determine why levels and gaps exist Select and implement improvement strategies Performance Measure Data To evaluate improvement strategies

5. Success is what counts. 5 Full-time/part-time Race and ethnicity Gender Age Income level 1 st -generation Developmental vs. college-ready What does indicator data reveal about student success and performance gaps? Completion of developmental courses Completion of “gateway” courses ( e.g. English Comp, College-level math ) Complete courses with a grade C or better Persistence from first semester to second and first to second year Attain a degree or credential Indicator data also reveals gaps in success among various groups: Indicator Data ATD’s 5 indicators of success:

6. Success is what counts. 6 Student focus groups CCSSE, Noel-Levitz survey data Faculty and administrative perspectives Classroom research Literature search Other? “Why” Data Why do we have current levels of student success, and why do gaps exist?

7. Success is what counts. 7 Performance Measure Data HOW MUCH DID WE DO?HOW WELL DID WE DO IT? WHAT DIFFERENCE DID IT MAKE? Short-tem differencesMedium-Term DifferencesLong-term Differences Evaluating improvement strategies A simple evaluation matrix*: *from Trying Hard is Not Good Enough, Mark Friedman (available from Amazon as a Kindle book)

8. Success is what counts. 8 Performance Measure Data HOW MUCH DID WE DO?HOW WELL DID WE DO IT? How many students were affected by the new practice or process? What % of the target group did we serve? Can this practice be scaled up to impact the desired number of students? WHAT DIFFERENCE DID IT MAKE? Short-tem differencesMedium-Term DifferencesLong-term Differences Evaluating improvement strategies A simple evaluation matrix:

9. Success is what counts. 9 Performance Measure Data HOW MUCH DID WE DO?HOW WELL DID WE DO IT? How did students and faculty rate the quality of the new practice? Are we satisfied with these results ? WHAT DIFFERENCE DID IT MAKE? Short-tem differencesMedium-Term DifferencesLong-term Differences Evaluating improvement strategies A simple evaluation matrix:

10. Success is what counts. 10 Performance Measure Data HOW MUCH DID WE DO?HOW WELL DID WE DO IT? WHAT DIFFERENCE DID IT MAKE? Short-tem differences Has the strategy improved course completion rates? Medium-Term Differences Has the strategy improved first-semester or first-year return rates? Long-term Differences Has the strategy improved completion rates for targeted student groups? Evaluating improvement strategies A simple evaluation matrix:

11. Success is what counts. 11 Performance Measure Data HOW MUCH DID WE DO?HOW WELL DID WE DO IT? How many students were affected by the new practice or process? What % of the target group did we serve? Can this practice be scaled up to impact the desired number of students? How did students and faculty rate the quality of the new practice? Are we satisfied with these results ? WHAT DIFFERENCE DID IT MAKE? Short-tem differences Has the strategy improved course completion rates? Medium-Term Differences Has the strategy improved first-semester or first-year return rates? Long-term Differences Has the strategy improved completion rates for targeted student groups? Evaluating improvement strategies A simple evaluation matrix:

12. Success is what counts. 12 From building a culture of evidence and inquiry to practice Indicator Data To track changing patterns in the indicators of student success “Why?” Data To determine why levels and gaps exist Select and implement improvement strategies Performance Measure Data To evaluate improvement strategies

13. Success is what counts. 13 Indicator data create a need for “courageous conversations”  Are we content with the rates of student success and the gaps that we see?  In what ways are we (faculty and staff) responsible for these rates and gaps?  If we don’t change our behavior, can we expect different results?  Will we hold ourselves accountable for achieving different results?  And what are some specific features of indicator data that will require courageous conversations at EMCC?

14. Success is what counts. 14 Practicing a culture of evidence and inquiry Indicator Data Data-driven, courageous conversations “Why?” Data To determine why levels and gaps exist Select and implement improvement strategies Performance Measure Data To evaluate improvement strategies Data-driven, courageous conversations

15. Success is what counts. 15 Practicing a culture of evidence and inquiry Indicator Data Data-driven, courageous conversations “Why?” Data To determine why levels and gaps exist Data-informed decision making Select and implement improvement strategies Performance Measure Data

16. Success is what counts. 16 Practicing a culture of evidence and inquiry Indicator Data Data-driven, courageous conversations “Why?” Data Data-informed decision making Select and implement improvement strategies Performance Measure Data Data rich evaluations

17. Success is what counts. 17

18. Eastern Maine Community College Achieving the Dream Kickoff August 25, 2011 Using Evidence to Improve Student Success David Buchdahl Ph.D, ATD Data Coach