1. www.inquiry2improvement.com Dr. Rob Johnstone Ohio Board of Regents Charting the Course for Student Completion Columbus, OH December 3rd, 2013 How a Culture of Inquiry Can Help Catalyze Completion Efforts

2. www.inquiry2improvement.com National Center for Inquiry & Improvement Acknowledgements  Much of the content in this presentation in conjunction with national projects such as Completion by Design, the Aspen Prize for CC Excellence, and Bridging Research, Information & Cultures (BRIC)  Content has also been developed by and with a host of national partners, including: Community College Research Center (CCRC) Jobs for the Future JBL Associates Public Agenda The Research & Planning (RP) Group WestEd  Infographics were primarily designed by Greg Stoup An Applied Inquiry Framework for Student Completion – April 2013

3. www.inquiry2improvement.com The Student Experience An Applied Inquiry Framework for Student Completion April 2013

4. www.inquiry2improvement.com National Center for Inquiry & Improvement

5. www.inquiry2improvement.com A Culture of Inquiry & Action 5 A RESOURCE for INSTITUTIONAL change April 2013

6. www.inquiry2improvement.com National Center for Inquiry & Improvement What is a Culture of Inquiry? 6 Institutional capacity for supporting open, honest and collaborative dialog focused on strengthening the institution and the outcomes of its students.

7. www.inquiry2improvement.com National Center for Inquiry & Improvement Culture of Inquiry: Features ●Widespread access to user-friendly information on student outcomes ●Encouraging more people to ask a wider collection of questions and use their evidence and conclusions to enhance decision making ●Reflective and dynamic discussions across constituency groups 7

8. www.inquiry2improvement.com National Center for Inquiry & Improvement Culture of Inquiry: More Features ●Continuous feedback so adjustments can be made along the way and processes can be adapted ●A sense of ownership over improving student outcomes – not blaming the student ●Using insight generated from inquiry to act at varying levels of the institution to create the conditions to improve outcomes 8

9. www.inquiry2improvement.com Applied Inquiry Framework for Student Completion Developed for Completion by Design 9 A Resource for Institutional Change

10. www.inquiry2improvement.com National Center for Inquiry & Improvement An Applied Inquiry Framework for Student Completion 10 Stage 1 – Explore how to improve outcomes Stage 2 – Gather meaningful evidence Stage 3 – Discuss evidence broadly Stage 4 – Use evidence to inform change Stage 5 – Measure the impact of change

11. www.inquiry2improvement.com National Center for Inquiry & Improvement Explore how to improve student outcomes Focus inquiry on designing approaches that improve student outcomes 11 STAGE 1

12. www.inquiry2improvement.com National Center for Inquiry & Improvement Key Questions  When was the last time you sat in a standing committee meeting on your campus that used evidence to explore a key student success outcome for more than 20 minutes?  What types of questions do we spend most of our organizational resources answering?

13. www.inquiry2improvement.com National Center for Inquiry & Improvement Gather meaningful evidence Collect high-quality, meaningful evidence at the student support, classroom, program, and institutional levels 13 STAGE 2

14. www.inquiry2improvement.com National Center for Inquiry & Improvement When gathering evidence, make sure you are focusing on the right data… 14

15. www.inquiry2improvement.com National Center for Inquiry & Improvement Persistence Rate Success Rate 1989 2008 20 year trend for California CC course success & persistence rates 15 What does that tell us about the usefulness of these metrics in setting institutional strategies?

16. www.inquiry2improvement.com National Center for Inquiry & Improvement Three Studies to Energize a Campus Conversation About Student Success Grades & Success in Sequenced Courses Cohort Tracking in Developmental Education Levels of Preparedness in GE Courses – success of simultaneously enrolled students in Math / English courses

17. www.inquiry2improvement.com National Center for Inquiry & Improvement 17 Study 1: Grades in Sequenced Courses Note: Data tracked from 2006-07 to 2011-12

18. www.inquiry2improvement.com National Center for Inquiry & Improvement Grades in Sequenced Courses Backdrop Course sequences analyzed in Mathematics, English, Biology, Chemistry, Physics & Economics Relationship between grade in 1 st course in sequence and success in 2 nd course in sequence was examined. Enrollment data tracked from 2006-07 to 2011-12 Cohort Ns ranged from 1,890 to 126,367

19. www.inquiry2improvement.com National Center for Inquiry & Improvement Segmentation of Students Taking Intermediate Algebra 19

20. www.inquiry2improvement.com National Center for Inquiry & Improvement Table for Intermediate Algebra Status in Math 310Count% of Total N Success Rate in Math 310 A in Math 3085,65910%81% B in Math 3087,72613%60% C in Math 3087,23412%37% Unclear Status in Math 308*1,6753%16% Math 310 Repeater / Non Success14,56825%41% Math 310 Attempt was First Math Course16,81229%57% Taking Math 308 / 310 in Same Semester4,5058%68% Other4871%44% Total58,666100%53% 20

21. A B C The grade received by students in Math 306 Intermediate Algebra Pre-Algebra Introductory Algebra 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math 308 27% 51% 76% Developmental Education Math Course Sequence Math 310Math 306Math 308 Introductory Algebra Math 308 B A Transfer Level Those placing directly into Math 308 60% Math 1300s

22. A B C The grade received by students in Math 308 Pre-Algebra Introductory Algebra Intermediate Algebra 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math 310 37% 60% 81% Developmental Education Math Course Sequence Math 306Math 308Math 310 Intermediate Algebra Math 310 B A Transfer- Level Those placing directly into Math 310 57% Math 1300s

23. A B C The grade received by students in Math 310 Intermediate Algebra Transfer- Level 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math 1300s 44% 63% 81% Intermediate Algebra – Transfer-Level Mathematics MATH 310MATH 1300s Transfer- Level MATH 1300s B A Those placing directly into Math 1300s 66%

24. A B C The grade received by students in Math 2413 Calculus ICalculus II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math 2414 50% 78% 90% Calculus Course Sequence MATH 2413MATH 2414 Calculus II MATH 2414 B A Those placing directly into Math 2414 72%

25. A B C The grade received by students in English 306 Two Levels below Transfer One Level below Transfer Transfer Level English 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English 307 49% 66% 77% Developmental Education Writing Course Sequence Those placing directly into English 307 ENGL 306ENGL 307ENGL 1301 One Level below Transfer B A 66%

26. A B C The grade received by students in English 307 One Level below Transfer Two Levels below Transfer 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English 1301 46% 61% 74% Developmental Education Writing Course Sequence Those placing directly into English 1301 ENGL 307ENGL 306 Transfer Level English ENGL 1301 B A 71%

27. A B C The grade received by students in English 1301 Composition I Composition II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English 1302 58% 77% 88% College English Course Sequence ENGL 1301ENGL 1302 Composition II ENGL 1302 B A Those placing directly into English 1302 68%

28. A B C The grade received by students in Biology 1406 Biology IBiology II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Biology 1407 60% 76% 90% College Biology Course Sequence BIO 1406BIO 1407 Biology II BIO 1407 B A Those placing directly into Biology 1407 70%

29. A B C The grade received by students in Biology 2401 AnatomyPhysiology 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Biology 2402 50% 72% 88% College Biology Course Sequence BIO 2401BIO 2402 Physiology BIO 2402 B A Those placing directly into Biology 2402 67%

30. A B C The grade received by students in Chemistry 1411 General Chemistry I General Chemistry II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Chemistry 1412 46% 70% 86% College Chemistry Course Sequence CHEM 1411CHEM 1412 General Chemistry II CHEM 1412 B A Those placing directly into Chemistry 1412 66%

31. A B C The grade received by students in Physics 1401 General Physics I General Physics II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Physics 1402 87% 94% University Physics Course Sequence PHYS 1401PHYS 1402 General Physics II PHYS 1402 B A Those placing directly into Physics 1402 78% 69%

32. A B C The grade received by students in Economics 2301 Macro Economics Micro Economics 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Economics 2302 74% 81% 89% Economics Course Sequence ECON 2301ECON 2302 Micro Economics ECON 2302 B A Those placing directly into Economics 2302 71%

33. www.inquiry2improvement.com National Center for Inquiry & Improvement 33 A Direct Look at Meaningful Analyses from Four-Year Schools

34. www.inquiry2improvement.com National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts Based on analyses from: Florida State University Georgia State University San Diego State University University of North Carolina – Greensboro University of Texas University of Southern California University of Wisconsin Eau Claire Virginia Commonwealth University 34

35. www.inquiry2improvement.com National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts 1.Targeting at-risk populations (UNCG) 2.Identifying At-Risk Students Using Historical Data (GSU, FSU) 3.Identifying At-Risk Students Using Milestones (FSU, GSU) 4.Tracking beyond Freshmen to Sophomore Retention Rates (FSU) 5.Tracking Second Year Students with Sophomore Standing (GSU) 35

36. www.inquiry2improvement.com National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts 6.Identifying Correlates of Success (VCU, UWEC, SDSU) 7.Identifying Correlates of Non-Success (USC) 8.Predictive Model for Retention (VCU, GSU) 9.Impact of Course Withdrawals (VCU) 10. Additional Analyses (USC, FSU, GSU, UWEC) 36

37. www.inquiry2improvement.com National Center for Inquiry & Improvement First Year to Second Year Retention, Georgia State University 3/30/10 37

38. www.inquiry2improvement.com National Center for Inquiry & Improvement First Year Retention & Progression Rates - Georgia State University 3/30/10 38

39. www.inquiry2improvement.com National Center for Inquiry & Improvement Discuss evidence broadly Engage a variety of campus stakeholders in evidence-based discussions about improvements in practice 39 STAGE 3

40. www.inquiry2improvement.com National Center for Inquiry & Improvement Data do not speak for themselves. 40

41. www.inquiry2improvement.com National Center for Inquiry & Improvement The vital role of exploration The vital role of exploration ●In order to make data useful, ample time and space are needed to discuss and analyze the information and connect it back to the original research question. ●Answers are not always immediately apparent, so skilled facilitation may be needed to dig out the deeper meaning. ●Multiple perspectives and types of information are often needed to make sense of individual data points. 41 !

42. www.inquiry2improvement.com National Center for Inquiry & Improvement 42 Study 3: English & Math Preparedness & Success in GE Courses –

43. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Not in an English course 64% 67% 33% Success Rate of those same students in Psychology 101 English course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an English course English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

44. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 43% 51% 63% 76% Not in an Math course 64% 63% 37% Success Rate of those same students in Psychology 101 Math course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an Math course Fundamentals Beginning Algebra Intermediate Algebra Transfer Level Math

45. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Psychology 101 Speech 101 Economics 101History 101HSCI 101 54% 57% 73% 82% 27% 56% 73% 79% 48% 43% 51% 64% 60% 74% 82% 88% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

46. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 48% 58% 68% 76% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Fitness 334 Accounting 101 Music 202Biology 250Sociology 101 37% 50% 66% 73% 31% 64% 74% 80% 40% 60% 65% 50% 59% 74% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B) 40% N/A

47. www.inquiry2improvement.com National Center for Inquiry & Improvement Use evidence to inform change Implement changes in practice and policy based on analyses and discussion of college evidence 47 STAGE 4

48. www.inquiry2improvement.com National Center for Inquiry & Improvement Use evidence to guide innovation ●In this context, research and applied inquiry are fundamentally interventionist in nature. ●We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions. ●Failure can be seen as an opportunity for learning, especially when outcomes are shared and used to inform further improvements in practice. 48

49. The process of inquiry is not a search for an absolute truth Domain of possible solutions We answer the questions that eliminate dead end solutions And what do we do when the evidence is ambiguous? Gregory M Stoup, Cañada College What to do when you reach the limits of your research and yet still face multiple choices? Trust your expertise & choose !

50. www.inquiry2improvement.com National Center for Inquiry & Improvement Measure the impact of change Evaluate the impact of practice changes on student performance 50 STAGE 5

51. www.inquiry2improvement.com Exploring the Preventing Loss, Creating Momentum Framework

52. www.inquiry2improvement.com National Center for Inquiry & Improvement Loss & Momentum Framework CONNECTION Interest to Application CONNECTION Interest to Application ENTRY Enrollment to Completion of Gatekeeper Courses PROGRESS Entry into Course of Study to 75% Requirements Completed COMPLETION Complete Course of Study to Credential with Labor Market Value POLICIES PRACTICES PROGRAMS PROCESSES

53. www.inquiry2improvement.com National Center for Inquiry & Improvement PROGRESS ENTRY COMPLETION CONNECTION Students never apply to college Students delay entry into college College counseling patterns that lead to: - under enrollment - little program- specific guidance - missed financial aid opportunities Unstructured programs / too many choices Extended onramps delay entry to programs of study Students fail to enroll/pass Gatekeeper courses Poor work- school balance Part-time enrollment forcing long completion times Progress not monitored / feedback given Life events / “Stop out or drop out” Transfer without credential Students accumulate credits (& debt) not aligned with completion Never complete college level math Credential doesn’t support needed wage & aren’t stackable Completion by Design Framing Model Some Known Loss Points Poor academic preparation

54. www.inquiry2improvement.com National Center for Inquiry & Improvement PROGRESS ENTRY COMPLETION CONNECTION foster college- going norms in High School expand awareness of college programs and requirements dual enrollment & AP credit Completion by Design Framing Model Momentum Strategies take placement test in high school educational planning in high school aggressive financial aid support accelerate entry to POS shorter, faster, cheaper course design effective academic catch-up programs mandatory intrusive advising focused on programs of study programs to incentivize optimal attendance student progress to completion monitored & feedback provided accelerated competency- based programs emergency aid for students remove barriers to graduation Learn & Earn and Career Pathway programs incentives to transfer with credentials mandatory intrusive advising toward certificates degrees & transfer First Time Student Successful Completion

55. www.inquiry2improvement.com Final Thoughts An Applied Inquiry Framework for Student Completion – September 2013

56. www.inquiry2improvement.com National Center for Inquiry & Improvement Culture of Inquiry: Why All the Fuss? ●Because this ongoing work is challenging but necessary! ●Work needs to marry insight & evidence ●Problems are large and recurring ●No silver bullet / evident answers ●Multiple solutions likely needed ●Progress is not linear ●Requires cross constituency interaction 56

57. www.inquiry2improvement.com National Center for Inquiry & Improvement Next Steps What are some “quick win” opportunities for demonstrating the value of a culture of inquiry to practitioners and stakeholders? What are some of the challenges to developing or further evolving a culture of inquiry? Completion by Design? Completion by Accident?

58. www.inquiry2improvement.com National Center for Inquiry & Improvement Find Out More 58 The National Center for Inquiry & Improvement website www.inquiry2improvement.com Dr. Rob Johnstone, Founder & President rob@inquiry2improvement.com CBD Inquiry Guides: http://www.inquiry2improvement.com/publications- resources http://www.inquiry2improvement.com/publications- resources