Dr. Rob Johnstone Ohio Board of Regents Charting the Course for Student Completion Columbus, OH December 3rd, 2013 How a Culture of Inquiry Can Help Catalyze Completion Efforts

National Center for Inquiry & Improvement Acknowledgements  Much of the content in this presentation in conjunction with national projects such as Completion by Design, the Aspen Prize for CC Excellence, and Bridging Research, Information & Cultures (BRIC)  Content has also been developed by and with a host of national partners, including: Community College Research Center (CCRC) Jobs for the Future JBL Associates Public Agenda The Research & Planning (RP) Group WestEd  Infographics were primarily designed by Greg Stoup An Applied Inquiry Framework for Student Completion – April 2013

The Student Experience An Applied Inquiry Framework for Student Completion April 2013

National Center for Inquiry & Improvement

A Culture of Inquiry & Action 5 A RESOURCE for INSTITUTIONAL change April 2013

National Center for Inquiry & Improvement What is a Culture of Inquiry? 6 Institutional capacity for supporting open, honest and collaborative dialog focused on strengthening the institution and the outcomes of its students.

National Center for Inquiry & Improvement Culture of Inquiry: Features ●Widespread access to user-friendly information on student outcomes ●Encouraging more people to ask a wider collection of questions and use their evidence and conclusions to enhance decision making ●Reflective and dynamic discussions across constituency groups 7

National Center for Inquiry & Improvement Culture of Inquiry: More Features ●Continuous feedback so adjustments can be made along the way and processes can be adapted ●A sense of ownership over improving student outcomes – not blaming the student ●Using insight generated from inquiry to act at varying levels of the institution to create the conditions to improve outcomes 8

Applied Inquiry Framework for Student Completion Developed for Completion by Design 9 A Resource for Institutional Change

National Center for Inquiry & Improvement An Applied Inquiry Framework for Student Completion 10 Stage 1 – Explore how to improve outcomes Stage 2 – Gather meaningful evidence Stage 3 – Discuss evidence broadly Stage 4 – Use evidence to inform change Stage 5 – Measure the impact of change

National Center for Inquiry & Improvement Explore how to improve student outcomes Focus inquiry on designing approaches that improve student outcomes 11 STAGE 1

National Center for Inquiry & Improvement Key Questions  When was the last time you sat in a standing committee meeting on your campus that used evidence to explore a key student success outcome for more than 20 minutes?  What types of questions do we spend most of our organizational resources answering?

National Center for Inquiry & Improvement Gather meaningful evidence Collect high-quality, meaningful evidence at the student support, classroom, program, and institutional levels 13 STAGE 2

National Center for Inquiry & Improvement When gathering evidence, make sure you are focusing on the right data… 14

National Center for Inquiry & Improvement Persistence Rate Success Rate year trend for California CC course success & persistence rates 15 What does that tell us about the usefulness of these metrics in setting institutional strategies?

National Center for Inquiry & Improvement Three Studies to Energize a Campus Conversation About Student Success Grades & Success in Sequenced Courses Cohort Tracking in Developmental Education Levels of Preparedness in GE Courses – success of simultaneously enrolled students in Math / English courses

National Center for Inquiry & Improvement 17 Study 1: Grades in Sequenced Courses Note: Data tracked from to

National Center for Inquiry & Improvement Grades in Sequenced Courses Backdrop Course sequences analyzed in Mathematics, English, Biology, Chemistry, Physics & Economics Relationship between grade in 1 st course in sequence and success in 2 nd course in sequence was examined. Enrollment data tracked from to Cohort Ns ranged from 1,890 to 126,367

National Center for Inquiry & Improvement Segmentation of Students Taking Intermediate Algebra 19

National Center for Inquiry & Improvement Table for Intermediate Algebra Status in Math 310Count% of Total N Success Rate in Math 310 A in Math 3085,65910%81% B in Math 3087,72613%60% C in Math 3087,23412%37% Unclear Status in Math 308*1,6753%16% Math 310 Repeater / Non Success14,56825%41% Math 310 Attempt was First Math Course16,81229%57% Taking Math 308 / 310 in Same Semester4,5058%68% Other4871%44% Total58,666100%53% 20

A B C The grade received by students in Math 306 Intermediate Algebra Pre-Algebra Introductory Algebra 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math % 51% 76% Developmental Education Math Course Sequence Math 310Math 306Math 308 Introductory Algebra Math 308 B A Transfer Level Those placing directly into Math % Math 1300s

A B C The grade received by students in Math 308 Pre-Algebra Introductory Algebra Intermediate Algebra 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math % 60% 81% Developmental Education Math Course Sequence Math 306Math 308Math 310 Intermediate Algebra Math 310 B A Transfer- Level Those placing directly into Math % Math 1300s

A B C The grade received by students in Math 310 Intermediate Algebra Transfer- Level 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math 1300s 44% 63% 81% Intermediate Algebra – Transfer-Level Mathematics MATH 310MATH 1300s Transfer- Level MATH 1300s B A Those placing directly into Math 1300s 66%

A B C The grade received by students in Math 2413 Calculus ICalculus II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Math % 78% 90% Calculus Course Sequence MATH 2413MATH 2414 Calculus II MATH 2414 B A Those placing directly into Math %

A B C The grade received by students in English 306 Two Levels below Transfer One Level below Transfer Transfer Level English 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English % 66% 77% Developmental Education Writing Course Sequence Those placing directly into English 307 ENGL 306ENGL 307ENGL 1301 One Level below Transfer B A 66%

A B C The grade received by students in English 307 One Level below Transfer Two Levels below Transfer 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English % 61% 74% Developmental Education Writing Course Sequence Those placing directly into English 1301 ENGL 307ENGL 306 Transfer Level English ENGL 1301 B A 71%

A B C The grade received by students in English 1301 Composition I Composition II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in English % 77% 88% College English Course Sequence ENGL 1301ENGL 1302 Composition II ENGL 1302 B A Those placing directly into English %

A B C The grade received by students in Biology 1406 Biology IBiology II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Biology % 76% 90% College Biology Course Sequence BIO 1406BIO 1407 Biology II BIO 1407 B A Those placing directly into Biology %

A B C The grade received by students in Biology 2401 AnatomyPhysiology 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Biology % 72% 88% College Biology Course Sequence BIO 2401BIO 2402 Physiology BIO 2402 B A Those placing directly into Biology %

A B C The grade received by students in Chemistry 1411 General Chemistry I General Chemistry II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Chemistry % 70% 86% College Chemistry Course Sequence CHEM 1411CHEM 1412 General Chemistry II CHEM 1412 B A Those placing directly into Chemistry %

A B C The grade received by students in Physics 1401 General Physics I General Physics II 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Physics % 94% University Physics Course Sequence PHYS 1401PHYS 1402 General Physics II PHYS 1402 B A Those placing directly into Physics % 69%

A B C The grade received by students in Economics 2301 Macro Economics Micro Economics 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Success Rate of those same students in Economics % 81% 89% Economics Course Sequence ECON 2301ECON 2302 Micro Economics ECON 2302 B A Those placing directly into Economics %

National Center for Inquiry & Improvement 33 A Direct Look at Meaningful Analyses from Four-Year Schools

National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts Based on analyses from: Florida State University Georgia State University San Diego State University University of North Carolina – Greensboro University of Texas University of Southern California University of Wisconsin Eau Claire Virginia Commonwealth University 34

National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts 1.Targeting at-risk populations (UNCG) 2.Identifying At-Risk Students Using Historical Data (GSU, FSU) 3.Identifying At-Risk Students Using Milestones (FSU, GSU) 4.Tracking beyond Freshmen to Sophomore Retention Rates (FSU) 5.Tracking Second Year Students with Sophomore Standing (GSU) 35

National Center for Inquiry & Improvement Simple and Effective Analyses to Inform Student Progress and Completion Efforts 6.Identifying Correlates of Success (VCU, UWEC, SDSU) 7.Identifying Correlates of Non-Success (USC) 8.Predictive Model for Retention (VCU, GSU) 9.Impact of Course Withdrawals (VCU) 10. Additional Analyses (USC, FSU, GSU, UWEC) 36

National Center for Inquiry & Improvement First Year to Second Year Retention, Georgia State University 3/30/10 37

National Center for Inquiry & Improvement First Year Retention & Progression Rates - Georgia State University 3/30/10 38

National Center for Inquiry & Improvement Discuss evidence broadly Engage a variety of campus stakeholders in evidence-based discussions about improvements in practice 39 STAGE 3

National Center for Inquiry & Improvement Data do not speak for themselves. 40

National Center for Inquiry & Improvement The vital role of exploration The vital role of exploration ●In order to make data useful, ample time and space are needed to discuss and analyze the information and connect it back to the original research question. ●Answers are not always immediately apparent, so skilled facilitation may be needed to dig out the deeper meaning. ●Multiple perspectives and types of information are often needed to make sense of individual data points. 41 !

National Center for Inquiry & Improvement 42 Study 3: English & Math Preparedness & Success in GE Courses –

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Not in an English course 64% 67% 33% Success Rate of those same students in Psychology 101 English course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an English course English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 43% 51% 63% 76% Not in an Math course 64% 63% 37% Success Rate of those same students in Psychology 101 Math course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an Math course Fundamentals Beginning Algebra Intermediate Algebra Transfer Level Math

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Psychology 101 Speech 101 Economics 101History 101HSCI % 57% 73% 82% 27% 56% 73% 79% 48% 43% 51% 64% 60% 74% 82% 88% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 48% 58% 68% 76% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Fitness 334 Accounting 101 Music 202Biology 250Sociology % 50% 66% 73% 31% 64% 74% 80% 40% 60% 65% 50% 59% 74% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B) 40% N/A

National Center for Inquiry & Improvement Use evidence to inform change Implement changes in practice and policy based on analyses and discussion of college evidence 47 STAGE 4

National Center for Inquiry & Improvement Use evidence to guide innovation ●In this context, research and applied inquiry are fundamentally interventionist in nature. ●We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions. ●Failure can be seen as an opportunity for learning, especially when outcomes are shared and used to inform further improvements in practice. 48

The process of inquiry is not a search for an absolute truth Domain of possible solutions We answer the questions that eliminate dead end solutions And what do we do when the evidence is ambiguous? Gregory M Stoup, Cañada College What to do when you reach the limits of your research and yet still face multiple choices? Trust your expertise & choose !

National Center for Inquiry & Improvement Measure the impact of change Evaluate the impact of practice changes on student performance 50 STAGE 5

Exploring the Preventing Loss, Creating Momentum Framework

National Center for Inquiry & Improvement Loss & Momentum Framework CONNECTION Interest to Application CONNECTION Interest to Application ENTRY Enrollment to Completion of Gatekeeper Courses PROGRESS Entry into Course of Study to 75% Requirements Completed COMPLETION Complete Course of Study to Credential with Labor Market Value POLICIES PRACTICES PROGRAMS PROCESSES

National Center for Inquiry & Improvement PROGRESS ENTRY COMPLETION CONNECTION Students never apply to college Students delay entry into college College counseling patterns that lead to: - under enrollment - little program- specific guidance - missed financial aid opportunities Unstructured programs / too many choices Extended onramps delay entry to programs of study Students fail to enroll/pass Gatekeeper courses Poor work- school balance Part-time enrollment forcing long completion times Progress not monitored / feedback given Life events / “Stop out or drop out” Transfer without credential Students accumulate credits (& debt) not aligned with completion Never complete college level math Credential doesn’t support needed wage & aren’t stackable Completion by Design Framing Model Some Known Loss Points Poor academic preparation

National Center for Inquiry & Improvement PROGRESS ENTRY COMPLETION CONNECTION foster college- going norms in High School expand awareness of college programs and requirements dual enrollment & AP credit Completion by Design Framing Model Momentum Strategies take placement test in high school educational planning in high school aggressive financial aid support accelerate entry to POS shorter, faster, cheaper course design effective academic catch-up programs mandatory intrusive advising focused on programs of study programs to incentivize optimal attendance student progress to completion monitored & feedback provided accelerated competency- based programs emergency aid for students remove barriers to graduation Learn & Earn and Career Pathway programs incentives to transfer with credentials mandatory intrusive advising toward certificates degrees & transfer First Time Student Successful Completion

Final Thoughts An Applied Inquiry Framework for Student Completion – September 2013

National Center for Inquiry & Improvement Culture of Inquiry: Why All the Fuss? ●Because this ongoing work is challenging but necessary! ●Work needs to marry insight & evidence ●Problems are large and recurring ●No silver bullet / evident answers ●Multiple solutions likely needed ●Progress is not linear ●Requires cross constituency interaction 56

National Center for Inquiry & Improvement Next Steps What are some “quick win” opportunities for demonstrating the value of a culture of inquiry to practitioners and stakeholders? What are some of the challenges to developing or further evolving a culture of inquiry? Completion by Design? Completion by Accident?

National Center for Inquiry & Improvement Find Out More 58 The National Center for Inquiry & Improvement website Dr. Rob Johnstone, Founder & President CBD Inquiry Guides: resources resources