1. www.inquiry2improvement.com How a Culture of Inquiry & A Focus on Completion Can Help You Move the Needle on Student Success Dr. Rob Johnstone Texas CC Instructional Administrators Fredericksburg, TX May 31, 2015

2. www.inquiry2improvement.com National Center for Inquiry & Improvement Acknowledgements  Much of the content in this presentation in conjunction with national projects such as Completion by Design, the Aspen Prize for CC Excellence, and Bridging Research, Information & Cultures (BRIC)  Content has also been developed by and with a host of national partners, including: Community College Research Center (CCRC) Jobs for the Future JBL Associates Public Agenda The Research & Planning (RP) Group  Infographics were primarily designed by Greg Stoup

3. www.inquiry2improvement.com National Center for Inquiry & Improvement Formalistic Doublespeak… Give your 1 st reaction to the following list of words: Accountability Accreditation Assessment Continuous Improvement Data-Driven Decision Making Evaluation Institutional Effectiveness Key Performance Indicators (KPIs) Learning Outcomes Assessment Performance-based Funding Program Review Strategic planning

4. www.inquiry2improvement.com National Center for Inquiry & Improvement Recapturing the Movement… Vivid imagery from previous slide Faculty, Student Services folks & leaders have been beaten over the head with these phrases they are correct to analyze that they too often haven’t led to authentic improvement and the more formal the process, often… There is good news for our improvement efforts – you can do all of the things captured by those words in more authentic and less formalistic ways

5. www.inquiry2improvement.com National Center for Inquiry & Improvement Changing the Conversations… We have to make the conversation about things faculty, staff & administrators care about – students, their learning, and improving their outcomes and lives Not everybody will come along – but we don’t need everybody Org Change Thought: Red light / Yellow light / Green light People need to see their expertise acknowledged and integrated – and the effect of their efforts on outcomes

6. www.inquiry2improvement.com A Culture of Inquiry & Action 6 A RESOURCE for INSTITUTIONAL change April 2014

7. www.inquiry2improvement.com National Center for Inquiry & Improvement An Applied Inquiry Framework for Student Completion (CBD) 7 Stage 1 – Explore how to improve outcomes Stage 2 – Gather meaningful evidence Stage 3 – Discuss evidence broadly Stage 4 – Use evidence to inform change Stage 5 – Measure the impact of change

8. www.inquiry2improvement.com National Center for Inquiry & Improvement What is a Culture of Inquiry? 8 Institutional capacity for supporting open, honest and collaborative dialog focused on strengthening the institution and the outcomes of its students.

9. www.inquiry2improvement.com Explore how to improve student outcomes 9 Focus inquiry on designing approaches that improve student outcomes STAGE 1

10. www.inquiry2improvement.com National Center for Inquiry & Improvement How We Spend Our Time Matters…  When was the last time you sat in a standing committee meeting on your campus that used evidence to explore a key student outcome for more than 20 minutes?  What types of questions do we spend most of our organizational resources answering?

11. www.inquiry2improvement.com National Center for Inquiry & Improvement Malcolm Gladwell talks about the right question… 11 Link to full video: http://www.ted.com/talks/malcolm_gladwell_o n_spaghetti_sauce http://www.ted.com/talks/malcolm_gladwell_o n_spaghetti_sauce

12. www.inquiry2improvement.com National Center for Inquiry & Improvement The Right Question in the CC World… Placement Tests & Cut Scores 12 What was the problem we were trying to solve? I’d posit it was something like: “Can we find a short instrument that will help us assign incoming students to various levels of math & English?” Or… “How do we ensure higher levels of course success in transfer-level math & English courses? Are these the best questions? Why or why not? Note: current system of placement tests may not even be the best solution for this question: LBCC / CCRC

13. www.inquiry2improvement.com National Center for Inquiry & Improvement What if we tried to solve… 13 What placement process is the most predictive of transfer-level course success? Or…. What is the optimal curricular structure to ensure that the greatest number of students pass transfer-level math / English courses with appropriate rigor? Or….

14. www.inquiry2improvement.com National Center for Inquiry & Improvement More questions… 14 What is the optimal math curriculum that produces computational learning outcomes that most students will need in the real world? Or… Is writing about literature the optimal way to teach students the writing skills they need in their general education courses? What about in the real world?

15. www.inquiry2improvement.com National Center for Inquiry & Improvement Organizing Question of Improvement Science: What problem are you trying to solve? 15

16. www.inquiry2improvement.com National Center for Inquiry & Improvement An Example from AB Tech in Asheville, NC… 16 We have to make sure our improvement efforts to trying to solve the right problem “Right” is slippery – the problem at hand may be important, but it might detract us from a larger problem that is of much higher impact to improving student outcomes AB Tech’s “One Stop” Onboarding process Original problem they were trying to solve: “How do we address the chaotic and disconnected nature of the onboarding of our new students?

17. www.inquiry2improvement.com National Center for Inquiry & Improvement One Stop Address the Chaotic Onboarding Problem, but… 17 As the AB Tech folks reflected on their efforts, they realized they might unintentionally communicate to students that it’s possible in four hours or one day to: assess interests match interests to careers pick a program register for classes figure out financial stability issues be ready for opening day be prepared for any bump in the road that might occur

18. www.inquiry2improvement.com National Center for Inquiry & ImprovementPerhaps… 18 A better question / problem to solve might be: “How do we ensure that students get the services they need – when they need them – as they move through their educational trajectory at our college?” Note that this still covers having a more streamlined onboarding experience – but recognizes that it sits in a larger context to be addressed

19. www.inquiry2improvement.com National Center for Inquiry & Improvement To sum up the starting line stage… 19 More focus on asking the right question, and ensuring we know what problem we are trying to solve – actually less focus on the data per se Sometimes exploring the data can help you realize you’ve been asking the wrong question - The Right Pepsi vs. The Right Pepsis Traditional questions: enrollment, course success, material covered Emerging questions: improving outcomes - completion, progress, learning, labor market, equity

20. www.inquiry2improvement.com Gather meaningful evidence 20 Collect high-quality, meaningful evidence at the student support, classroom, program, and institutional levels STAGE 2

21. www.inquiry2improvement.com National Center for Inquiry & Improvement When gathering evidence, make sure you are focusing on the right data… 21

22. www.inquiry2improvement.com National Center for Inquiry & Improvement 22 Progress Example: Look for Examples from Four-Year Schools: Retention & Progress at Georgia State http://www.edtrust.org/higher_ed_practice_guidehttp://www.edtrust.org/higher_ed_practice_guide for more examples

23. www.inquiry2improvement.com National Center for Inquiry & Improvement A Mystery… The graduation rate at Georgia State University was 31% in the early 2000s Not unusual for an urban, regional 4-year state university As they evolved their culture of inquiry, they asked a range of questions designed to identify factors they could work on changing to move the needle and improve this outcome So they looked at a common metric – Fall-to- Fall retention, but didn’t stop there…

24. www.inquiry2improvement.com National Center for Inquiry & Improvement First Year to Second Year Retention, Georgia State University 24

25. www.inquiry2improvement.com National Center for Inquiry & Improvement First Year Retention & Progression Rates - Georgia State University 25

26. www.inquiry2improvement.com National Center for Inquiry & Improvement Ten Years later… GSU has implemented a wide range of targeted strategies – such as strong freshmen LCs, more structured programs of study, monitoring progress and customizing interventions / support strategies, accelerated dev ed The culture at GSU has continued to evolve into a deep culture of inquiry, action & improvement The graduation rate at Georgia State University is now 54% - a 75 percent increase in the last decade – with the same student population they started with

27. www.inquiry2improvement.com Discuss evidence broadly 27 Engage a variety of campus stakeholders in evidence-based discussions about improvements in practice STAGE 3

28. www.inquiry2improvement.com National Center for Inquiry & Improvement Key Concept: Data do not speak for themselves Time & Space Needed to Explore Data, Make Meaning & Generate Insight 28

29. www.inquiry2improvement.com National Center for Inquiry & Improvement 29 Exploring Data Example: English & Math Preparedness & Success in GE Courses

30. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Not in an English course 64% 67% 33% Success Rate of those same students in Psychology 101 English course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an English course English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

31. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 43% 51% 63% 76% Not in an Math course 64% 63% 37% Success Rate of those same students in Psychology 101 Math course taking profile for students in Psychology 101 Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in Psychology 101 for students simultaneously enrolled in an Math course Fundamentals Beginning Algebra Intermediate Algebra Transfer Level Math

32. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 31% 48% 68% 75% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Psychology 101 Speech 101 Economics 101History 101HSCI 101 54% 57% 73% 82% 27% 56% 73% 79% 48% 43% 51% 64% 60% 74% 82% 88% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B)

33. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 48% 58% 68% 76% Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade Success in five highly enrolled GE courses by English enrollment level Fitness 334 Accounting 101 Music 202Biology 250Sociology 101 37% 50% 66% 73% 31% 64% 74% 80% 40% 60% 65% 50% 59% 74% English 826 (Two Below) English 836 (One Below) English 100 (Transfer A) English 110 (Transfer B) 40% N/A

34. www.inquiry2improvement.com Use evidence to inform change 34 Implement changes in practice and policy based on analyses and discussion of college evidence STAGE 4

35. www.inquiry2improvement.com National Center for Inquiry & Improvement Use evidence to guide innovation ●In this context, research and applied inquiry are fundamentally interventionist in nature. ●We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions. ●Failure can be seen as an opportunity for learning, especially when outcomes are shared and used to inform further improvements in practice. 35

36. The process of inquiry is not a search for an absolute truth Domain of possible solutions We answer the questions that eliminate dead end solutions And what do we do when the evidence is ambiguous? Gregory M Stoup, Cañada College What to do when you reach the limits of your research and yet still face multiple choices? Trust your expertise & choose !

37. www.inquiry2improvement.com Measure the impact of change 37 Evaluate the impact of practice changes on student outcomes STAGE 5

38. www.inquiry2improvement.com Final Thoughts on a Culture of Inquiry & Improvement

39. www.inquiry2improvement.com National Center for Inquiry & Improvement Final Reflections Creating or evolving your culture of inquiry isn’t magic; there are clear steps and resulting artefacts of such a culture Don’t focus too much on the data – the questions you ask are exponentially more important Remind yourself and your team to constantly ask: “What problem are we trying to solve?” 39

40. www.inquiry2improvement.com National Center for Inquiry & Improvement What we are shooting for… Great statement of the desired end state from a CC President at the Aspen / ATD Leadership Symposium: “A wider range of people on a campus ask a better set of questions about outcomes & act on their reflections to improve them.” 40

41. www.inquiry2improvement.com Walking Through a Student’s Exploratory Process: A Website Journey

42. www.inquiry2improvement.com National Center for Inquiry & Improvement Demonstration: A Student Interested in Psychology Institution was selected at random If you think it’s different at your college / within your system, click on a couple of pages of your college’s website Yes this is the website and we have advising – but it’s still the primary source for most students – and the face of the college http://www.sc.edu/ - see appendix for screenshots http://www.sc.edu/

43. www.inquiry2improvement.com National Center for Inquiry & Improvement Post-Mortem… Can’t stress this enough – this is but one example of hundreds at 2/4-year colleges… Also important to keep in mind that for CC students – this is a huge simplification over what they need to navigate, as the 4-year student has a single goal – graduating with a BA / BS… CC students have with the options of short term certificates, long term certificates, “terminal” AA degrees, transfer + AA degree, and transfer without AA degree all as options

44. www.inquiry2improvement.com Redesign for Completion: A Guided Pathways Overview

45. Status Quo Pathways Little upfront career and college planning Paths unclear, poorly aligned with end goals Too many choices; requirements confusing Developmental dead-end Students’ progress not monitored Limited on-going feedback and support Poor alignment with hs and other feeders

46. GPS Design Principles Help students with goal-setting from the start Simplify their choices with clear roadmaps Redesign intake with goal of helping students choose and successfully enter a POS Monitor students’ progress, giving frequent feedback and support as needed Empower faculty and staff to lead the redesign process

47. www.inquiry2improvement.com National Center for Inquiry & Improvement Four Big Ideas for Redesign 1.Mapping Paths to Student End Goals 2.Helping Students Get on a Path 3.Keeping Students on a Path 4.Ensuring that Students are Learning * Based on the work of Davis Jenkins & Rob Johnstone

48. www.inquiry2improvement.com National Center for Inquiry & Improvement Clarifying Curricular Paths 1.Are our programs well-designed to prepare students to enter further education and employment in fields of importance to our region? 2.How clearly are programs mapped for our students – especially to the common transfer institutions? 3.Do students know what courses they should take in what sequence? 4.Are the courses critical for success in each program clearly identified?

49. www.inquiry2improvement.com National Center for Inquiry & Improvement Helping Students Get on a Path 1.What do we do to help new students choose a program of study? 2.Do we integrate the exploration of career options as part of their onboarding experience? 3.How well do we help students succeed in gatekeeper courses not only in Math or English but in the main program areas (e.g. A&P, ACTG 101)? 4.Do we ensure that required math courses are aligned with the students field of study? 5.What do we do for students unlikely to get into limited access programs such as nursing?

50. www.inquiry2improvement.com National Center for Inquiry & Improvement Keeping the Students on Path 1.Do students know exactly how far along they are in their programs – and what they have left to do to complete? 2.How effective are we in ensuring that students are following their plans and staying on track? 3.Are we able to identify when students are at risk of falling of their path? How effective are we at intervening when this happens? 4.Do we ensure that students can take the courses they need when they need them?

51. www.inquiry2improvement.com National Center for Inquiry & Improvement Ensuring that Students Are Learning 1.Are learning outcomes clearly defined for each of our programs, including pre-transfer? 2.Are learning outcomes aligned with requirements for success in further education and employment outcomes (by program)? 3.Do faculty assess whether students are building skills across their program? 4.Do faculty use results of learning outcomes to improve the effectiveness of instruction?

52. www.inquiry2improvement.com Top 10 Frequently Asked Questions About Redesigning Colleges to Improve Completion

53. www.inquiry2improvement.com National Center for Inquiry & Improvement Top 10 FAQs - Redesigning for Completion 1.Won’t we lose enrollment when we get rid of the swirl with increased structure? 2.Isn’t college a meritocracy, where the strong / smart succeed, and the weak / underprepared / unmotivated don’t succeed?

54. www.inquiry2improvement.com National Center for Inquiry & Improvement 3.Isn’t free choice the cornerstone of American higher education? 4.Don’t students benefit when they “find themselves” by what looks like wandering to the observer? Top 10 FAQs - Redesigning for Completion

55. www.inquiry2improvement.com National Center for Inquiry & Improvement 5.Aren’t we going to sacrifice quality when move to more structured pathways? 6.Don’t we lose a liberal arts education when we make things more structured? 7.Isn’t all of this “hand-holding” going to create graduates that can’t navigate the workplace and the “real world”? Top 10 FAQs - Redesigning for Completion

56. www.inquiry2improvement.com National Center for Inquiry & Improvement 8.How can students be expected to make career decisions when they are 18 or 19? 9.Don’t students change careers four to seven times? Given this, why would we put them on structured pathways? 10. Won’t faculty lose control over what is taught in their discipline? Top 10 FAQs - Redesigning for Completion

57. www.inquiry2improvement.com National Center for Inquiry & Improvement Find Out More 57 The National Center for Inquiry & Improvement website www.inquiry2improvement.com Dr. Rob Johnstone, Founder & President rob@inquiry2improvement.com CBD Inquiry Guides on Applied Inquiry & Nuances of Completion: http://www.inquiry2improvement.com/publications- resources http://www.inquiry2improvement.com/publications- resources