Presentation is loading. Please wait.

Presentation is loading. Please wait.

East Bay Career Pathways Regional Convening #2 December 4, 2014

Similar presentations

Presentation on theme: "East Bay Career Pathways Regional Convening #2 December 4, 2014"— Presentation transcript:

1 East Bay Career Pathways Regional Convening #2 December 4, 2014
Let Icarus Fly: Unleashing student achievement through multiple measures assessment John J. Hetts, Ph.D. Senior Director of Data Science Educational Results Partnership East Bay Career Pathways Regional Convening #2 December 4, 2014 John Hetts, Senior Data Scientist Ken Sorey, VP ERP Short Bios go here.

2 Overview Standardized tests systematically underestimate student capacity Students of color First generation college students Lower SES Women Multiple measures (esp. GPA) fairer and far more accurate predictor of college performance/graduation Increase grad rates, decrease time to completion Very low cost, exceptionally high ROI To students To colleges To state (optional intro) 0) Standardized assessments systematically underestimate student’s capacity to do college-level work, particularly students of color, first-generation college students, students with lower SES, and women, and do so so both at four year colleges and community colleges. 1) Using multiple measures, particularly high school GPA, in assessment, admissions, and merit-based aid, provides a fairer, far more accurate assessment of students’ ability both to perform well in college and to graduate. 2) As a result, using multiple measures will increase graduation rates, decrease time to graduation, and meaningfully help close equity gaps and can do so at next to no cost. 3) In fact, using multiple measures will save public education systems and taxpayers tens of millions of dollars annually in California alone in unnecessary costs of reteaching successfully mastered skills and millions of dollars annually in unnecessary assessments. (for longer elevators) 4) Even more importantly, using multiple measures will save California students billions of dollars in lost salary/other opportunity costs that they currently lose to the time necessary to retake the same courses they successfully completed in high school and will meaningfully improve a wide range of their long-term life outcomes as well as their productive contributions to the economy and their communities.

3 Daedalus and Icarus Daedalus crafted the labyrinth of inescapable complexity for King Minos To escape from Minos, Daedalus built wings of feather and wax for his son Icarus and himself Don’t fly too high, lest sun melt the wax and you plummet to your doom Dangers of innovation/invention, hubris, Importance of knowing your limits, listening to your wiser elders But most of us forget the rest of that story…

4 Student transition to college
Community colleges rely nearly entirely on standardized assessment Most CC students placed below college-level Significant barrier (Bailey, Jeong, and Cho, 2010) First interaction is to tell students they don’t belong Imply that most students are not ready for college and are likely to fail Convinces many, including our students Substantial majority of community colleges rely heavily if not nearly entirely on standardized assessment Most community college students placed into coursework below college-level Significant barrier to completion (Bailey, Jeong, and Cho, 2010) First interaction with most students is to tell them they’re not ready for college Implies that most students are not ready for college and are likely to fail Convinces many Including our students Very first interaction with student is to tell them they’re not ready for college “You’re not good enough” You don’t belong You’re not really college material Again, and again. Internalized by faculty, staff, administration, counselors, public Eventually, tragically, many of the students Placement into remediation is a significant barrier to completion Two enormous systemic changes: More, longer, slower remediation Reduction of access (offerings at bottom of sequences first to get restricted)

5 Conventional Wisdom Explaining Assessment Results
It is a problem with today’s students Students are simply, vastly unprepared for college Kids these days …. It is a problem with public education Public education is failing to prepare students Teachers these days… Not my students, not my teachers.. Teachers in general, students in general, sometimes other problematic adjectives added like urban.

6 What If the Conventional Wisdom is Wrong?
Substantial, long-term increase in IQ: 18-24 with HS degree: 90% - highest ever: National Assessment of Educational Progress: at all-time highs in virtually every demographic Research increasingly questions effectiveness of standardized assessment for placement Little relation to college course outcomes. (e.g., Belfield & Crosta, 2012; Edgescombe, 2011; Scott-Clayton, 2012; Scott-Clayton & Rodriguez, 2012): NAGB, 2012: Incredible variability in cutscores; 2-year colleges often use HIGHER cutscores than 4-year Hiss & Franks, 2014: Underestimates capability of students of color, women, first gen college students, low SES: the National Assessment of Educational Progress from the NCES National Center for Educational Statistics National Assessment Governing Board Austin Peay – 50% remediation, poor completion – eliminate remediation, less prepared students enroll in college-level with 2 hours/week in learning lab required. Pre-53% pass developmental math, 30% college level w/in 2 years. Post – 67% complete college-level. English – 54% to 76%:

7 Big questions What if the problem is not with our students, but with how we have assessed their capabilities? OR What if one of the barriers to our students’ successful transition to college is one that we fully control?

8 LBCC Research Five longitudinal cohorts tracking more than 7,000 HS grads who attend LBCC directly after high school built with help of Cal-PASS Examined predictive utility of wide range of high school achievement data most notably 11th grade California Standards Test (CST) scores and high school grades For predicting: How students are assessed and placed into developmental skills sequences How students perform in those classes (and for understanding alignment between them) Highly simplified presentation of unstandardized ordinal and logistic regression Lots more too it and RP has posted a lot of it

9 * p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10
Alignment in English (unstandardized ordinal and logistic regression coefficients) High school, course you took, A-G classes, more. What’s more important than 11th or 12th grade English – Math CST score – almost as strong as overall GPA. Most important predictor of performance – what course you took – after that – it’s your overall GPA. * p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10

10 * p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10
Alignment in Math Math much more complicated than this – the last course you took/which CST matters a lot in performance as does the difficulty level of the which course you take. * p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10

11 Key Takeaways Assessment should predict how students will perform at our colleges Instead: Standardized tests best predict standardized tests Classroom performance best predicts classroom performance More information tells us more about students than less information Replicated statewide by STEPS project, conceptually replicated by CCRC work Significant opportunities exist to improve placement, student achievement, and students’ college experience.

12 Multiple Measures Placement: Transfer-level Placement Rates F2012
Attending a community college >75% students of color, >50% Hispanic Largely eligible for the BOGG fee waiver and Pell grants Two major communities of English as a Second Language students Mostly first person in family to go to college Many first or second generation immigrants

13 F2012 Promise Pathways vs. Fall 2011 2-year rates of achievement
Compares first year rates of achievement to most recent 6-year cohort. Pathways is ahead, in many cases quite significantly.

14 Success rates in transfer-level courses Fall 2012
Neither of these differences approach significance, p >.30

15 Equity impact: F2011 Baseline Equity Gaps for 2-year rates of achievement
Our equity gaps are typical of equity gaps seen at most California Community Colleges…

16 Equity impact: F2012 Pathways 2-year rates of achievement
Our equity gaps are typical of equity gaps seen at most California Community Colleges…

17 How might this change how we understand college readiness?
Notes: unduplicated by student but counting bottom category first each time. Majority of the LBCC placement is the alternative placements.

18 What was gained through evidence based approach to transition to college
Dramatic increases in students attaining early educational milestones, & shorter times to do so New discussion of research and instructional pedagogy, kick-starting experimentation and innovation Strong challenges to conventional wisdom and perceptions of students by administration, staff, faculty, and students themselves Saved thousands of units of unnecessary remediation Reminder of the forgotten second instruction of Daedalus We keep on using these tests. I do not think they mean what we think they mean… Just as important not to fly too low. Concrete achievable steps that any college can take to dramatically improve all of our students’ futures. Can your college do this research? Can this research help your students too? Can your college afford to implement something like this? Making the primary changes to assessment and placement are not only virtually cost-free – it can save your college money and direct its resources more effectively and improve outcomes Low cost, outcome accelerating, potential long-term savings – can your college afford not to try? We’ve built a model, made it available through the work of the RP Group, and have already been providing support to over 20 California Community College

19 Contact Information Research questions/data requests John Hetts, Educational Results Partnership ext. 208 cell General questions about Promise Pathways or Long Beach College Promise Alicia Kruizenga, Director of School Relations and International Education: (562)

20 Additional Resources Background research
Achieving the Dream/Jobs for the Future summary of alternative assessment CCRC research on Assessment, Placement, and Progression in Developmental Education and RP Group’s Student Transcript-Enhanced Placement (STEPS) Project Step by Step process for replication: More information about our research Similar CCC research and implementation: Peralta CCD:, Grossmont-Cuyamaca: State-level redesigns of assessment and placement North Carolina – 2.6 GPA & 4 years of math transfer-level Florida – developmental educational optional Virginia – re-design of assessment in, modularization of math Connecticut – dev. education cut to 1 semester, coreq only Hawaii CCs- pilots using HSGPA in assessment Colorado, Montana, Mississippi* - move to corequisite models National level attention on assessment and placement 2013: US Governmental Accountability Office report on Developmental Education in Community Colleges 2014: Center for the Analysis of Postsecondary Readiness established by CCRC, MDRC

Download ppt "East Bay Career Pathways Regional Convening #2 December 4, 2014"

Similar presentations

Ads by Google