Introducing the Educational Value Added Assessment System (EVAAS) Phase I

Presentation Outcomes At the end of this presentation you should: 1.Understand the difference between achievement and growth. 2.Be a little less anxious about value added. 3.Have the answers to a few frequently asks questions. 4.Want to know more.

Student Learning Instruction Assessment Academic Content Standards/Learning Targets Professional Learning Community

Value Added 101 It is very complicated statistical procedure to measure student growth, but it is highly reliable and valid.It is very complicated statistical procedure to measure student growth, but it is highly reliable and valid. It’s not meant to be a hammer or stick to prove that people are not doing their jobs. It’s another data tool to help us improve.It’s not meant to be a hammer or stick to prove that people are not doing their jobs. It’s another data tool to help us improve.

Ohio Revised Code & Value Added “…implement a value-added progress dimension for school districts & buildings…”“…implement a value-added progress dimension for school districts & buildings…” “…shall incorporate the value-added progress dimension into the report cards and performance ratings issued for districts and buildings…”“…shall incorporate the value-added progress dimension into the report cards and performance ratings issued for districts and buildings…” “A scale for describing the levels of academic progress in reading and math relative to a standard year of academic growth in those subjects…”“A scale for describing the levels of academic progress in reading and math relative to a standard year of academic growth in those subjects…”

Ohio’s Value Added Timeline –Governor’s Commission for Student Success How can we make it clear to students, parents and educators what students should know and be able to do in each grade and before graduation?How can we make it clear to students, parents and educators what students should know and be able to do in each grade and before graduation? How should we measure student performance and progress in each grade?How should we measure student performance and progress in each grade? –Senate Bill 1: Established new Ohio standards and testing schedule –Battelle begins Project SOAR and pilots with districts

Ohio’s Value Added Timeline –Ohio House Bill 3: New Ohio standards and testing Aligned Ohio’s testing with NCLBAligned Ohio’s testing with NCLB Value added analysis added to accountability systemValue added analysis added to accountability system Created the Ohio Accountability Task ForceCreated the Ohio Accountability Task Force

Ohio’s Value Added Timeline –DVAS trained statewide –4 th grade reading/math analysis from spring 2006 testing –Statewide Pilot: Grades 4-8 reading/math analysis from spring 2007 testing 2008 and beyond2008 and beyond –Added accountability measure –Will appear on the district and building report cards

Ohio’s Value Added Created and operated by ODECreated and operated by ODE Piloted in 2006Piloted in 2006 Requires at least 2 data points to make a VA determination.Requires at least 2 data points to make a VA determination. Analyzes math/reading in grades 4-8Analyzes math/reading in grades 4-8 Uses data from state achievement testsUses data from state achievement tests Expected growth is set by State Board of EducationExpected growth is set by State Board of Education 27 out of 50 states use a growth model27 out of 50 states use a growth model National trendNational trend

Achievement vs. Growth This is a fundamental understanding

Suppose this box represents a group of students for a given grade level and subject area… Low High

Low … and we fast forward to the end of the given school year

An Achievement Status Consideration Considerations for Gauging Effectiveness: 1) External Standard (proficiency level) 2) Student’s Ending Point of Achievement ? (Yes/No) … sets a proficiency level students are expected to meet.

A Value-Added Consideration Considerations for Gauging Effectiveness: 1) Student’s Prior Achievement Level 2) Student’s Ending Point of Achievement … sets one end of the measurement at the students’ entry point, not at an external standard. How much gain?

Introducing Value Added Progress Measures DVD

Commonly Asked Questions

Why is measuring achievement & progress important? Achievement gives a snapshot of achievement at a particular point in time.Achievement gives a snapshot of achievement at a particular point in time. Achievement tests don’t tell growth.Achievement tests don’t tell growth. A measure of progress begins to fill in missing pieces to the puzzle.A measure of progress begins to fill in missing pieces to the puzzle. A balanced way of measuring student learning.A balanced way of measuring student learning. It is the law.It is the law.

Success?!? Failure?!?

How will this help me as an educator? Measures growth from where the student enters the grade level.Measures growth from where the student enters the grade level. Measures progress at building/grade levels.Measures progress at building/grade levels. Shows if programs/instruction are impacting student growth.Shows if programs/instruction are impacting student growth. Aligns classroom resources to help all.Aligns classroom resources to help all.

But I’m a teacher who isn’t directly affected by value added… Our fundamental goal as professionals is to help students achieve and growOur fundamental goal as professionals is to help students achieve and grow Pre-2 teachers lay the foundation for student learning and impact growthPre-2 teachers lay the foundation for student learning and impact growth Will most likely extendWill most likely extend to other grades and subjects

How can you compare tests from different grades? A statistical measure is used to make achievement tests from different years “apples to apples”.A statistical measure is used to make achievement tests from different years “apples to apples”. The “metric”, or formula, used to determine progress is complex, but has been demonstrated in other states to be valid and reliableThe “metric”, or formula, used to determine progress is complex, but has been demonstrated in other states to be valid and reliable It takes into account past tests (including reading tests when looking at math results).It takes into account past tests (including reading tests when looking at math results).

How is value added different than what we’re currently doing? Current system measures only achievement (what percentage of kids pass the test).Current system measures only achievement (what percentage of kids pass the test). New system measures progress towards the standard, so can more accurately reflect the growth of ALL students, especially those who may be hard-pressed to ever reach the passing rate.New system measures progress towards the standard, so can more accurately reflect the growth of ALL students, especially those who may be hard-pressed to ever reach the passing rate.

Will we be required to test more? No…VAAS uses the currently administered achievement testsNo…VAAS uses the currently administered achievement tests

How can value added positively impact gifted and talented students? GT kids achieve – but how much did they grow?GT kids achieve – but how much did they grow? EVAAS helps plan for growth among all student populations.EVAAS helps plan for growth among all student populations. Can high achieving students demonstrate growth?Can high achieving students demonstrate growth? –YES—the OAT and OGT have what’s called adequate stretch in order to measure growth in high achievers

Is it harder to determine growth for students in high achieving districts like ours? No.No. Deals with the concept of “test stretch”/ceiling effect.Deals with the concept of “test stretch”/ceiling effect. Tested group is divided into quintiles.Tested group is divided into quintiles. Ceiling effect would exist if a large number of the students in the 5 th quintile (top group) all tested at the highest possible level.Ceiling effect would exist if a large number of the students in the 5 th quintile (top group) all tested at the highest possible level.

How does this relate to the professional development that has been done? PD focus has been on:PD focus has been on: –Instructional strategies (Forget and Zimmerman)…these were meant to increase effectiveness We will continue to identify ways to increase teacher quality and effectiveness (using data, focusing on essential learning outcomes, etc.)We will continue to identify ways to increase teacher quality and effectiveness (using data, focusing on essential learning outcomes, etc.) Creates a continued need for assessment literacy and assessment fluency.Creates a continued need for assessment literacy and assessment fluency. Data may ID need for moreData may ID need for more professional development.

Where do you see value added going? Will it impact other subjects/grades? Valuable tool.Valuable tool. Refine our data management capabilities.Refine our data management capabilities. Accomplish our core mission = high levels of achievement for all students.Accomplish our core mission = high levels of achievement for all students. Value added data in reading/math for grades 4-8Value added data in reading/math for grades 4-8 Most likely will extend to high school level.Most likely will extend to high school level. Ohio is just beginning its roll out.Ohio is just beginning its roll out.

What happens next with EVAAS in our district? We need to quickly scale this up.We need to quickly scale this up. Phase I (today)Phase I (today) –Overview of Educational Value Added Concepts Phase II (Jan/Feb)Phase II (Jan/Feb) –Looking at previous data and reading graphs Phase III (Feb/Mar)Phase III (Feb/Mar) –Looking at current data & creating a plan –Plan most likely will include further PD