Designing Content Targets for Alternate Assessments in Science: Reducing depth, breadth, and/or complexity Brian Gong Center for Assessment Web seminar.

Slides:



Advertisements
Similar presentations
Performance Assessment
Advertisements

National Accessible Reading Assessment Projects Defining Reading Proficiency for Accessible Large Scale Assessments Principles and Issues Paper American.
K-6 Science and Technology Consistent teaching – Assessing K-6 Science and Technology © 2006 Curriculum K-12 Directorate, NSW Department of Education and.
NCLB Basics From “What Parents of Students with Disabilities Need to Know & Do” National Center on Educational Outcomes University of Minnesota
The Network of Dynamic Learning Communities C 107 F N Increasing Rigor February 5, 2011.
1 Functions of Assessment Why do we assess students? Discuss in your group and suggest the three most important reasons.
Learning Teaching Enhancing Supporting Sharing ISLN March, 2015.
Oregon’s Alternate Assessment: Past, Present, and Future Tense Oregon Department of Education Dianna Carrizales, PhD Office of Student Learning and Partnerships.
Leadership for the Common Core in Mathematics, University of Wisconsin-Milwaukee Linking Assessment Targets to Instructional Tasks and DOK This.
A Presentation of the New Hampshire State Task Force on Mathematics Instruction Report to the State Board of Education, March 2012 Patty Ewen (603)
Educators Evaluating Quality Instructional Products (EQuIP) Using the Tri-State Quality Rubric for Mathematics.
Principles to Actions: Ensuring Mathematical Success
Consistency of Assessment
1 Alignment of Alternate Assessments to Grade-level Content Standards Brian Gong National Center for the Improvement of Educational Assessment Claudia.
Prepared by Jan Sheinker, Ed.D Points of view or opinions expressed in the paper are not necessarily those of the U.S. Department of Education, or Offices.
MCAS-Alt: Alternate Assessment in Massachusetts Technical Challenges and Approaches to Validity Daniel J. Wiener, Administrator of Inclusive Assessment.
New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Standard Setting Inclusive Assessment Seminar Marianne.
Planning Value of Planning What to consider when planning a lesson Learning Performance Structure of a Lesson Plan.
Who Are The “2% Students” …eligible to be judged as proficient based on modified grade-level academic achievement standards? Naomi Zigmond University of.
C R E S S T / U C L A Improving the Validity of Measures by Focusing on Learning Eva L. Baker CRESST National Conference: Research Goes to School Los Angeles,
Overview of the Dynamic Learning Maps Alternate Assessment System
New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Alignment Inclusive Assessment Seminar Brian Gong Claudia.
NCCSAD Advisory Board1 Research Objective Two Alignment Methodologies Diane M. Browder, PhD Claudia Flowers, PhD University of North Carolina at Charlotte.
Assessing Content Proficiency MAINE CURRICULUM LEADERS’ ASSOCIATION, SEPTEMBER 29, 2014 HEIDI MCGINLEY, EXECUTIVE DIRECTOR,
Shifting to a Standards- Based Mindset Through Quality Assessments and Backwards Design LMS Department Everett High School October 10, 2014.
The Common Core Curriculum By Dean Berry, Ed. D. Gregg Berry, B.A.
Australian Curriculum Science K-6
Wisconsin Extended Grade Band Standards
Building Effective Assessments. Agenda  Brief overview of Assess2Know content development  Assessment building pre-planning  Cognitive factors  Building.
Nicole Paulson CCSSO Webinar March 21, 2012 Transition to the Common Core State Standards in Elementary Math.
Alternate Assessment Changes. 9/14/20152 Important Information that You Need to Know The new Alternate Assessment will be a test given to students.
Standards-Based Science Instruction. Ohio’s Science Cognitive Demands Science is more than a body of knowledge. It must not be misperceived as lists of.
Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau.
Elementary & Middle School 2014 Mathematics MCAS Evaluation & Strategy.
Better Assessment for Students with Disabilities by Design: Going beyond inclusion in the “Common Assessment” RFP Brian Gong Center for Assessment Presentation.
SLOs for Students on GAA February 20, GAA SLO Submissions January 17, 2014 Thank you for coming today. The purpose of the session today.
Assessing Students With Disabilities: IDEA and NCLB Working Together.
Committee on the Assessment of K-12 Science Proficiency Board on Testing and Assessment and Board on Science Education National Academy of Sciences.
SLOs for Students on GAA January 17, GAA SLO Submissions January 17, 2014 Thank you for coming today. The purpose of the session today.
Developing Assessments for and of Deeper Learning [Day 2b-afternoon session] Santa Clara County Office of Education June 25, 2014 Karin K. Hess, Ed.D.
Achievethecore.org 1 Setting the Context for the Common Core State Standards Sandra Alberti Student Achievement Partners.
Illustration of a Validity Argument for Two Alternate Assessment Approaches Presentation at the OSEP Project Directors’ Conference Steve Ferrara American.
1 Alignment of Alternate Assessments to Grade-level Content Standards Brian Gong National Center for the Improvement of Educational Assessment Claudia.
Construct-Centered Design (CCD) What is CCD? Adaptation of aspects of learning-goals-driven design (Krajcik, McNeill, & Reiser, 2007) and evidence- centered.
Assessment for learning
Professional Certificate in Electoral Processes Understanding and Demonstrating Assessment Criteria Facilitator: Tony Cash.
1 Instructional Practices Task Group Chicago Meeting Progress Report April 20, 2007.
Georgia will lead the nation in improving student achievement. 1 Georgia Performance Standards Day 3: Assessment FOR Learning.
Early Childhood Outcomes Center New Tools in the Tool Box: What We Need in the Next Generation of Early Childhood Assessments Kathy Hebbeler ECO at SRI.
Standards-Based Science Assessment. Ohio’s Science Cognitive Demands Science is more than a body of knowledge. It must not be misperceived as lists of.
Fundamentals of Governance: Parliament and Government Understanding and Demonstrating Assessment Criteria Facilitator: Tony Cash.
No Child Left Behind Impact on Gwinnett County Public Schools’ Students and Schools.
Curriculum Focal Points PK-8— A Quest for Coherence Released September 12, 2006.
National Science Education Standards. Outline what students need to know, understand, and be able to do to be scientifically literate at different grade.
Ohio’s Alternate Assessments for Students with Disabilities Thomas Lather Office for Exceptional Children (614)
COMMON CORE STANDARDS C OLLEGE - AND C AREER - READINESS S TANDARDS North East Florida Educational ConsortiumFall 2011 F LORIDA ’ S P LAN FOR I MPLEMENTATION.
COMMON CORE STANDARDS C OLLEGE - AND C AREER - READINESS S TANDARDS North East Florida Educational ConsortiumFall 2011 F LORIDA ’ S P LAN FOR I MPLEMENTATION.
Spring 2012 Ohio’s Academic Content Standards - Extended for Students with Significant Cognitive Disabilities Increasing grade-level standard accessibility.
Casimir Middle School TUSD Transition to the Common Core.
Design Question 4 – Element 22
Take-Home Message: Principles Unique to Alternate Assessments
American Institutes for Research
Using Cognitive Science To Inform Instructional Design
Smarter Balanced Assessment Results
Federal Policy & Statewide Assessments for Students with Disabilities
Claudia Flowers, Diane Browder, & Shawnee Wakeman UNC Charlotte
Brian Gong Center for Assessment
TAKS, Inquiry, Standards and Assessment
Assessing Students With Disabilities: IDEA and NCLB Working Together
Claudia Flowers, Diane Browder, & Shawnee Wakeman UNC Charlotte
Presentation transcript:

Designing Content Targets for Alternate Assessments in Science: Reducing depth, breadth, and/or complexity Brian Gong Center for Assessment Web seminar on “Best Practices in Science Teaching and Testing for Students with Significant Cognitive Disabilities” Sponsored by the National Center on Education Outcomes March 12, 2007

2Gong - Science AA design - 3/12/07 Context for Policy Decisions State assessment in science annually once per grade span (elementary, middle, high school) State assessment in science annually once per grade span (elementary, middle, high school) State content standards and state achievement levels State content standards and state achievement levels Alternate assessment of science Alternate assessment of science Content targets “linked” to grade span content Content targets “linked” to grade span content Alternate achievement standards Alternate achievement standards Alternate assessments Alternate assessments Required to include all students; 1% cap on students “proficient” on alternate achievement standards; implement by ; Peer Review of standards and assessments in science; not in AYP accountability now, but proposed by USED for reauthorization of NCLB Required to include all students; 1% cap on students “proficient” on alternate achievement standards; implement by ; Peer Review of standards and assessments in science; not in AYP accountability now, but proposed by USED for reauthorization of NCLB

3Gong - Science AA design - 3/12/07 Designing Content Standards Content standards are a policy decision Content standards are a policy decision Informed by purpose, values, and context Informed by purpose, values, and context Content standards for regular assessment form foundation – get these strong before developing the content targets for the AA if possible Content standards for regular assessment form foundation – get these strong before developing the content targets for the AA if possible Content targets for alternate assessment must be related to grade-level regular content standards – but may have “reduced depth, breadth, and/or complexity” Content targets for alternate assessment must be related to grade-level regular content standards – but may have “reduced depth, breadth, and/or complexity” We’ll first consider establishing regular science content standards, and then consider ways to “reduce” for the alternate assessment, and justify those decisions We’ll first consider establishing regular science content standards, and then consider ways to “reduce” for the alternate assessment, and justify those decisions

4Gong - Science AA design - 3/12/07 Key Steps in Science Assessment Design 1. Be clear on purpose for assessment 2. Design reports and think about uses 3. Identify key science learning targets 4. Identify key science assessment targets 5. Decide what is most important and what should/can be reduced for alternate 6. Check your reduction: item content/skill, cognition/behavior, domain representation 7. Check your system

5Gong - Science AA design - 3/12/07 Be Clear on Your Purposes for Assessment Give valid information about student’s knowledge and skills in science in relation to state science standards Give valid information about student’s knowledge and skills in science in relation to state science standards Not NCLB Essential Purposes Encourage inclusion in instructional environment Encourage inclusion in instructional environment Evaluate effectiveness of a program or person Evaluate effectiveness of a program or person Encourage teaching of particular content and skills Encourage teaching of particular content and skills Get information about students’ performance in order to hold school accountable Get information about students’ performance in order to hold school accountable Inform how to help a learner on a particular skill, concept, etc. Inform how to help a learner on a particular skill, concept, etc. Other purposes Other purposes

6Gong - Science AA design - 3/12/07 Design Assessment Information and Uses Standards-based achievement levels (e.g., Proficient) Standards-based achievement levels (e.g., Proficient) Regular achievement levels – “A proficient student can…” Regular achievement levels – “A proficient student can…” Alternate achievement levels – “A proficient student can…” Alternate achievement levels – “A proficient student can…” Not NCLB Designs Individual student-referenced – “right for IEP,” “show what student can do no matter if off grade-level” – but… what about growth models? Individual student-referenced – “right for IEP,” “show what student can do no matter if off grade-level” – but… what about growth models? Individual item-referenced – “got item right 80% of time,” “scored a 3, so proficient” Individual item-referenced – “got item right 80% of time,” “scored a 3, so proficient” Norm-referenced – “in second quartile of reference group,” “in top 90% of this subgroup” Norm-referenced – “in second quartile of reference group,” “in top 90% of this subgroup” Diagnostic – “what can/can’t student do; why; so what should be done instructionally” Diagnostic – “what can/can’t student do; why; so what should be done instructionally”

7Gong - Science AA design - 3/12/07 Identify key learning targets State science standards – get these right first! State science standards – get these right first! What (content standards) What (content standards) How well (verbs, proficiency level descriptions) How well (verbs, proficiency level descriptions) When (grade span expectations) When (grade span expectations) Selection and Prioritization – a challenge in science! Selection and Prioritization – a challenge in science! Can it be learned well? Is it centrally important? – e.g., fewer than 15 big concepts per grade/year? Can it be learned well? Is it centrally important? – e.g., fewer than 15 big concepts per grade/year? Balance of emphasis – Life, Earth/Space, Physical, Inquiry, etc. Balance of emphasis – Life, Earth/Space, Physical, Inquiry, etc. Depth of knowledge – facts, skills, concepts, principles, models, evidence, applications, etc.? Depth of knowledge – facts, skills, concepts, principles, models, evidence, applications, etc.? Linkages – within disciplines, across disciplines, math, technology, etc.? Linkages – within disciplines, across disciplines, math, technology, etc.? Sequencing w/in grade span – e.g., if assessed in grade 5, is all content/performance level at grade 5? Sequencing w/in grade span – e.g., if assessed in grade 5, is all content/performance level at grade 5?

8Gong - Science AA design - 3/12/07 Identify key assessment inferences and claims Construct-performance interpretation Construct-performance interpretation Inference to proficiency in domain – “In science, the student will likely demonstrate proficient performance…” Inference to proficiency in domain – “In science, the student will likely demonstrate proficient performance…” Inference to proficiency across situations, time – “across these types of situations and time periods…” Inference to proficiency across situations, time – “across these types of situations and time periods…” Inference to proficiency across student performances – “because the student has performed like this…” Inference to proficiency across student performances – “because the student has performed like this…” What is sufficient evidence for the state and others to justify these claims (reliable and valid) What is sufficient evidence for the state and others to justify these claims (reliable and valid)

9Gong - Science AA design - 3/12/07 Decide what is most important and what can be reduced Reducing content scope Reducing content scope Reducing complexity Reducing complexity Reducing inferential scope Reducing inferential scope Prerequisites Prerequisites Reducing grade-level Reducing grade-level

10Gong - Science AA design - 3/12/07 Reducing Scope Select most important content Select most important content “Big ideas” – most significant science from scientists’ point of view “Big ideas” – most significant science from scientists’ point of view Learning power standards – needed to learn other important content and skills Learning power standards – needed to learn other important content and skills Unifying themes – what cuts across, unifies topics and science disciplines Unifying themes – what cuts across, unifies topics and science disciplines Central messages vs. validations, applications, examples Central messages vs. validations, applications, examples Instructional power – most engaging, persuasive, illustrative of science Instructional power – most engaging, persuasive, illustrative of science Select assessable in this context Select assessable in this context Justify! – primarily content and assessment validity views, not “what students are currently learning” Justify! – primarily content and assessment validity views, not “what students are currently learning”

11Gong - Science AA design - 3/12/07 Reducing Complexity Identify “Essences” Identify “Essences” in content in content in cognition/performance in cognition/performance Develop multiple entry points of performance Develop multiple entry points of performance More complex to less complex More complex to less complex Presentation/response Presentation/response Prerequisites Prerequisites

12Gong - Science AA design - 3/12/07 Reducing the Scope of Inferences Reduce the domain – e.g., “Provide evidence in three of eight content standards,” “These 20 tasks that assess these ten content standards are the assessment target” Reduce the domain – e.g., “Provide evidence in three of eight content standards,” “These 20 tasks that assess these ten content standards are the assessment target” Reduce the complexity – e.g., “provide various supports and assistance up to fully independent work,” “reduce the depth of knowledge from analyze to identify,” “student chooses from six provided options rather than generating options” Reduce the complexity – e.g., “provide various supports and assistance up to fully independent work,” “reduce the depth of knowledge from analyze to identify,” “student chooses from six provided options rather than generating options” Reduce the situations, time – e.g., “Perform in these (few) given situations,” “situations are known ahead of time,” “situations will not change over time,” “Performance is ‘banked’ before the end of the year,” “Must have multiple raters agree” Reduce the situations, time – e.g., “Perform in these (few) given situations,” “situations are known ahead of time,” “situations will not change over time,” “Performance is ‘banked’ before the end of the year,” “Must have multiple raters agree” Reduce the number and type of student performances – e.g., “Best performance counts rather than typical performance,” Reduce the number and type of student performances – e.g., “Best performance counts rather than typical performance,” Should provide justifications for reductions in scope of inferences – more than just a process description! Should provide justifications for reductions in scope of inferences – more than just a process description!

13Gong - Science AA design - 3/12/07 Prerequisites and “Linkage” Within-grade (span) prerequisite Within-grade (span) prerequisite Cognitive or content analysis, curriculum Cognitive or content analysis, curriculum USED guidance and Peer Review practice not clear Below-grade prerequisite Below-grade prerequisite Grades P-minus, P, K, 1, 2, 3, 4, 5, 6… Grades P-minus, P, K, 1, 2, 3, 4, 5, 6… What is the “grade-level” relationship? What is the “grade-level” relationship? The lowest levels are prerequisites for everything above The lowest levels are prerequisites for everything above Task analysis prerequisite Task analysis prerequisite Is the task decomposition retaining the essence, or is it substituting an out-of-grade or out-of-content area procedural skill or factoid? – example: “Number sense and operation  multiplication  two-digit multiplication  line-up columns procedure  one-to-one correspondence  discriminate symbolic numerals (identify ‘6,’ ‘7,’ etc.) Is the task decomposition retaining the essence, or is it substituting an out-of-grade or out-of-content area procedural skill or factoid? – example: “Number sense and operation  multiplication  two-digit multiplication  line-up columns procedure  one-to-one correspondence  discriminate symbolic numerals (identify ‘6,’ ‘7,’ etc.) Instructional context as content Instructional context as content

14Gong - Science AA design - 3/12/07 Cautions About Inferences from Prerequisites Inference: If student doesn’t know X, then s/he probably doesn’t know Y. Inference: If student doesn’t know X, then s/he probably doesn’t know Y. If student doesn’t know X, then s/he must learn X in order to learn Y. If student doesn’t know X, then s/he must learn X in order to learn Y. Inference: If student knows X, then s/he probably knows Y. Inference: If student knows X, then s/he probably knows Y. Cautions – need strong prerequisite chains for claims like these (be especially cautious about instructional claims) Cautions – need strong prerequisite chains for claims like these (be especially cautious about instructional claims) “Grade level/grade span” – will the state specify? – many different ways to structure and order topics by grade “Grade level/grade span” – will the state specify? – many different ways to structure and order topics by grade

15Gong - Science AA design - 3/12/07 Check your reduction Item content/skill Item content/skill Domain representation per grade span Domain representation per grade span Domain cohesion across grade spans Domain cohesion across grade spans Evidence required Evidence required Do they make sense? Do they make sense?

16Gong - Science AA design - 3/12/07 Check your system Correct, Complete, Coherent, Constructive Correct, Complete, Coherent, Constructive Fair, Useful, Sustainable Fair, Useful, Sustainable Monitoring validity Monitoring validity Validity of assessment inferences about student Validity of assessment inferences about student Validity of assessment design Validity of assessment design Validity of accountability decisions, uses, and impact Validity of accountability decisions, uses, and impact What are the differences between your alternate and your regular assessment systems? Justified? What are the differences between your alternate and your regular assessment systems? Justified?

17Gong - Science AA design - 3/12/07 Center for Assessment Brian Gong For more information: