Research on Using Observation Systems with Special Educators Nathan Jones Boston University Office of Special Education Program’s Project Directors’ Meeting July 15, 2013

Observation Systems: Overview 9/19/2018 Observation Systems: Overview Overall characteristics: Typically mid- to high-inference and rubric-based Require significant training Designed to be used live or on video of classroom lessons Crucially: These have been developed in research settings Examples: Danielson’s Framework for Teaching (FFT) Meant to be comprehensive of all aspects of one’s teaching Classroom Assessment Scoring System (CLASS) Focuses on teacher-student interactions Mathematical Quality of Instruction (MQI) protocol Protocol for Language Arts Teaching Observations (PLATO)

Observation Systems: Questions and Answers from Research 9/19/2018 Observation Systems: Questions and Answers from Research What are the qualities of observation scores being produced? What are the design characteristics of observation systems and their underlying theoretical structures? How do observation scores relate to other measures vis-a-vis larger validity arguments? How do we conceptualize and research issues associated with fairness? This is the big unanswered question for special educators

9/19/2018 Potential Issues that might undermine the validity of observation systems with special educators Many administrators—who are tasked with conducting observations—have not spent considerable time watching special ed. instruction Quality/content of instruction provided in special ed. classes is in many cases impacted by general ed. teaching in the same subject area Instruction makes up a smaller proportion of special educators’ day than it does for general educators Is there a misalignment of definitions of teaching quality? FFT Effective teachers design activities that allow students to construct own knowledge Students should take an active role in their learning by initiating activities, asking questions, and assessing the quality of their own work Special Ed. Research Effective teachers provide instruction that is explicit and focused on essential strategies and skills Effective teachers have a deep knowledge of research-based practices Effective teachers provide repeated practice opportunities

2 Proposed Studies Validation of FFT for special education 9/19/2018 2 Proposed Studies Validation of FFT for special education 80-100 special educators in Rhode Island Sample Research Questions: How reliable and independent are the dimensions of FFT? Do the dimensions of FFT conform to theory? How do FFT scores correlate with other markers of teaching quality, including student growth, and expert rankings of special ed. teachers? Development of a measure of teamwork For use in contexts in which special educators and general educators share responsibility for instruction of SWDs Draw on work done in other fields, including medicine and the military

Some initial hunches (based on existing research) 9/19/2018 Some initial hunches (based on existing research) Within-teacher variance in scores on FFT will likely be high Rater background and expertise in special education will likely make a difference Special educators will systematically struggle on a subset of student-driven Elements (items) within the FFT BUT: It is unlikely that districts/states will adopt a separate or modified protocol. We advocate for scoring support documents to assist in scoring special educators A) Video exemplars and B) Modified rubric descriptions