1. Research on Using Observation Systems with Special Educators
Nathan Jones
Boston University
Office of Special Education Program’s Project Directors’ Meeting
July 15, 2013

2. Observation Systems: Overview
9/19/2018
Observation Systems: Overview
Overall characteristics:
Typically mid- to high-inference and rubric-based
Require significant training
Designed to be used live or on video of classroom lessons
Crucially: These have been developed in research settings
Examples:
Danielson’s Framework for Teaching (FFT)
Meant to be comprehensive of all aspects of one’s teaching
Classroom Assessment Scoring System (CLASS)
Focuses on teacher-student interactions
Mathematical Quality of Instruction (MQI) protocol
Protocol for Language Arts Teaching Observations (PLATO)

3. Observation Systems: Questions and Answers from Research
9/19/2018
Observation Systems: Questions and Answers from Research
What are the qualities of observation scores being produced?
What are the design characteristics of observation systems and their underlying theoretical structures?
How do observation scores relate to other measures vis-a-vis larger validity arguments?
How do we conceptualize and research issues associated with fairness?
This is the big unanswered question for special educators

4. 9/19/2018
Potential Issues that might undermine the validity of observation systems with special educators
Many administrators—who are tasked with conducting observations—have not spent considerable time watching special ed. instruction
Quality/content of instruction provided in special ed. classes is in many cases impacted by general ed. teaching in the same subject area
Instruction makes up a smaller proportion of special educators’ day than it does for general educators
Is there a misalignment of definitions of teaching quality?
FFT
Effective teachers design activities that allow students to construct own knowledge
Students should take an active role in their learning by initiating activities, asking questions, and assessing the quality of their own work
Special Ed. Research
Effective teachers provide instruction that is explicit and focused on essential strategies and skills
Effective teachers have a deep knowledge of research-based practices
Effective teachers provide repeated practice opportunities

5. 2 Proposed Studies Validation of FFT for special education
9/19/2018
2 Proposed Studies
Validation of FFT for special education
special educators in Rhode Island
Sample Research Questions:
How reliable and independent are the dimensions of FFT? Do the dimensions of FFT conform to theory?
How do FFT scores correlate with other markers of teaching quality, including student growth, and expert rankings of special ed. teachers?
Development of a measure of teamwork
For use in contexts in which special educators and general educators share responsibility for instruction of SWDs
Draw on work done in other fields, including medicine and the military

6. Some initial hunches (based on existing research)
9/19/2018
Some initial hunches (based on existing research)
Within-teacher variance in scores on FFT will likely be high
Rater background and expertise in special education will likely make a difference
Special educators will systematically struggle on a subset of student-driven Elements (items) within the FFT
BUT: It is unlikely that districts/states will adopt a separate or modified protocol. We advocate for scoring support documents to assist in scoring special educators
A) Video exemplars and B) Modified rubric descriptions