Recent public laws such as Individuals with Disabilities Education Improvement Act (IDEIA, 2004) and No Child Left Behind Act (NCLB,2002) aim to establish a climate of accountability for educators. Teachers and administrators are now mandated to collect and report data on all students prior to making educational decisions. Educators are in need of empirically based and efficient data collection methods. To date, there has been considerable research on academic behavior monitoring methods such as Curriculum Based Measurement and high stakes testing. Unfortunately, social behavior monitoring techniques have not enjoyed the same attention in the research literature. The current study aims to gather empirical support for a behavior rating tool called Direct Behavior Ratings (DBRs) that can be used by teachers to rate social or academic student behaviors in the classrooms. DBR is a time-efficient observational system in which target behaviors are operationally defined and ratings are collected on a daily or weekly basis to be shared with parents and administrators. DBR is a less complicated method than Systematic Direct Observation and may require less training. Some older research studies concluded that training increased the accuracy of Direct Behavior Ratings (Madle, Neisworth, Kurtz, 1980) while a more recent study found that short training sessions did not result in a significant difference (Chafeouleas, McDougal, Riley- Tillman, Ponahon, & Hilt, 2005). The purpose of this study was to investigate whether training the raters in the use of DBRs, increases the accuracy of the ratings. Training of educators is a time-consuming, financially costly endeavor. Prior to investing a considerable amount of time and funds on teacher training, we need to be fully informed on the effects of training and weigh its potential costs and benefits. Two independent samples t-tests assuming unequal variances were applied to the DBR data obtained by the trained (n=26) and untrained control (n=33) group of observers for the two target behaviors: actively manipulating legos, an on-task behavior and visually distracted, an off task behavior. Null hypothesis stating that the population means are equal for both the trained and the untrained group: [H0: µ 1= µ 2 ] for the actively manipulating legos (on-task) behavior was rejected, because the t-tests yielded significant t-values [t (54) = 1.67, p<.05] between the two means. The null hypothesis stating that the population means are equal for both the trained and the untrained group: [H0: µ 1= µ 2 ] for the visually distracted (off-task) behavior was also rejected, because the t-tests yielded significant t-values [t (58) = 1.67, p<.05] between the two means. The trained group’s mean for the actively manipulating legos (on-task) behavior (M trained = 16.06) was significantly different than the untrained group’s mean (M untrained = 29.35). The resulting means for the visually distracted behavior was also significantly different (M trained = and M untrained = 25.63). Coding reliability was assessed by having an independent judge code approximately 10% of the protocols collected on each of the three sessions of the study. Interrater reliability, as assessed by Pearson correlation, was.99. The mean DBR ratings for brief familiarization and formal training groups were compared to the SDO data resulting in the following effect sizes: For the visually distracted behavior, effect size of formal training versus a brief familiarization was very large: d = For the actively manipulating legos behavior, effect size of a formal training versus brief familiarization was large: d =.89. Study utilized a between-subjects design and was set in a controlled environment. Fifty-nine undergraduate students from a large Southeastern university were recruited as participants. Students received course credit for their participation in the study. The training condition included 26 participants who were trained for 30 minutes on DBR methods. Practice video clips were utilized and immediate feedback was provided. The control condition included 33 participants who were presented with a 5 minute brief-familiarization session on DBR methods. Mock practice video clips were utilized and no feedback was provided. Both groups of participants then viewed the same 12 video clips and rated the same two target behaviors using the DBR method. Results suggest that observers who are formally trained for 30 minutes in the use of DBRs rate more accurately than observers who are only familiarized for 5 minutes. Current study supports the premise that a 30-minute group training of teachers may be a worthwhile endeavor in increasing the accuracy of the DBRs and therefore result in empirically supported, well-informed educational decision making for the students. Future research should recruit public school teachers as observers in order to explore the training’s effects on accuracy of the ratings in natural classroom settings. Figure 1. This line graph illustrates the mean DBR ratings for the visually distracted behavior that were completed by 26 formally trained participants. Each participant’s mean rating is a line on a continuous scale (0-100 mm) as a function of the 4 students in the video clips. Students are labeled 1 through 4. Figure 2. This line graph illustrates the mean DBR ratings for the visually distracted behavior that were completed by 33 briefly familiarized participants. Each participant’s mean rating is a line on a continuous scale (0-100 mm) as a function of the 4 students in the video clips. Students are labeled 1 through 4. Figure 3. This line graph illustrates the mean DBR ratings for the actively manipulating legos behavior that were completed by 26 formally trained participants. Each participant’s mean rating is a line on a continuous scale (0-100 mm) as a function of the 4 students in the video clips. Students are labeled 1 through 4. Figure 4. This line graph illustrates the mean DBR ratings for the actively manipulating legos behavior that were completed by 33 briefly familiarized participants. Each participant’s mean rating is a line on a continuous scale (0-100 mm) as a function of the 4 students in the video clips. Students are labeled 1 through 4. SUMMARY AND CONCLUSIONS INTRODUCTIONRESULTS MATERIALS & METHODS RESULTS Effects of Training on the Accuracy of Direct Behavior Ratings Mine Dincer Schlientz & T. Chris-Riley-Tillman, Ph.D. Name Department School or College of Arts and Sciences East Carolina University Greenville, North Carolina BIBLIOGRAPHY Madle R. A., Neisworth, J. T. & Kurtz P. D. (1980). Biasing of hyperkinetic behavior ratings by diagnostic reports: Effects of observer training and assessment method. Journal of Learning Disabilities, 13, Chafouleas, S. M., McDougal, J. L., Riley-Tillman, T. C., Ponahon C. J., & Hilt, A. M. (2005). What do daily behavior report cards (DBRCs) measure?An initial comparison of DBRCs with direct observation for off-task behavior. Psychology in the Schools, 42, Project VIABLE: Direct Behavior Ratings (DBR) Principal Investigators: Chafouleas, S. M., Riley-Tillman, T. C., Christ, T. J. & Sugai, G. Preparation of this poster was supported by a grant from the Institute for Education Sciences (IES), U.S. Department of Education (R324B060014).