1. University of Virginia
CASTL
Center for Advanced Study
of Teaching and Learning
Improving impacts of classrooms: Professional Development and Classroom Observation
Robert C. Pianta, Ph.D.
University of Virginia
CASTL

2. Questions and challenges for policy, research, and training in education
What are offered to students in classrooms?
Do interactions with teachers and experiences in classrooms matter for students?
Can observation leverage improvements quality, and effectiveness of teachers/teaching?
Measure, validate, improve teacher quality at scale through standardized observation

3. Results of large-scale observational studies
National-level studies
National Center for Early Development and Learning (NCEDL)
NICHD Study of Early Child Care
Up to 1,000 settings observed at preschool, K, 1, 3, 5 ― more than 4,000 classrooms
Largest set of systematic standardized classroom observation in U.S. schools
All teachers credentialed/certified

4. Observations and large-scale applications
Trade-offs and decisions
Multiple versus single occasions
Length of the “window”
Time of day / content of instruction
Unit of analysis – global or micro
Classroom-level or child-level
Training demands and reliability
Applicable across diverse settings
No system can address every concern
Ultimate criterion is link to child outcomes

5. Describing opportunities to learn: Counting behaviors, activities, practices
Vast majority of interaction/activity is whole group or individual seatwork
Few, if any, social or instructional interactions between teacher and individual child
Mostly literacy activities
Exceptional variation within and across grades
Consistent patterns from pre-k to 5th grade

6. How do students spend time?
High levels (30%) of “business/routine” activity
pk-5: managing materials, routines
High levels of “basic skills” focus
7:1 in pk-1; 14:1 in 3-5
Ratio of listening, sitting, watching: Doing
10: pk-1 1,3,5
Summary characterization of the classrooms
(NICHD ECCRN 2002, 2004)

7. Rating interactions: What is the CLASS?
Derived from developmentally-informed analysis of settings and putative impacts on broad-based outcomes.
Focus on dimensions of teacher-student interaction in 3 domains - a theoretical claim about latent structure of classroom settings
Emotional Support
Organization / Management
Instructional Support

8. Instructional Support Emotional Classroom Organization
DOMAINS
Positive climate
Negative climate
Sensitivity
Regard for student perspective
Concept development
Quality of feedback
Language modeling
Behavior management
Productivity
Instructional learning
formats
DIMENSIONS
Relationships, Affect, Respect, Communication
Punitive, Sarcasm/ disrespect, Negativity
Aware, Responsive, Address problem,
Comfort
Flexibility, Autonomy, Student expression
Clear expectation, Proactive, Redirection
Maximize time, Efficient routines and transitions
Variety, Promote student interest, Clarity, Engaging
Analysis/reasoning, Creativity, Integration
Feedback loops, Encourage responses, Expand performance
Conversation, Open-ended, Repeat/extend, Advanced language
INDICATORS

9. Classroom ratings: CLASS PK-5
Positive climate
Negative climate
Teacher sensitivity
Regard for student perspectives
Effective behavior management
Learning formats/engagement
Productivity
Concept development
Evaluative feedback
Language modeling
Emotional
Support
Organization/
Management
Instructional
Support

10. What is the quality of the classroom setting?
Positive emotional climate
Productivity
Quality of feedback 1 2 3 4 5 6 7 1 2 3 4 5 6 7
General distributions of classrooms on selected individual qualitative scales. Do not see significant differences in distributions across grade, so can compile them this way. 1 2 3 4 5 6 7

11. Profiles of classroom quality: First grade7 6 5 4
Quality
Emotional 3
Instructional 2 1
Positive emotional
climate
Low academic
demand
Very positive
emotional climate
High instructional
quality
Mediocre,
Low academic
demand
Negative emotional
climate,
Low instructional
quality
31%
23%
29%
17%

12. Associations with structural/selection factors
Exceptional variability within and across grades, including across the same grade/curriculum. Stability across grades is low – 15%
Little to no association of observed interactions with:
Teacher experience or training
Teacher salary
Small associations with structure and selection ( )
Class size: larger classes more structured; smaller classes more social and higher instructional quality
Family income/education related to more positive ratings
Students needing access to stable high-quality instruction do not typically receive it – 10% rate for low achievers
1) Very wide variation - classrooms vary as much as kids do. This variation may undermine classroom effects. Called into question some assumptions about school. Also note all teachers were credentialed.
2) No connections to “regulables” - raises question about defining “teacher quality” in terms of degrees, etc.

13. Classroom interactions and children’s social and academic performance
Designs that isolate effects for instructional and emotional inputs controlling for other influences – in growth models
Family and demographic factors
Child’s prior performance
Structural features of schooling
Primarily small main effects (+/- .10)
Instructional and emotional quality predict more positive achievement and social outcomes
Larger effects on more proximal outcomes (e.g., child engagement)
More instruction in literacy and math also predict to those outcomes
Stronger effects for different groups of children
Low maternal education
Adjustment problems in K
Poor
Summary of results from papers on effects. See recent Hamre & Pianta Child Development paper on gap-closing 1st grade classrooms (Hamre, B., & Pianta, R. (2005). Can instructional and emotional support in the first grade classroom make a difference for children at risk of school failure? Child Development, 76(5), ).

14. Pre-k quality and growth in child outcomes
Where should we focus attention in policy, program development, and teacher preparation?
Predicting achievement growth during preK from:
Structural features (teacher ed., curriculum, etc.)
Observed interactions (ECERS, CLASS)
No association of structure with outcome, singly or in combination (e.g., NIEER index)
Instructional and Emotional Supports (CLASS) predict positive changes in literacy, language, and math skills – small effect sizes – persist into kindergarten

15. Gains in Gr. 1 achievement in instructionally supportive classrooms
107
106
105
Standardized tests of achievement adjusted
104
103
High educ.
102
101
Low educ.
100 99 98
Low
Moderate
High
1st Grade Instructional Support

16. Gains in Gr. 1 achievement in emotionally supportive classrooms
107
106
Kindergarten
adjustment
problems
105
104
Standardized tests of achievement adjusted
103
No problems
102
101
Multiple problems
100 99 98
Low
Moderate
High
1st Grade Emotional Support

17. Implications of our work P-5
Redefine “teacher quality” in terms of performance/interactions in classroom
Strive toward moving instructional dimensions and implementation “up” the scale of quality interactions
Develop training and support approaches that address teachers’ interactions with children
Approach these goals systematically and scientifically with standardized, validated observations at core
A science of teaching and teacher-training that relies on direct and validated observation

18. Measurement issues/directions
Develop extension of CLASS for grades 6-12
Ecometrics – Partitioning variance related to rater, time, day, season, window, consider alternative units of analysis.
Global features more stable, valid. Reliability improved through adding raters in the system. Implications for scalability and for research on intervention effects, etc.
Informant-report version; empirical tests of content-knowledge hypotheses

19. Standardized observation: Support for high-quality interactions
Professional development/ training
Observational Assessment
Social and academic outcomes for children
Resource allocation
CLASS
Instructional
Organization
Social
Improved teacher outcomes
Evaluation
Curriculum

20. myteachingpartner.net

21. Focus on interactions in classrooms
Teacher-child interactions and relationships are the means by which the curricula are implemented
MTP uses CLASS as the basis for defining high-quality implementation and as the target/focus of professional development
Goals of MTP are to:
Increase teachers’ observation skills in identifying interactive behaviors and cues related to CLASS
Identify children’s differential responses to teacher behaviors
Increase teachers’ skills in identifying alternative responses to children's’ cues – create opportunity
myteachingpartner.net

22. MTP support for teachers
Two forms of web-based support for quality implementation
MTP Activities
Teacher-child interactions and relationships: CLASS
Student growth in language, literacy and social relationships
myteachingpartner.net

23. Project design 240 Participating PreK teachers statewide - VPI
3 Conditions of participation / levels of support
Consultancy plus website plus curriculum
Website access plus curriculum
Materials/curriculum only
All conditions
Receive iBook laptops
Receive MTP activities
Complete surveys and assess 4 randomly selected children
Bullet 3: 3 Levels of involvement. ALL 3 get computer and some level of activities and access to the MTP website
DIFFERENCES:
Materials Group: basic access and not required to implement
Consultancy and Website Access: access to video examples on website of implementation and videotape their implementation
Consultancy only: Work with consultant over 2 years….(Grace will describe shortly)
myteachingpartner.net

24. Quality teaching videos: PK-3
myteachingpartner.net

25. CLASS examples: PK-3
myteachingpartner.net

26. MTP consultation cycle
Classroom video recording
at an established time 1
Teacher and consultant
meet and discuss
teaching practices 4
Consultant reviews
and edits video clips 2
Teacher reviews clips
and reflects on practice 3
myteachingpartner.net

27. Prompts
Intended to identify positive aspect of teacher’s instruction in relation to a class dimension
“This clip is a nice demonstration of concept development. You ask the children why the girl in the book is happy. You receive one answer and then go on and get ideas from two other children. What other strategies do you use to promote concept development?”
myteachingpartner.net

28. Prompts
Intended to help a teacher identify CLASS dimensions and examples in her teaching interactions
“Here you quickly and effectively redirect the children back to the story. As you watch this clip, tell me what you are doing to help the children remember the rules.”
myteachingpartner.net

29. Feedback on how the teacher implements lessons
Prompts
Feedback on how the teacher implements lessons
“Some of the other children are
not engaged in this lesson.
What kinds of learning formats
might you have used to draw
all of the children in?”
myteachingpartner.net

30. MTP-S Prompts: Feedback for teachers
myteachingpartner.net

31. Effects of MTP support on interactions
Examine both effects of condition on outcome (web vs consultation), treatment on treated, and moderation with regard to classroom demands.
Teachers receiving consultation show greater increases in quality of instructional interactions; early career teachers who view CLASS videos show gains in interactions with children; effects seem attributable to video review
Consultation moderates poverty effect

32. Changes in sensitivity for teachers in the MTP consultation and web only study conditions

33. Changes in language stimulation for teachers in the MTP consultation and web conditions

34. Moderating effects of study condition on the association between classroom poverty and changes in teacher sensitivity

35. Moderating effects of study condition on the association between visiting video pages and changes in teacher sensitivity

36. Effects of MTP support on child outcomes
Examine effects of condition and treatment on treated with Consultancy, Web, and Activities groups.
When teachers participate in consultation, children show greater gains in tests of early literacy

39. Associations between Teachers’ Exposure to the MyTeachingPartner Consultancy, Language and Literacy Activities and Web-Site, and Children’s Development of Language and Literacy Skills during Pre-K
Definitional Vocabulary
Phonological Awareness
Print Knowledge
PALS
Total
Level-1 B SE
Child Characteristics
Pre-test
0.45***
0.03
0.53***
0.48***
0.02
0.49***
Boy
-0.06
0.17
-0.92**
0.27
-1.19**
0.39
-1.33*
0.64
Non-English
-0.66
0.34
-0.26
0.52
0.46
0.70
0.51
1.18
Maternal Education (years)
0.11*
0.05
0.30***
0.08
0.13
0.11
0.12
0.18
Poverty
-0.24
0.24
-0.07
0.40
-0.21
0.47
-0.58
0.83
Second Year of Intervention
0.55**
0.62*
0.28
0.50
0.35
1.62**
0.59
*p≤.05. **p≤.01. ***p≤.001.

40. Associations between Teachers’ Exposure to the MyTeachingPartner Consultancy, Language and Literacy Activities and Web-Site, and Children’s Development of Language and Literacy Skills during Pre-K (continued)
Definitional
Vocabulary
Phonological
Awareness
Print
Knowledge
PALS
Total
Level-2 B SE
Classroom Characteristics
Mean Pre-test Score
0.17**
0.06
0.07
0.08
0.12
-0.11
Proportion in Poverty
0.04
0.60
0.44
1.06
0.42
1.72
0.53
2.80
Proportion Non-English
-0.35
0.63
-0.90
1.01
-3.00
1.67
-10.2***
2.70
Mean Mother’s Education
-0.05
0.16
0.30
0.28
-0.04
0.47
-0.27
0.79
Teacher Characteristics
Advanced Degree
0.22
-0.44
0.36
-0.32
0.62
0.03
1.05
Major in ECE
0.33
0.21
0.59
0.35
2.69**
0.99
Years Teaching PK
-0.00
0.01
0.02
Intervention Components
GT 20 hours/0 hours
1.03**
-0.12
0.54
0.88
0.95
2.45
1.50
GT 20 hours/LT 20 hours
0.76*
0.31
-0.22
0.57
1.40
0.90
2.23
1.54
Hours per week MTP-LL Activities
0.14
0.67**
0.24
0.38
1.41*
0.64
Low Web Use
0.32
-0.33
1.09
0.94
1.98
1.52
Medium Web Use
0.29
-0.54
0.48
0.41
0.82
1.53
1.34
*p≤.05. **p≤.01. ***p≤.001.

41. Moderating Effects of Teachers Years of Experience on the Association Between Exposure to the Consultancy and Children’s Development of Print Knowledge

42. Standardized observation of interactions
Is feasible, reliable and valid. A scalable language and lens for classroom settings
Three domains: Emotional, Organizational, Instructional appear valid across grades
A lever for research on teacher professional development and preparation to increase setting quality and child outcomes
Implications for accountability systems, teacher quality, research on teacher ed.