1. 1 Innovative Teaching and Learning (ITL) Research Corinne Singleton SRI International

2. 2  Study the ecosystems of educational innovation across country contexts  Inform our understanding of how innovative teaching practices look in classrooms across the globe  Inform our understanding of the factors that promote innovative teaching practices, and the conditions that support effective technology integration  Understand how those practices (innovative teaching and integration of ICT) relate to one another, and tie to improved learning outcomes for students  Support schools in their quest to improve teaching and learning ITL Research Project Goals

3. 3 Research is designed & coordinated globally  Common research questions, design, and instruments  Global analysis and reporting … and executed locally  Four countries participating in 2009-10 pilot year: Finland, Indonesia, Russia, Senegal (with four new countries added for 2010-11)  In each country, a research partner carries out the global research design with adaptations appropriate to the local context and interests of local education leaders Global/Local Research Design  Country research partners are trained to use research instruments and procedures in standard ways, enabling program- wide reporting

4. 4 National/Regional Level School/Teacher Level National or Regional Program Supports Innovative Teaching Practices Educator Attitudes School Culture and Supports Students’ 21 st Century Skills National or Regional Education System ICT Access and Support Classroom Level Student Level Research questions 4 Context and InputsPracticesOutcomes 1. To what extent are innovative teaching practices associated with 21st century learning outcomes? 2. What school-level conditions are associated with innovative teaching practices? 3. What national or regional program supports are associated with innovative teaching practices?

5. 5 MethodCountry Sampling GuidelinesActual Four-Country Sample Survey Schools 20-50 schools per country (enough to yield 600 teachers as below), representing 4 selected geographical regions 91 schools Teacher survey All teachers in survey schools (N=600 total), including participants and non-participants in country-specific focus programs 2406 teacher surveys School leader survey 1 school leader in each survey school (N=20 to 50) 82 school leader surveys Site Visit/LASW Schools 6 schools per country; a subset of the survey schools in 1-2 geographic regions, selected for innovative teaching practices 24 schools Teacher interviews Teacher observations 4 teachers interviewed per site visit school (N=24); 1 classroom observation per teacher 96 teacher interviews 96 classroom observations School leader interviews 1 school leader per site visit school (N=6) 24 school leader interviews Learning activities 6 learning activities from each of 8 teachers per site visit school (N=288); teachers of humanities and sciences 650 learning activities Student work 6 samples of student work for each of 4 learning activities collected per teacher (N=768) 3,647 samples of student work 5 ITL Research methods & samples

6. 6 Things like “21st-century learning opportunities” are easy to talk about but hard to implement, and just as hard to measure Learning activities (the assignments that students are asked to do) and student work (the work they submit in response) can serve as objective artifacts of actual classroom practice. Careful analysis of these artifacts: Helps us see what’s really happening in classrooms Gives us specific metrics for measuring instructional change and learning outcomes Gives teachers a common language to discuss teaching practice Progress toward instructional change can be measured across educational contexts, subject areas, and grades Why look at learning activities and student work?

7. 7 Evaluators collect samples of the assignments teachers give and the work that students do in response Evaluators recruit and train experienced teachers to act as independent coders within each country –Teachers of the same subjects/age levels as the teachers who submitted learning activities, but from different schools –This workshop can be a valuable professional development opportunity for the coders How it works

8. 8 This project developed rubrics and definitions that make “21 st century learning opportunities” concrete and measurable Coders analyze learning activities in 5 “dimensions,” looking for evidence that they encouraged students to develop skills in these areas: –Knowledge building –Problem-solving and innovation –Use of ICT for learning –Collaboration –Self-regulation Coding follows a common framework

9. LA Rubric for Problem Solving Dimension: Decision Steps Requires implementation in the real world? Code 1 Code 2 Code 4 Code 3 Requires implementation in the real world? NO YES Students choose their own problem? Main requirement is problem solving ? Code 3 NO YES

10. Sample Learning Activity #1

11. 11 Sample Learning Activity #2

12. 12 The same coders analyzed student work done in response to the learning activities, using rubrics to look for evidence of: –Knowledge construction –Problem-solving and innovation –Use of ICT for learning –Skilled communication The results (plus some statistical methods) allow us to paint a picture of the “21 st -century-ness” of teaching and learning, and how that changes over time Analysis of student work

13. 13 Mean scores, LA and SW On every dimension, for both LA and SW, over 50% of artifacts scored a “1” (the lowest possible score). Most teachers in the sample are at early stages of implementing innovative teaching practices. This chart shows mean scores across countries for learning activities and student work for each dimension.

14. 14 The quality of a teacher’s assignment strongly predicts the quality of the work that a student does in response. – In analysis of opportunities to build students’ 21st century skills (LA) and evidence of those skills in student work (SW), >90% of the variance in student work scores was due not to differences in the students but differences in the task they were asked to do. – In addition, the data suggest a “ceiling effect” imposed by teacher assignments: while it is possible for students to build and exhibit a greater level of 21st century skills than their learning activities call for, they rarely do Within these four countries, >50% of the learning activities and associated student work sampled by researchers achieved the lowest score on any given dimension of innovative teaching and learning. – While innovative teaching practice was typically a goal at these schools, actual classroom instruction does not yet reflect these goals. 14 Key findings: Innovative teaching and student learning

15. 15 Both LA and SW rubrics achieved high levels of inter-rater reliability within each country, suggesting that dimensions were clearly defined and consistently understood Distribution of results showed very high percentage of both LA and SW that scored ‘1’ (the lowest possible score) – Inter-rater reliability (above) may be artificially inflated due to the low variation in scores – This pattern limits the use of some analytic methods (e.g., regression) Design implications – Consider “lowering the bar” in the rubrics to allow more sensitivity to initial steps in innovative teaching 15 Pilot year methodological considerations: LASW

16. 16 In Year 2 of ITL research, we will extend the LASW research methodology for use as a tool for teacher professional development This strategy is based on a PD program used in Ireland, as part of a precursor program to ITL research In that context, our Irish research partners observed the following outcomes of the LASW PD program – Pedagogical shifts in the classroom – Emerging culture of self-evaluation – Development of a common language through which teachers discussed teaching practices – Deepening professional learning community among teachers Looking ahead: using LASW to support teaching & learning

17. 17  Visit: http://www.itlresearch.com/  Email: corinne.singleton@sri.com For more information…