Developing Potentials for Learning: Evidence, assessment, and progress

Developing Potentials for Learning: Evidence, assessment, and progress
John Hattie Visible Learning Lab University of Auckland New Zealand EARLI 2007

Three parts to the story
To identify major influences on achievement (using 700+ meta-analyses) To outline an assessment model based on these influences To demonstrate the power and joy of Educational Psychology

Influences on Achievement ?
Decreased Enhanced Zero 3

Reducing Class Size on Achievement?
What is the effect of reducing class size Hundreds of evaluations of reducing class size …. Decreased Enhanced Zero 4

Effect on Achievement over time?
Reducing Class Size .20 1.0 Decreased Enhanced Zero An effect-size of .20 1.0 advancing achievement 9 mths 3 yrs % improving rate of learning 10% 45% r variable & achievement .10 .45 % of students with treatment exceeding those not treated 8 34

The typical influence on achievement
So what is the typical effect across 750+ meta-analysis 50,000 studies, and 200+ million students

Effect on Achievement over time?
Typical Effect Size .20 1.0 .40 Decreased Enhanced Zero

Distribution of effects

Influences on Achievement
.40 .30 .15 .50 .60 .70 .80 .90 1.0 REVERSE Developmental Effects Typical Teacher ZONE OF DESIRED EFFECTS

Rank these 11 effects: Reducing disruptive behavior in the class
Feedback Acceleration of gifted students Reading Recovery Integrated curriculum programs Homework Individualized instruction Ability grouping Open vs. traditional classes Retention (holding back a year) Shifting schools (from 1 = highest effect to 11 = lowest effect)

Rank these 11 effects: Answers
Reducing disruptive behavior in the class Feedback Acceleration of gifted students Reading Recovery Integrated curriculum programs Homework Individualized instruction Ability grouping Open vs. traditional classes Retention (hold back a year) Shifting schools

Major domains of interest
Curricula Home School Student Teacher Teaching

The Disasters ... -.34 -.16 -.14 -.09 .01 .04 .06 .08 .09 Rank
Category Influence Studies Effects ES 100 Mobility (shifting schools) 181 540 -.34 99 Retention 207 2675 -.16 98 Television 31 235 -.14 97 Summer vacation 39 62 -.09 96 Open vs. traditional 315 333 .01 95 Multi-grade/age classes 94 72 .04 Inductive teaching 24 .06 93 Reading: Whole language 64 197 92 Perceptual-motor programs 180 637 .08 91 Out of school experiences 52 50 .09

The Well belows... .09 .11 .12 .15 .16 Rank Category Influence Studies
Effects ES 90 Distance education 788 1545 .09 89 Web based learning 10 88 Ability grouping 494 1363 .11 87 Teacher training 53 286 86 Diet on achievement 23 125 .12 85 Teacher subject matter knowledge 27 64 84 Gender (girls – boys) 2926 6051 83 Multi-media methods 244 133 .15 82 Problem based learning 203 345 81 Home-school programmes 14 .16

Not Worth it yet ... .17 .18 .19 .20 .21 .22 Rank Category Influence
Studies Effects ES 80 Extra-curricula programs 96 68 .17 79 Family structure 660 846 .18 78 Co-/team teaching 136 47 .19 77 Learning hierarchies 24 76 Aptitude/treatment interactions 61 340 75 Individualised instruction 581 1030 .20 74 Charter schools 18 73 Religious schools 56 72 Class size 785 .21 71 Teaching test taking 267 364 .22

Typical “average teacher” territory ...
Rank Category Influence Studies Effects ES 70 Finances 189 681 .23 69 Summer school 105 600 68 Competitive learning 831 203 .24 67 Programmed instruction 464 362 66 Within class grouping 148 297 .25 65 Mainstreaming 150 370 .28 64 Desegregation 335 723 63 Exercise/relaxation 227 1971 62 Audio-based teaching 146 48 61 Home visiting by teachers 71 52 .29 16

Close to average Rank Category Influence Studies Effects ES 60
Reducing anxiety 69 904 .30 59 Principals/school leaders on student achievement 344 1008 58 Ability grouping for gifted students 125 202 57 Homework 261 275 .31 56 Inquiry based teaching 205 420 55 Simulations and gaming 342 449 .32 54 Reading: Exposure to reading 145 324 .36 53 Bilingual programs 128 666 .37 52 Teacher positive expectations 635 745 51 Computer assisted instruction 4481 8079

Average … .39 .41 .43 .46 .47 .48 .49 Rank Category Influence Studies
Effects ES 50 Enrichment on gifted 214 543 .39 49 Integrated curriculum programs 61 80 48 Adjunct aids 138 323 .41 47 Hypermedia instruction 46 143 Behavioral organisers/adjunct questions 577 1933 45 Self-concept on achievement 324 2113 .43 44 Frequent/effects of testing 1077 .46 43 Early intervention 1627 9050 .47 42 Motivation on learning 322 979 .48 41 Small group learning 78 155 .49

Getting there … .49 .50 .51 .52 .53 Rank Category Influence Studies
Effects ES 40 Questioning 214 342 .49 39 Cooperative learning 2285 1519 38 Reading: Second/third chance programs 52 1395 .50 37 Play programs 70 36 Visual based/audio-visual teaching 468 3860 .51 35 Outdoor programs 187 429 .52 34 Concept mapping 91 105 33 Peer influences 12 122 .53 32 Keller's mastery learning program 263 162 31 Reading: Phonics instruction 407 5950

Let’s have them .... .55 .56 .57 .58 Rank Category Influence Studies
Effects ES 30 Reading: Visual-perception programs 762 5244 .55 29 Parental Involvement 694 1761 28 Peer tutoring 767 1200 27 Goals - challenging 454 671 .56 26 Mastery learning 369 284 .57 25 Social skills programs 540 3068 24 Socio-economic status 499 957 23 Home environment 35 109 22 Providing worked examples 62 151 21 Reading: Comprehension programs 365 2416 .58

Exciting …. .59 .60 .61 .64 .67 Rank Category Influence Studies
Effects ES 20 Direct instruction 304 597 .59 19 Time on task 64 100 18 Study skills 656 2446 17 Acceleration of gifted 60 412 .60 16 Problem solving teaching 221 719 .61 15 Teacher professional development on student achievement 450 1790 .64 14 Reading: Repeated reading programs 54 156 .67 13 Reading: Vocabulary programs 301 800 12 Meta-cognition strategies 43 123 11 Teaching students self-verbalisation 92 1061

The Winners ... 1.44 .86 .80 .77 .74 .73 .72 .70 Rank Category
Influence Studies Effects ES 1 Self-report grades 209 305 1.44 2 Absence of disruptive students 140 315 .86 3 Classroom behavioural 160 942 .80 4 Quality of teaching 141 195 .77 5 Reciprocal teaching 38 53 .74 6 Prior achievement 3387 8758 .73 7 Teacher-student relationships 229 1450 .72 8 Feedback 1276 1928 9 Providing formative evaluation to teachers 21 .70 10 Creativity programs 658 814 22

Various Influences .50 .45 .43 .39 .35 .23 .40 Teacher Curricula
Metas Studies People Effects ES se Teacher 29 2,052 .5m 5,379 .50 .05 Curricula 135 6,892 7m 29,476 .45 .07 Teaching 344 24,906 52m 50,953 .43 Student 133 10,735 37,308 .39 .04 Home 31 1,998 10m 3,968 .35 .06 School 96 4,019 4m 13,609 .23 Average 768 50,602 82/241m* 140,693 .40

Identifying what matters
Percentage of Achievement Variance Students Teachers Home Peers Schools Principal

Visible teaching & Visible learning
What some teachers do! In active, calculated and meaningful ways Providing multiple opportunities & alternatives Teaching learning strategies Around surface and deep learning That leads to students constructing learning

Visible Teaching – Visible Learning

Teachers Clear learning intentions Challenging success criteria
Range of learning strategies Know when students are not progressing Providing feedback Visibly learns themselves

Students … Understand learning intentions
Are challenged by success criteria Develop a range of learning strategies Know when they are not progressing Seek feedback Visibly teach themselves

The Contrast An active teacher, passionate for their subject and for learning, a change agent OR A facilitative, inquiry or discovery based provider of engaging activities

Activator or Facilitator ?

Activator or Facilitator ?
31

Teaching or Working Conditions?

An emphasis on learning strategies
NS NE ES Creativity Programs Teaching student self-verbalization Meta-cognition strategies Problem solving teaching Study skills Concept mapping Motivation on learning

Message for Learning Balance of surface, deep, & constructed knowing
Teachers preach deep, students see surface! Learning strategies To reduce cognitive load To use when stuck (welcome error!) Requires deliberative practice Builds expectations of “can do” Thrives on challenge Requires feedback

Self Regulation Need surface and deep to maximize meta-cognition
Involves questioning our constructions of knowledge & knowing Requires highly structured or direct teaching Is the heart of Visible learning and Visible teaching

Some worrying details

Assessment and Feedback

Assessment… Influence Effect Size Self-report grades 1.44 Feedback .72
Providing formative evaluation to teachers .70 Frequent/ Effects of testing .46 Teaching test taking skills .22

Teaching/Teachers needs to be evidence based
Whenever we test in classes it is primarily to help teachers know: Whether their teaching methods have been successful or not Whether their learning intentions are worthwhile & challenging Whether students are attaining their desired success criteria Which students have learnt or not learnt Where teachers can capitalize on student strengths & minimize gaps Where students are on the learning ladder Whether they have a shared conception of progress What is optimal to teach next

Priority to maximize FEEDBACK to THE TEACHER
Feedback is information provided by an agent (e.g., teacher, peer, book, parent, self/experience) regarding aspects of one’s performance or understanding.

Purpose of feedback provide alternative strategies to understand material increase effort, motivation or engagement confirm that the responses are correct or incorrect indicate that more information is available or needed point to directions that could be pursued to restructure understandings

Feedback is evidence about:
Where am I going? How am I going? Where to next?

Enhancing Learning Gap How ??? Levels

Summary of effect-sizes relating to types of feedback (adapted from Kluger & DeNisi, 1996)
Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect

Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect FB about changes from previous trials Yes No

Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect FB about changes from Yes previous trials No FB designed to Yes discourage student No

Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect FB about changes from Yes previous trials No FB designed to discourage Yes the student No Praise FB Yes No

Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect FT about changes from previous trials Yes No FT designed to discourage the student Yes No Praise FT Yes No No. of times FT was Lots provided Little

Moderator No. ES Correct feedback ‘Tis correct ‘Tis incorrect FB about changes from previous trials Yes No FB designed to discourage the student Yes No Praise FB Yes No No. of times FB was provided Lots Little Goal setting Difficult goals Easy, do your best goals

Thus … Provide correct feedback .43 About previous attempts .55
Related to more difficult goals .51 That does not discourage .33 or threaten their self-esteem .47

Assessment and Feedback: asTTle (Assessment Tools for Teaching and Learning)

What is Visible Learning?
Where are we going? related to both the rich ideas underlying the curricula, and to the desired levels of proficiency as expected by the ‘National’ Curricula; How are we going ? current status of performance comparable to appropriate (similar sub-) groups, and to expected performance; Where to next? directions related to future teaching, learning, curriculum innovations, and system policies. Visible Learning is about Feedback 52

What is Visible Learning ?
Critical Infrastructure for the Knowledge Economy 53 53

A language for progression 54

Empowerment of Teachers in collaborative ways 55

Welcome screen for Teachers
56

Customize a test 57

Choose Curriculum Strands
58

Choose difficulty

Paper and Pencil Administered On screen Administered
Computer adaptive On screen 61

Create a test

Welcome screen for Teachers
63 63

So the key message is They know where they are going, how they are going, and know where to next Go back to the US study ,…. 64

Individual Learning Pathways
65

Group Learning Pathway
66

Curriculum Level Report
67

Progress Report

What Next Report Diagnostic advancement and appropriately targeted online references 69

Student Welcome Screen
70

Student Test 71

Other research options
Computer adaptive sequential testing Interactive video for assessing reading efficacy Target Setting Integrated with school student management system Evaluating teachers’ skill in estimating difficulty Progression maps in curricula The Human Computer Interactions – mode effects Linear programming Open-ended scoring combining technology and teachers How students read screens, effects of scrolling Available (free and voluntary) for all NZ schools Used in elementary & secondary schools If teachers talk about the technology, we have failed If teachers talk to each other about teaching, we have succeeded

Identify major influences on achievement: Feedback, Challenge, and Learning Strategies All effects are relative to the .40 typical value Outline an assessment model based on these influences: An emphasis of feedback to the teacher To demonstrate the power and joy of Educational Psychology 73

John Hattie Visible Learning Lab University of Auckland New Zealand EARLI 2007 74

Developing Potentials for Learning: Evidence, assessment, and progress

Similar presentations

Presentation on theme: "Developing Potentials for Learning: Evidence, assessment, and progress"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Developing Potentials for Learning: Evidence, assessment, and progress

Similar presentations

Presentation on theme: "Developing Potentials for Learning: Evidence, assessment, and progress"— Presentation transcript:

Similar presentations

About project

Feedback