1. EVIDENCE AND DATA FOR LEADERS

2. WHAT IS IN THIS PRESENTATION
New ways to think about evidence
A data use strategy
Pre and post-testing
Gathering evidence via rubrics
Analysing data

3. NEW WAYS TO THINK ABOUT evidence

4. Teacher experience
Research
Evidence
Assessment data

5. DEVELOPMENTAL vs STANDARD MODEL
Assessment occurs after instruction is complete
Teachers don’t question each others’ data or strategies
Teach whole class at once, with a bit of help for the lower kids and a bit of extension for the top kids if possible
Compares students to norms and focus on what students cannot do
Deficit thinking: students must be at a certain year level norm and I must correct all the deficits they have
Developmental model
Assessment is used to improve teaching
Teachers hold each other accountable based on their data and teaching strategies they use
Targeted teaching as much as possible – ideally individually but even 3-5 levels is usually sufficient
Compares students to criteria and focuses on where students are ready to learn
Developmental thinking: assessment tells me where a student is in their development and I teach them from there

6. HIERARCHY OF EVIDENCE Level Whose main responsibility? Student level
Classroom teacher, student
Class level
Classroom teacher
Year level
Head of learning area
School level
Head of learning
System level
Government agency

7. Clinical model of teaching

8. A data use strategy

9. A DATA PROCESS – PRE AND POST TESTING
pre and post test students
Analyse the data
Use the data to:
improve courses
improve teaching
identify teacher effectiveness

10. A DATA PROCESS – using data from existing assignments
get all teachers using skill-based rubrics
get them recording information on spreadsheets that they send to you
collate this information
analyse it to produce empirical progression
use progression to:
improve courses
improve teaching
help students track their own growth
show students what improvement looks like

11. Pre and post-testing

12. Why pre and post-test?
We should be aiming to get a year’s worth of progress for a year’s worth of input
How do we measure a year’s worth of progress?
Effect size of 0.4 or more

13. What to pre and post-test?
historical skills
pick some from the curriculum
Our school has chosen four:
identifying cause and effect
using historical concepts
sequencing events
understanding historical perspectives

14. Ask questions linked to each level

15. Create an item bank

16. How to administer pre and post tests
By hand (painful)
Microsoft Forms
Google Docs + automatic scoring script like “Flubaroo”

20. Gathering evidence via rubrics

21. WHY USE RUBRICS? Students: Parents: know how to get better
Get higher quality feedback on their performance
assessment data used as information rather than as judgement
Parents:
Know what their child can do, not how they compare
Know the next thing their child is ready to learn
Sees more motivated students – especially those at the top and at the bottom

22. WHY USE RUBRICS? Teachers:
more consistent judgements between teachers (easier moderation)
you don’t have to write as many comments
more detailed information for reporting
rewarding professional discussions between teachers
promotes development:
linked to skills not what is “normal”
the teacher knows where a student is ready to learn
can target teaching intervention to use with that student or group of students

23. WRITING QUALITY CRITERIA
Advice
Use student work samples to describe the levels
Use learning taxonomies to help you come up with ideas
e.g. Bloom’s, Dreyfus, Krathwohl, SOLO
Good quality criteria should…
no counts of things right/wrong or pseudo-counts (some, many)
Avoid ambiguous terms like “appropriate”, “suitable”
Describe increasing quality – no procedural steps
Full guidelines:

24. GATHERING DATA USING A RUBRIC3 2 1 1 1

25. RECORDING DATA ON A SPREADSHEET USING A RUBRIC

28. ANALYSING DATA

29. ANALYSING PRE AND POST TEST DATA
use unique student identifier (id number)
use excel function to rule out all students that didn’t do both the pre and the post test
3. calculate effect sizes for:
overall
teacher effect
year level effect

32. WHAT CAN YOU DO WITH THIS DATA?
Which skills are taught well?
Figure out how it was taught and replicate that elsewhere
Which skills are taught badly?
Is this because of how it is taught or its place in the course?
Which teachers taught well?
Figure out what they do that is transferrable and suggest others try that as well
Get other teachers to observe a teacher teaching a skill well
Film a teacher teaching a skill they teach well

33. ANALYSING ASSIGNMENT DATA
Combine year level cohort into one large spreadsheet and complete a Guttman analysis on it
via the attached “Guttman Boss”

36. ANALYSING ASSIGNMENT DATA
Combine year level cohort into one large spreadsheet and complete a Guttman analysis on it
via the attached “Guttman Boss”
Turn this into an empirically-derived learning progression

38. WHAT CAN YOU DO WITH THIS DATA?
Get students to track their own progress
Show students what improvement looks like
Target teaching of new skills at the right level
Design ability based groupings and teaching material

40. BABY STEPS

41. Start persuading people
pre and post-testing
online testing
using evidence to inform teaching
using assessment data formatively not just summatively
skill-based rubrics
writing decent quality criteria in rubrics
Using data

42. Pre and post-testing Rubrics measure just one thing?
begin with just one class / one year level / one subject?
see if other learning areas in your school do the same thing and learn off them?
Rubrics
begin with just one assignment / class / year level / subject?
Critique your existing ones by comparing them with ARC guidelines

43. Data use
Complete a empirical progression for one year level and show it off

44. advice Staff buy-in is extremely important
Leverage early adopters to help persuade the rest
Use Dale Carnegie’s strategies:
Try honestly to see things from the other’s point of view
Let them feel it is their idea
Praise the slightest improvement
Give the other person a fine reputation to live up to

45. support

46. This presentation and associated files: https://lawlesslearning
This presentation and associated files: “Assessment for Teaching” Patrick Griffin (editor)