1. Unit 5: Measuring what we value The Year of the Curriculum What are we trying to achieve? How shall we organise learning? How shall we evaluate success? How do we make it happen? How shall we know if we are successful? Module 3 The programme consists of four modules, each with two units: © Curriculum Foundation1

2. Welcome to Unit 5. Did you do your homework? It was to look at the piece of learning you designed for Unit 3, or another piece of learning you are preparing for this term, and see how far it can take advantage of the opportunities presented by your locality. Was it possible? Do you want to share it? You can do so at YOC@NUT.org.uk. YOC@NUT.org.uk © Curriculum Foundation2

3. 3 Unit 5 Measuring what we value: skills, understanding and personal development In this unit we shall look at: 1.Some technical stuff about assessment and testing 2.How we can assess across our wider aspirations 3.Various approaches to assessment 4.Assessing skills 5.A balanced scorecard 6.References and answers

4. © Curriculum Foundation4 Do you remember these images from Unit 4? And do you remember what they were about? We shall be looking now at how these ideas impact on assessment.

5. © Curriculum Foundation5 If you have a long memory, you may recall this exercise from Unit 1: We said that people all around the world share the same sort of aspirations for education. These things are what we value – but do we assess them?

6. . As usual, we shall be looking at what some people have written about this. Do you recognise any of these people? Some are the usual suspects © Curriculum Foundation6 All will be revealed as you read on.

7. © Curriculum Foundation7 Part 1 Some technical stuff. (But not too heavy – so don’t worry!) Part 1 Some technical stuff. (But not too heavy – so don’t worry!) But first, some questions …..

8. © Curriculum Foundation8 What’s the difference between assessment and evaluation? What is the difference between assessment and testing? And what about examinations? What’s the difference between “validity” and “reliability”? Do we always measure something when we assess it? How many different types of assessment are there? Write a list – don’t just click through to the answers!!

9. © Curriculum Foundation9 Have you written down the answers? Don’t forget that this is a Unit about assessment – this is a test! You can either go to Part 6 here to see all the answers, or just keep reading and the answers will appear as we get to them in context.here But first we need to think about assessment and evaluation

10. © Curriculum Foundation10 Assessment concerns finding out what or how much a person has learned. Evaluation concerns finding out how effective a system is at delivering its goals. In some books, especially from the USA, you will find these terms used interchangeably. In French there is only one word for both. However the distinction is useful. In this Unit we shall be looking at assessment. In the next Unit we shall look at evaluation.

11. © Curriculum Foundation11 How many different type of assessment could you think of? Did you think that the question required a number, or a list of what the different types are? (It shows how careful we must be in framing our questions!) Which sort of answer did you give? See how many of these you wrote down. (You did write them down, didn’t you??) FormativeSummativeIpsative Diagnostic Initial Standards- based Performance- based Norm- referenced Criterion- referenced Multiple choice Open- response Formal & informal Self- assessment Best fit Quantitative Qualitative

12. © Curriculum Foundation12 You could place these into five different groups according to: Purpose Type of question The learning being assessed Referencing Setting These groups are set out in Part 6. That seems to make sixteen different types. Did you get them all? So, what do they all mean? If you go to Section 5, you will find definitions for all of these – but, again, we shall be explaining them as we go along.

13. © Curriculum Foundation13 Patrick Terenzini ( ‘Assessment with open eyes’ 1989) looked at it in a different way, and put forward a taxonomy of these types in the following model. His three dimensions are: purpose, object and level.

14. © Curriculum Foundation14 None of these mention ‘Teacher Assessment’. In this Unit we shall be focusing particularly on the assessments made by teachers as part of their work, and not on the technicalities of setting examination papers, constructing standardised tests or compiling item-banks – each of which is huge subject by itself. Teacher assessments are usually carried out for formative reasons. That is, they are used to help the teacher guide learning, give support where it is needed and plan the next steps in learning. There can also be a summative purpose when teachers make end of year assessments and fill in reports. Formative assessment is sometimes referred to as ‘Assessment for Learning’ (see over). The assessment specialist Dr Mark Zelman points out that when a cook tastes the soup, that’s formative. When the guests taste the soup, that’s summative. (That’s one picture explained!)

15. © Curriculum Foundation15 Classroom-based teacher assessments are often qualitative rather than quantitative because judgments are being made rather than test scores being added up. However, some teachers mark students’ work with a score such as 9/10 or B+. This is a form of quantitative assessment. The value of teacher assessment is that is can be based on intimate knowledge of a learner’s performance and so can identify where a student is struggling or has misunderstandings. This sort of assessment is diagnostic or evaluative. Teacher assessment is most effective where is it used to give guidance to the student about how they can improve. This is where a qualitative approach is more effective than a quantitative one. Being given 7 out of 10 for your scotch egg in cookery (this actually happened!) doesn’t really help you make a better one next time. This formative evaluative assessment needs to become embedded in practice. Perhaps the most common form of classroom based assessment is the informal question: “Hands up if you know the capital of France”. Formal assessments such as SATs and GCSEs are much less frequent. Of course, both are valuable in their different ways.

16. © Curriculum Foundation16 And here’s another picture explained. Did you recognise him? It’s Dylan William – a member of the ‘Expert Panel’ whose report laid the basis for the new national curriculum (even if it did not develop quite as they expected!). He is author of ‘Embedded formative assessment’. He also refers to ‘Assessment for Learning’. William suggests that: ‘The evidence that formative assessment is a powerful lever for improving outcomes for learners has been steadily accumulating over the last quarter of a century. Over that time, at least 15 substantial reviews* of research, synthesizing several thousand research studies, have documented the impact of classroom assessment practices on students. The general finding is that across a range of different school subjects, in different countries, and for learners of different ages, the use of formative assessment appears to be associated with considerable improvements in the rate of learning. ‘ * the references are in Section 5

17. © Curriculum Foundation17 Willam also refers to ‘Assessment for Learning’, pointing out that: ‘The term ‘assessment for learning’ is often mistakenly attributed to Rick Stiggins (2002), although Stiggins himself has always attributed the term to authors in the United Kingdom. In fact, the earliest use of this term in this sense appears to be a paper given at the annual conference of the Association for Supervision and Curriculum Development (James, 1992) while three years later, the phrase was used as the title of a book (Sutton, 1995). However, the first use of the term ‘assessment for learning’ in contrast to the term ‘assessment of learning’ appears to be Gipps & Stobart (1997), where these two terms are the titles of the second and first chapters respectively. The distinction was brought to a wider audience by the Assessment Reform Group in 1999 in a guide for policymakers (Broadfoot, Daugherty, Gardner, Gipps, Harlen, James & Stobart, 1999).’

18. © Curriculum Foundation18 You may recall the 2002 Strategy document that made Assessment for Learning (AfL) almost ‘official’ in England.

19. © Curriculum Foundation19 http://www.bbc.co.uk/northernireland/forteachers/curriculum_in_action/assessment_for_lear ning.shtml The significant feature of AfL, as opposed to other forms of formative assessment, was the involvement of the students – or their “meta-cognitive awareness” of the own learning. There is a link to a classroom example below. The significant feature of AfL, as opposed to other forms of formative assessment, was the involvement of the students – or their “meta-cognitive awareness” of the own learning. There is a link to a classroom example below.

20. © Curriculum Foundation20 Here’s another picture named! It’s Prof Lorna Earl of the Ontario Institute for Studies in Education. She also values the role of the student, but has a slightly different take of ‘Assessment AS Learning’ in which she argues that we need to: ‘reinforce the role of formative assessment by emphasising the role of the student, not only as a contributor to assessment and learning programmes, but as the central critical connection between them. Students as active, engaged and critical assessors can make sense of information, relate it to prior knowledge and master the skills involved. This is the regulatory process of meta-cognition. Assessment as learning is the ultimate goal where students are their own best assessors.’

21. © Curriculum Foundation21 But before we get any deeper into this, we need to remember what we are trying to do when we assess. In essence we are trying to find out what someone has learned, either as we go along (formative) or at the end (summative). And the bit of learning that we are interested in is usually to do with the aims we set in our lesson, unit or whole curriculum. Or we might be checking before we start to see whether they have already learned what we are about to teach them (initial assessment). The learning concerned is actually contained inside someone’s head.

22. © Curriculum Foundation22 Any idea what this is? Yes, it’s a picture of the neural networks of the brain. What about this next picture?

23. © Curriculum Foundation23 These are the synapses – where the neural networks join, When we learn something, new connections are made and pathways becomes more complex.

24. © Curriculum Foundation24 So, if learning is about making new connections in the brain, then assessment is about finding out what new connections have been made. Perhaps the process should look like this. (That’s a brain they’re all looking at, by the way. This could be a GCSE exam of the future.)

25. © Curriculum Foundation25 As we cannot actually get inside our pupils’ brains, we just have to do the best we can. Assessment is not an exact science. In order to find out what is going on inside someone’s brain, we have to observe their behaviour in certain controlled situations. At the simplest, we ask them some questions and see what they say. If they say “Paris” when we ask them what is the capital of France, we conclude that they know what the capital of France is. So the big issue for assessment is how valid are the conclusions we can draw from our observations. Simple pieces of knowledge are easy enough, but it starts getting more complex when we get into deeper understanding. Can we really conclude that someone understands something complex from the basis of their answers to some questions? And would everyone respond similarly to the same questions, even if they had equal understanding?

26. © Curriculum Foundation26 This is where reliability and validity come in. Reliability in assessment means that whoever carries it out would come to the same conclusion. Validity means that we are actually assessing the thing we want to assess. Unfortunately, the two are often at the opposite ends of a line of tension. The more reliable an assessment is, the less valid it seems to be. And vice versa. IQ tests are a good example. They are high in reliability, but we are not sure if they are measuring intelligence or the ability to answer IQ test questions – so low in validity. Assessments made by teachers in the course of their work are often the most valid, but there is always a question of subjectivity and whether all teachers would come to the same conclusion. So they are low on reliability. Reliability Validity