Presentation is loading. Please wait.

Presentation is loading. Please wait.

Www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 1 Consider the Evidence Evidence-driven.

Similar presentations


Presentation on theme: "Www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 1 Consider the Evidence Evidence-driven."— Presentation transcript:

1 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 1 Consider the Evidence Evidence-driven decision making for secondary schools A resource to assist schools to review their use of data and other evidence 6 Getting to Information

2 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 2 Evidence-driven decision making This module is part of a resource about how we use data and other evidence to improve teaching, learning and student achievement Today we are looking at the second stage of this process – collecting and analysing evidence, and getting to information we can use

3 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 3 Evidence Any facts, circumstances or perceptions that can be used as an input for an analysis or decision how classes are compiled, how classes are allocated to teachers, test results, teachers’ observations, attendance data, portfolios of work, student opinions … Data are one form of evidence

4 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 4 Data Known facts or measurements, probably expressed in some systematic or symbolic way (eg as numbers) assessment results, gender, attendance, ethnicity … Data are one form of evidence

5 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 5 What evidence does a school have? Demographics Student achievement Perceptions School processes Other practice

6 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 6 The evidence-driven decision making cycle Explore Check data and evidence to explore the issue Question Clarify the issue and ask a question Assemble Decide what data and evidence might be useful Analyse data and evidence Trigger Data indicate a possible issue that could impact on student achievement Evaluate the impact on the intervention Intervene Plan an action aimed at improving student achievement Interpret Insights that answer your question Speculate A teacher has a hunch about a problem or a possible action Act Carry out the intervention Reflect on what has been learned, how practice will change

7 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 7 The evidence-driven decision making cycle. Explore data Survey of students shows that this is only partially true Question What are the characteristics of students who are poor at writing ? Assemble more data & other evidence: asTTle reading, homework, extracurric, Attendance, etc Analyse NQF/NCEA results by standard Trigger Significant numbers not achieving well in writing Analyse non NQF/NCEA data and evidence Intervene Create multiple opportunities for writing; include topics that can use sport as context; connect speaking and writing. PD for staff. Interpret information Poor writers likely to play sport, speak well, read less, do little HW A teacher has a hunch - poor writers might spend little time on homework Evaluate Has writing improved? Reflect How will we teach writing in the future?

8 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 8 Evidence-driven decision making Getting to information Trigger Clues found in data, hunches ExploreIs there really an issue? Question What do you want to know? AssembleGet all useful evidence together Analyse Process data and other evidence Interpret What information do you have? Intervene Design and carry out action Evaluate What was the impact? Reflect What will we change?

9 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 9 The evidence-driven decision making cycle Trigger Clues found in data, hunches ExploreIs there really an issue? Question What do you want to know? >AssembleGet all useful evidence together Analyse Process data and other evidence Interpret What information do you have? Intervene Design and carry out action Evaluate What was the impact? Reflect What will we change?

10 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 10 Assembling the evidence We want to know if our senior students are doing better in one area of NCEA Biology than another. So … we need NCEA results for our cohort. It could be that all Biology students do better in this area than others. So … we also need data about national differences across the two areas.

11 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 11 Are our data any good? A school found that a set of asTTle scores indicated that almost all students were achieving at lower levels than earlier in the year. Then they discovered that the first test had been conducted in the morning, but the later test was in the afternoon and soon after the students had sat a two-hour exam.

12 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 12 Think critically about data Was the assessment that created this data assessing exactly what we are looking for? Was the assessment set at an appropriate level for this group of students? Was the assessment properly administered? Are we comparing data for matched groups?

13 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 13 Cautionary tale 1 You want to look at changes in a cohort’s asTTle writing levels over 12 months. Was the assessment conducted at the same time both years? Was it administered under the same conditions? Has there been high turnover in the cohort? If so, will it be valid to compare results?

14 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 14 Cautionary tale 2 You have data that show two classes have comparable mathematics ability. But end-of-year assessments show one class achieved far better than the other. What could have caused this? Was the original data flawed? How did teaching methods differ? Was the timetable a factor? Did you survey student views? Are the classes comparable in terms of attendance, etc?

15 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 15 The evidence-driven decision making cycle Trigger Clues found in data, hunches ExploreIs there really an issue? Question What do you want to know? AssembleGet all useful evidence together >Analyse Process data and other evidence Interpret What information do you have? Intervene Design and carry out action Evaluate What was the impact? Reflect What will we change?

16 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 16 Analysing data and other evidence Schools need some staff members who are responsible for leading data analysis Schools have access to electronic tools to process data into graphs and tables All teachers do data analysis Data is not an end in itself - it’s one of the many stages along the way to evidence-driven decision making

17 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 17 Types of analysis We can compare achievement data by subject or across subjects for an individual student groups of students whole cohorts The type of analysis we use depends on the question we want to answer

18 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 18 Inter-subject analysis Have my students not achieved a particular history standard because they have poor formal writing skills, rather than poor history knowledge?

19 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 19 Intra-subject analysis What are the areas of strength and weakness in my own teaching of this class?

20 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 20 Longitudinal analysis Are we producing better results over time in year 11 biology?

21 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 21 Basic analysis

22 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 22 Basic analysis Divide the class into three groups on the basis of overall achievement Identify students who are doing so well at level 2 that they could be working at a higher level Find trends for males and females, those who are absent often, or have many detentions Compare this group’s external assessment success rate with the national cohort.

23 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 23 Reading levels – terms 1 and 4.

24 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 24 Making sense of the results Think about significance and confidence How significant are any apparent trends? How much confidence can we have in the information?

25 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 25 Making sense of the results Is this information? Can we act on it? This table shows that reading levels overall were higher in term 4 than in term 1. Scores improved for most students. 20% of students moved into Level 5. But the median score is still 4A.

26 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 26 The evidence-driven decision making cycle Trigger Clues found in data, hunches ExploreIs there really an issue? Question What do you want to know? AssembleGet all useful evidence together Analyse Process data and other evidence >Interpret What information do we have? Intervene Design and carry out action Evaluate What was the impact? Reflect What will we change?

27 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 27 Making sense of information Data becomes information when it is categorised, analysed, summarised and placed in context. Information therefore is data endowed with relevance and purpose. Information is developed into knowledge when it is used to make comparisons, assess consequences, establish connections and engage in dialogue. Knowledge … can be seen as information that comes laden with experience, judgment, intuition and values. Empson (1999) cited in Mason (2003)

28 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 28 Information Knowledge gained from analysing data and making meaning from evidence Information is knowledge (or understanding) that can inform your decisions. How certain you will be about this knowledge depends on a number of factors: where your data came from, how reliable it was, how rigorous your analysis was. So the information you get from analysing data could be a conclusion, a trend, a possibility.

29 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 29 Information Summative information is useful for reporting against targets and as general feedback to teachers. Formative information is information we can act on – it informs decision-making that can improve learning.

30 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 30 Reading levels – terms 1 and 4

31 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 31 Questions to elicit information Did the more able students make significant progress, but not the lower quartile? How have the scores of individual students changed? How many remain on the same level? How much have our teaching approaches contributed to this result? How much of this shift in scores is due to students’ predictable progress? Is there any data that will enable us to compare our students with a national cohort? How does this shift compare with previous Year 9 cohorts?

32 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 32 Words, words, words … Information can … establish, indicate, confirm, reinforce, back up, stress, highlight, state, imply, suggest, hint at, cast doubt on, refute … Does this confirm that …? What does this suggest? What are the implications of …? How confident are we about this conclusion?

33 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 33 Interrogate the information Is this the sort of result we envisaged? If not, why? How does this information compare with the results of other research or the experiences of other schools? Are there other variables that could account for this result? Should we set this information alongside other data or evidence to give us richer information? What new questions arise from this information?

34 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 34 Interrogate the information Does this relate to student achievement - or does it actually tell us something about our teaching practices? Does this information suggest that the school’s strategic goals and targets are realistic and achievable? If not, how should they change, or should we change? Does the information suggest we need to modify programmes or design different programmes? Does the information suggest changes need to be made to school systems?

35 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 35 Interrogate the information What effect is the new 6-day x 50-min period structure having on student engagement levels?

36 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 36 Interrogate the information What effect is the new 6-day x 50-min period structure having on student engagement levels? Do student views align with staff views? Do positive effects outweigh negative effects? Is there justification for reviewing the policy? Does the information imply changes need to be made to teaching practices or techniques? Does the information offer any hint about what sort of changes might work?

37 www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 37 The evidence-driven decision making cycle Trigger Clues found in data, hunches ExploreIs there really an issue? Question What do you want to know? AssembleGet all useful evidence together Analyse Process data and other evidence Interpret What information do you have? Intervene Design and carry out action Evaluate What was the impact? Reflect What will we change?


Download ppt "Www.minedu.govt.nz © New Zealand Ministry of Education 2009 - copying restricted to use by New Zealand education sector. Page 1 Consider the Evidence Evidence-driven."

Similar presentations


Ads by Google