Presentation on theme: "Teachers use of data to support student learning"— Presentation transcript:
1 Teachers use of data to support student learning Patrick GriffinAssessment Research CentreMelbourne Graduate School of Education
2 Agenda for the sessionTimeFocus9.00The team approach to the use of data9.15Teacher collaboration9.30Use of data9.45Team leadership10.15Changing the culture10.45Morning Tea11.10Scaling up and sustainability12.30Plenary - questions and comments
3 The aetiology of a team approach CEO(M) review of testsReview of test dataLinking tests to a common empirical continuumUse an existing PLT structure in 20 schoolsThe importance of team leadershipFocus on intervention and dataObservation and documentation of what workedSituating in theoryRasch, Vygotsky and GlaserTiered accountabilityEvidence not inferenceChallenge not share
4 Progressive achievement School level data showed growth but this was not surprising given the effects of maturation and normal expected progress of students.Example of typical school’s data -The horizontal axis represents the eight levels in the reading comprehension scale.The vertical axis represents the percentage of students at each level. The two superimposed bar charts represent the assessments at October in each of 2005 and The shift to the right is interpreted as growth.Was it just maturation? It might be, but if it was, it was astonishing and uniform maturation across the 19 schools.Based on large scale studies using item response analyses there is solid evidence of a substantial shift in reading comprehension development.In national, state and international studies the general gain per year level is one half logit per school year. This includes general gains on the AIM test used in this project when it is used in a state level cohort testing. The average gain per school in this study was approximately 1 to 1.5 levels or 1.5 logits – three times the normal expected gain.But the average gain is not the only way to describe the shift. Consider the lowest group. They have moved upwards by 2 levels – four times the expected growth! Less growth is evident at the upper levels, but this could be because of the limits of the measurement instruments. In any pre-test post-test design, as used in this program, students who are lower initial performers will appear to grow or improve more than those students at higher initial performance levels. This effect, known as “regression-to-the-mean”, will occur irrespective of any intervention. So while the gains in the lower levels are impressive, some might be attributed to maturation, some to regression and some to practice effect due to the retesting procedures.4
5 Progressive achievement School B: Three testing periods October - ,March - and OctoberAnother analysis of student data over three testing periods indicated that students had made progress as measured across the developmental progression.Not only had the cohort moved up the scale but that the spread had not increased. This suggested that that all students were developing and that the ‘tail’ of the distribution was not being left behind or remaining static.It was also clear that a year’s expected gain (about one half a logit) was exceeded many times over by groups, but there were also individual students who appeared to regress. Teachers set about specific intervention with those individuals but always emphasised the readiness issue in determining the what and how of teaching and learning. This investigation is on going.5
8 Teachers using data make better decisions What did we learn?Student outcomes are a function of teacher attitudes, skills and knowledge!!!!Teachers using data make better decisionsTeachers collaborating are more effective than working solo!!!Structured approaches to collaboration are more effective than ad hoc approaches!!!Schools providing support and infrastructure are more effective!!!Leadership needs to be strong and focused on learning outcomes.Differentiated and targeted instruction is more effective than whole class teaching!!
11 Linking assessments to developmental learning Reading and Number The assessmentsLinking assessments to developmental learningReading and NumberSWANS instrumentssocial skillsemotional self-management and cognitive development,communication and literacyMonitoring and promoting student development
15 Monitoring and accountability Team ProceduresMeetings?TimeFundingLeadershipSizeStructureMonitoring and accountability
16 Team log records and accountability Student Code: ………………………Level: ……………………….Review date: ………………..Is the student’s level what was expected? What makes you say that?What goals are set for this student’s learning?What teaching strategies could be used to achieve the goals?What resources are needed?What evidence would show the goals are met?Where is s/he?Where does he/she need to go next?(progress/consolidate?How will s/he get there?How will we know?What are the implications across the curriculum?
18 What might be the implications for your network schools? DiscussionWhat might be the implications for your network schools?How might teams be structured in your network schools?Who are the first contact points – how would it be initiated?
20 Harvard approach for school leadership Data wise
21 Available assessment tools PROGRESS tests - mainstreamVCAA Reading and Number – VELS 2 to 5Student completion ~ 60 minsTwice a year to monitor growthSWANS schedulesfor students not able to respond to Progress TestsEmotional and Cognitive,Interpersonal, -Communication and LiteracyTeacher completion onlineNeed to monitor teacher activities (LDF, e5? and PLT logs, PND?)
22 Monitoring with the Progress Tests for teachers
23 Test selection: Progress Tests Progress Test VELS 4.5 – 5.0Progress Test VELS 4.0 – 4.5Progress Test VELS 3.5 – 4.0Progress Test VELS 3.0 – 3.5Progress Test VELS 2.5 – 3.0Progress Test VELS 2.0 – 2.51221SWANS
24 Monitoring Comprehension Development Progress tests
29 Professional Learning Team log Student Code: ………………………Level: ……………………….Review date: ………………..Is the student’s level what was expected? What makes you say that?What goals are set for this student’s learning?What teaching strategies could be used to achieve the goals?What resources are needed?What evidence would show the goals are met?Where is s/he?Where does he/she need to go next?(progress/consolidate?How will s/he get there?How will we know?What are the implications across the curriculum?
30 How to improve teachers’ capacity to use data? Developmental models emphasise all students’ growth?Developing collaborative decision making?Professional development of the team members?
33 The analysis and interpretation cycle We can envisage the process as a cycle. At each of the meetings of the team leaders our first task will always be a focus on the overall project and the data in its aggregate form. Our research team will report on analyses and interpretations in each of the three domains and illustrate how the instruments are performing and how each developmental progression can be interpreted.Each team leader will be asked to report on their team progress and procedures from the previous period of time between professional learning team meetings. This will be an important sharing process in which we will learn a great deal from the good work of the team leaders and the way in which they have helped the PLT members understand the data and use it to benefit the pupils. At this point we will need to ask each other why certain things were done and what was our evidence that we used to justify the procedure. Our whole approach is one that must be evidence based and we will emphasise that in the leaders meetings to create the expectation that the leaders will replicate this in the schools within the PLTsAfter we examine the aggregate data and we hear from the schools we’ll will distribute the software and examine the way in which the data has developed and captured the progress and gains, or losses, of individual pupils. We will have a look at a mockup of this sort of procedure during this workshop.In our team meetings the leaders will mostly form working groups. Members of the groups will discuss and interpret their school and individual pupil data in such a way that we will begin the initial sharing of potential strategies and learning possibilities for each pupil recorded in the data. This discussion should develop over time into a very sophisticated discussion of teaching and learning resource allocation and goal setting for pupils depicted in the data. The implications of the data and the position of pupils on the developmental continuum should provide a great deal of information for planning new approaches to teaching and learning.It should also enable team leaders working with their peers to plan their professional learning team meeting that will follow the presentation and interpretation of data. This will be an important planning session to form the basis of discussion of what they are about to do back in school and to make notes of this intention. We will provide record sheets for this discussion so that there is a record of the interpretation and planning that is intended for the professional learning team meetings.When the team leader returns to school they will need to organise their professional learning team meeting following an agenda that they have discussed with their peers. Their team will review the software, the data, and discuss amongst themselves the potential interpretation using the range of graphs and charts and ready reckoners. The discussion in the PLT should also develop over time into a sophisticated examination of data and the link between the data and the teaching and learning interventions. One of the roles of the team leaders will be to guide the development of the discussion and to assist in steering the discussion to put a link between evidence and decision-making. During this project our research team will make no attempt to discuss how the children should be taught. We will not make recommendations about specific teaching strategies nor will we give advice about appropriate resources or time that might be devoted to specific cases. Our task is to steer the discussion to the link between data used as evidence and decision-making in terms of learning and teaching. We will focus only on the use of data to inform decision-making. At all times we regard the team leaders as the experts and the professional learning team members as the professional practitioners. We do not make any claims in that area at all.We do expect the team leader to prepare a report on the conduct of the professional learning team meetings and the decisions and resources that were recommended as a result of the examination of the data. This will certainly become the generic and powerful evidence-based decision-making that can be disseminated particularly when these data are linked to successful growth amongst the pupils.
45 Promote a focus on teaching and learning Communication The team leader’s rolePromote a focus on teaching and learningCommunicationFocus on evidence not inference.Link data to developmental learning.Accountability to school leadershipAccountability to other team leaders.Professional development of team membersReplace sharing with challengeChanging the culture
46 How would be the criteria for their selection? DiscussionWho can be the leaders?How would be the criteria for their selection?What are the prior conditions for successful leadership?What support would the leaders need?What infrastructure is needed in the school?
48 Team differencesThe data collection can take place in a number of ways. The ideal procedure would be for each pupil to be discussed in turn by all members of the professional learning team. The least ideal is for each teacher to work solo and complete each questionnaire on each of the pupil without any discussion amongst their colleagues. They would both take time and the solo activity possibly takes less coordination and leadership by the team leader. In between these two extremes there are modifications that may provide greater of the quality that we seek. The team leader will need to provide advice to the project team regarding the manner in which the instruments were completed, while we cant control this process, and we are aware of the time demands, we will want to know what process was used. Its an important data quality check.
49 Tiered Peer Accountability “My class” to “our students” PLT focusTiered Peer Accountability“My class” to “our students”Collaboration and joint ownershipEvidence not inferenceSet expectations for all studentsDevelopment not deficit modelsTeach to the construct not the testChallenge - not share
50 Assessment is for teaching Evidence not inference mantrasDo say make writeAssessment is for teachingEvidence not inferenceFormal and informal assessmentsTalk about students – not teachersChallenge and defend not share
53 Purpose of the assessment To inform teaching…identifying the level of student development- To promote student developmentestablish baseline measures against which to evaluate change over timeExplain process of how individual forms of the Progress Tests are scored in order to identify difficulty levels of items; that items are then (back) analysed / subjected to a skills audit in order to identify bundles of items that appear to sample specific skills; that these bundles are then arranged along a continuum that demonstrates a growing sophistication in terms of the skills set; and that students’ scores are then plotted along this continuum to show each student’s zone of proximal development.
54 Changing the cultureWhat needs to be done?How- and by whom?
56 SUSTAINABILITYContact from the network at the start of each year re plans for involvementProfessional reading and research updates prompted by team leaderEfficient online assessment, analysis and reporting systemsRevisit the developmental progressions to check currency and validityUpdates on work in other schools and networksData and intervention discussions at start each yearFormalise the assessment schedule for two data collections per yearAvoid ‘watering down’ the processDocument PLT proceduresAccess to Web site for information - ‘Ultranet’Publish the list of schools involved with their contact detailsList the experienced PLT leadersMaintain the action research led by PLT members and leadersEmphasise focused and targeted teachingMaintain leaders network meetings
57 SUCCESSION PLANNINGNetwork with experienced PLT leadersMaintain the action research records across schoolsEmphasise focused teaching in PLT meetingsDecide on the network PLT leaders group – regular per term? Who to convene? funding?Have a deputy or proxy at cross school meetings for succession planningProject folder on the server at school with up to date information and materialsEnsure that project materials are available as a resource bank (Ultranet?)Have a network induction plan for new leadersFind ways of developing PLT procedure skills at a school level
58 Maintain contact across schools NETWORKINGMaintain contact across schoolsRegular and scheduled team leaders meetings as part of the school’s PD programMaintain contact with experienced team leadersMeeting of the leaders group – regular per term? Who to convene? funding?Structure Leaders meetings with peer reporting dutiesFormalise partnerships and links across schoolsCross school reporting on strategies and resources
59 Dependence on team leaders The weaknessesDependence on team leadersFocus on data only and ignoring interventionLack of accountability within and between teamsTeaching to the testNeed for whole of school support
68 Involvement Making Time, Improving access to data Supporting the work and promoting collaborationLeadership Team , External specialists
69 Systemic support Targeted Professional development E5 and its applications within teamsLeadership Development frameworkP&D frameworkThe Ultranet as a resourceOnline reporting and analysisCoachesTeaching and learningLiteracyNumeracyUltranetNet books and it initiatives for the studentsSSSP for the SWANS materialsRegenerationSchool and community partnershipsEarned autonomyRole for the Institute of Educational leadership for team leaders
70 ActionHow can these infrastructure elements be used to improve data driven, evidence – based teaching and learning decisions?What is in place to support data driven learning and teaching in your network?What support is needed?
72 Identify the Team leader Form the teams IT administrator TimelineimmediateIdentify the Team leaderForm the teamsIT administratorCheck that On Demand Testing is set up and teachers know how to use itSchool AdministrationAllow for time and leadershipAdmin staff need to know what is happeningCoordinators and information networks in the school
73 TimelineTesting Period – Two week periodStudents sit VCAA Progress testsTeachers complete SWANS onlineSchool IT Administrator to upload resultsAfter a further two weeksPrint reports for Team Leader MeetingTeam Leader MeetingOngoingAt end of testing period upload the analyseDiscuss at the PLT meetingTeam leader to document and defend decisions for Leaders’ meetingOnline support