Revisiting Target Setting: A Complex Task Judy M. Parr University of Auckland EHSAS Seminars October 2008

Target Setting Target setting should be multi-focussed and student achievement targets differentiated Targets should relate to the elements in the cycle for enquiry Targets should relate to your theory for improvement Targets should help you to see the nature of the relationships that influence achievement (diagnose where the links need to be strengthened )

The Linked Cycle Inquiry and knowledge-building cycles for professional learning of leaders and teachers (Timperley, 2008) –What educational outcomes are valued for our students and how are our students doing in relation to those? –What knowledge and skills do we as teachers need to enable our students to bridge the gap between current understandings and valued outcomes?

The Linked Cycle (cont’d) –How can we as leaders promote the learning of our teachers to bridge the gap for our students –Engagement of students in new learning & teaching practices –Engagement of teachers in further learning to deepen professional knowledge and refine skills –Impact of changed actions on our students

Theory for Improvement As a group, we (hopefully) have some shared theory of improvement May draw on research findings like fact student achievement related to: –Clear emphasis on academic & learning goals –Goals embedded in school routines –Systematic monitoring of student progress –Teachers using evidence of learning formatively to adjust practice –(Leadership BES, Black, Wiliam et al),

Theory for Improvement Targets need to allow you to examine elements of your theory –if your theory says that teacher’s ability to work with data impacts student achievement, then have goals, criteria and monitor relating to this – If your theory includes teacher practice (specific features of it) impacting then have goals, criteria and monitor practice –If your theory says leadership is important, then have goals, targets and monitor

Overview: Theme 1 Goals of a different order: Cycle of enquiry goals- goals for factors related to or impacting on longer term student achievement goal Group discussion

Overview: Theme 2 Differentiating goals for student achievement –Recap: Student achievement target setting –Setting differentiated targets Group discussion Conclusion: Further Issues

Goals of a Different Order related Contributing factor related Contributing factor related Contributing factor

Goals of a Different Order These concern variables shown to have causal link to ultimate goal; evidence about them helps provide explanations –e.g. student attendance, amount instructional time; specific teacher practices; amount reading; teacher PCK; leadership practices

Examples of Factors Predictive of Student Achievement –Teacher knowledge –Teaching practice –Organisational/ leadership –Student attendance/ attitudes etc Student achievement Classroom practice Teacher knowledge

Using Evidence: Student Achievement Analysing and using evidence of student achievement requires knowledge (including PCK) and skill –Theory of task (knowledge of domain) in order to select/ develop tool –Knowledge of tool (appropriateness & limitations) –Knowledge of basic statistics -interpreting –PCK to apply findings (Parr & Timperley, 2008)

Teacher Knowledge Goals These could relate to content knowledge in writing –E.g. Teacher understanding of asTTle writing criterion statements Could relate to knowledge of comprehension strategies –E.g. Teacher ability to recognise comprehension skill or strategy in different questions in tests from asTTle: Reading

Teacher Knowledge Goals Teacher ability to analyse data –Understand key concepts (e.g. mean, SD, range, distribution etc) –To extrapolate major trends from data Teacher ability to ‘use’ data- articulate & demonstrate application to practice –In grouping, identifying & planning specific foci, individual & small group teaching

Teacher Knowledge Goal Operationalised: Example Content knowledge of writing: Demonstrated in terms of consistency and accuracy in scoring scripts using asTTle –Target = In moderation, obtain 70% agreement with ‘expert’ (agreement is exact or adjacent sub-level but not across levels) in all dimensions

Teacher Knowledge Goal Operationalised: Example Ability to interpret data: Demonstrated though teachers/ leaders writing a summary of assessment results from their class/ year level/ school. Target = all summaries meet following criteria Accurate Include major points specific, not generalised Reference a point to specific data Use terminology appropriately

Teaching Practice Generic indicators of effective practice (e.g. instructional time in curriculum area; differentiation) Specific indicators – achievement related feedback in curriculum area –Explicit teaching of self-regulatory strategies in writing

Setting Targets to Find Out About Teacher Practice Targets need to relate to specific aspects of practice that research shows are effective in raising achievement (and relate to needs of current students) “Increased collaboration to strengthen professional community” –Research shows PLC per se does not relate to enhanced teacher learning or student achievement. Specific features may.

Setting Targets to Find Out About Teacher Practice Practices need to be able to be described/ specified in a way that can be “observed”. e.g. what might “improve writing teaching in more depth” look like? Systematic observation necessary (self- report inaccurate)

Examples of Specific Practices Enhanced practice in in terms of –Achievement related feedback during literacy learning –Communicating expectations of what counts as successful achievement of learning goal –Activating links between learning goals, expectations and prior knowledge Observe with an instrument that has ordinal categories with descriptors

Example of Categorical Descriptors: Feedback 1. Teacher’s feedback is not directly related to achievement - rather it is approving, rewarding, and / or disapproving of behaviour 2. Teacher’s feedback refers in a general manner to: - learning goal / expectations regarding successful achievement  - generic aspects of literacy learning 

Categories re Feedback 3. Teacher tells the learner about whether and / or how their work has met / not met: - learning goal / expectations regarding successful achievement  - generic aspects of literacy learning 

Categories re Feedback 4. Learner and teacher discuss (with learner ‘taking the lead’) whether and / or how their work has met / not met: - learning goal / expectations regarding successful achievement  - generic aspects of literacy learning  Could set targets with respect to number of teachers demonstrating levels 3 & 4

Indicators of Student Learning Research also suggests certain student behaviours & attitudes that relate to achievement e.g. –self efficacy (e.g. asTTle writing study) –ability to self regulate (LitPD example) –attending (body and mind!) –Motivation (Guthrie in reading) –Time spent- reading (Allington) Be aware that amount variance accounted for may be quite small

Targets for Proximal Indicators Reading mileage: Time target differentiated by year –(logs, organisation, materials, incentives) Self regulatory behaviours: Proportion of students target –X% of students will be able to articulate (gather data by asking a sample of students questions) e.g. what learning; how will know if learnt; how are going in relation to learning goal; what need to get better at; how will go about this

Issues in Operationalisation Practicality: e.g. 100% of teachers will be using high levels of descriptive formative practice (C & D category – most D- in typology of Tunstall and Gibbs). Finding an appropriate measurement tool Having a reference point for target Attributing influence

Issues in Operationalising Targets Method of obtaining data. Who provides data? Self=report? Cross checks –“Survey of principals to report degree to which believe cluster operating as network of PLC’s” –“Survey of teachers on degree to which have improved understanding and repertoire” Not necessarily a relationship between reported skill/ confidence and actual ability to do it

Group Discussion 20 mins Do you have goals/ targets that are NOT student achievement ones? Would there be an advantage in working out some “different order” goals? What might these concern (given your theory for improvement?)

Student Achievement Targets How to decide what is focus (area for improvement)? –contributes to strategic vision, wider goal, priorities –evidence for focus What / who to focus on? Considerations –Significance, specificity, pragmatics, acceptance, challenging but achievable

Student Achievement Targets How to express target: In terms of: –standards (criteria) e.g. all or X% will meet specified criteria (criterion statements) –standard (level) e.g. X% will be at or above national norms (in asTTle mean total score or curriculum level) –progress rate (against expected) e.g. All year levels will make average progress beyond expected gain (total scores in asTTle: Writing) – improvement (beyond chance/ error)

Student Achievement Targets To decide if target appropriate consider –Baseline data –Levels achieved or rate of improvement nationally (perhaps initially in clusters/ schools like yours) –Intensity of focus including PD –Current staffing

Differentiated Targets for Student Achievement Like success criteria, targets should be differentiated Generalised targets like 60% above mean ignore sizeable numbers students Generalised targets for progress ignore fact that potential for progress may be different at different parts of distribution Targets that increase growth rate each year may ignore the exponential factor (or limits of certain tools)

Setting Differentiated Targets Ensure these best meet your goals. If goal to raise tail following OK –“90% will be reading at CA. 10% tail group will move a minimum of 1 chronological year” Consider setting by class or year level To make it differentiated, need to stipulate – e.g. bottom 20% will progress at twice national average rate –Of 40% currently in stanines 1-3, 30% will move out of them.

Issues in Operationalising Targets Stipulating/ having a definition of both variable and target: –Critical thinking ability will improve Having a reference point for goal –E.g. Where did % come from in “Yr 3, 80% at stanine 5 by Oct; 20% at 8 and 9”? Tool to measure outcomes –does it relate to focus? –Is it a “sound” tool?

Group Discussion: 20 mins Discuss potential for differentiated targets for student achievement in relation to your current focus What might these targets look like? Be clear what reference point and rationale is for each sub goal

Conclusion: Further Issues Where national norms based on appropriately obtained sample of what students can do, do not accord with theoretically derived curriculum levels for years Where focus concerns an area with no developed tools- expertise required Intensive nature of examining classroom practice goals

Comments? Questions?