1 of x Standardising and Annotation Copyright © AQA and its licensors. All rights reserved. Version 3.0 Regional Support Network Meetings Autumn 2013

2 of x Copyright © AQA and its licensors. All rights reserved. PLEASE NOTE In order to meet regulatory requirements, this meeting will be recorded.

Internal Standardising In this session we will: consider how TOLS might be used discuss the marking of candidates’ work explore the importance of good annotation consider models of internal standardising 3Copyright © AQA and its licensors. All rights reserved.

TOLS Three candidates in each of the four units. One set of standardising materials does not ‘replace’ another; there is a library of materials. Has to be in the secure area so username / password necessary. Each person completes the appropriate units on-line allowing others to see their marks by a deadline specified by the Head of Department. One person completes the appropriate units, gains access to the commentaries and marks and arranges department standardising when appropriate. Departments focus on particular sections close to when they are marking that task. 4Copyright © AQA and its licensors. All rights reserved.

The Importance of Annotation Work with at least one other person: read the list of paragraphs from candidates’ work which are accompanied by Centre annotation discuss their appropriateness choose one of the complete annotated responses decide your own mark consider the appropriateness of the annotation 5Copyright © AQA and its licensors. All rights reserved.

The Importance of Rigorous Internal Standardising The challenges: finding time lack of centre support ‘experienced’ staff ‘inexperienced’ staff pressure to produce results Why it is important how Moderators operate: ‘samples’, ‘tolerance’ what ‘regression’ is 6Copyright © AQA and its licensors. All rights reserved.

Moderation 7Copyright © AQA and its licensors. All rights reserved. On a sample of 20: Sub-sample10 Stage 215 Stage 3 20 English has 6 separate marks – tolerance for total is 3 If four of the six marks are one mark out, accumulated errancy = 4, so whole submission becomes out of tolerance and the moderator has to move to stage 2. If one piece is four out, the same applies.

Moderation We train moderators by standardising them with additional materials designed to show them issues they will encounter We monitor moderators’ work by checking samples of what they do We ensure that problems in moderation are passed to a senior moderator for second opinion We check speaking and listening records Moderators check S&L records for coverage of tasks and evidence of applied criteria Moderators select separate samples for each component and report back separately 8Copyright © AQA and its licensors. All rights reserved.

Models of Internal Standardising little, and as often as is reasonable focus on key borderlines task-based rather than folder-based use centre materials as well as AQA materials teacher pairing make sure the demands are very clear to school leadership – up to three different units; six separate marks in English, five in English Language; uniquely tight tolerances; uniquely pressurised subject as far as CA is concerned; dealing with bands when the centre wants grades - tackle over and under-rewarding as it is found tackle ineffective task-setting as it is found 9Copyright © AQA and its licensors. All rights reserved.

BandEnglish Language Criteria 5 ‘Sophisticated impressive ’ sophisticated interpretations of texts sophisticated engagement with writers’ ideas and attitudes; sophisticated interpretations using imaginatively selected supporting textual detail sophisticated analysis of aspects of language and structure 4 ‘Assured’ confident and developed interpretations of the text engagement with writers’ ideas and attitudes and provide perceptive interpretations using precisely selected supporting textual detail analysis of aspects of language and structure in convincing detail 3 ‘Clear, consistent’ clear evidence of understanding of significant meanings of the text ability to explain writers’ ideas clearly offering relevant and appropriate supporting textual detail ability to display understanding of features of language and structure supported by relevant and appropriate quotation 2 ‘Some’ 7-12 some awareness of more obvious meanings in the text some awareness of writers’ ideas supported by relevant textual detail some awareness of obvious features of language and structure supported by some relevant textual detail

BandEnglish Reading Criteria 5. ‘Sophisticated, impressive ’ sustained, sophisticated interpretation of the text sustained and sophisticated engagement with writers’ ideas and attitudes. Sophisticated interpretations using imaginatively selected supporting textual detail sophisticated analysis of aspects of language and structure perceptive and mature comment on the significance of the contexts 4. ‘Assured’ sustained and developed interpretations of the text engagement with writers’ ideas and attitudes and provide perceptive interpretations using precisely selected supporting textual detail analysis of aspects of language and structure in convincing detail. convincing connections between the text and its context 3. ‘Clear, consistent’ 7-9 clear evidence of understanding of significant meanings of the text ability to explain writers’ ideas clearly offering relevant and appropriate supporting textual detail ability to display understanding of features of language and structure supported by relevant and appropriate quotation ability to explore the significance of aspects of the contexts matched to textual details 2 ‘ some ’ 4-6 some awareness of more obvious meanings in the text some awareness of writers’ ideas supported by relevant textual detail some awareness of obvious features of language and structure supported by some relevant textual detail some relevant comments about the significance of the contexts

Band ‘sophisticated’ ‘impressive’ sophisticated engagement with writers’ ideas and attitudes sophisticated interpretations using imaginatively selected supporting textual detail sophisticated analysis of aspects of language and structure perceptive and imaginative exploration of points of links and comparisons perceptive and imaginative comment on the significance of the contexts Band ‘confident’ ‘assured’ sustained and developed appreciation of writers’ ideas & attitudes convincing interpretations using precisely selected supporting textual detail analysis of aspects of language and structure in convincing detail. thoughtful consideration of links & comparisons between the texts thoughtful consideration of the significance of the contexts of the texts Band ‘clear’ ‘consistent’ clear/consistent understanding of writers’ ideas with relevant appropriate supporting textual detail clear/consistent understanding of features of language and structure supported by relevant and appropriate quotation clear/consistent understanding of links and comparisons between texts clear/consistent grasp of the significance of some aspects of the contexts Band ‘some’ some familiarity with writers’ ideas supported by a range of textual detail some familiarity with obvious features of language and structure supported by some relevant textual detail some relevant comments about links between texts some relevant comments about the significance of the contexts Band ‘limited’ limited awareness of writers’ ideas using a limited range of textual detail limited awareness of obvious features of language and structure limited awareness of links between texts limited awareness of contexts and very limited comment on their significance