Fred McBride Alta Systems assessment tomorrow Dublin - 19 October 2006 Informative Assessment in Northern Ireland using Alta Systems

Informative Assessment in Northern Ireland Alta fundamentals CCEA’s evaluation of June 2006 McAlister’s study of Current projects Future Demonstration

Alta fundamentals Alta is a ‘liberal’ information system –it has no pre-defined rules to govern how it should be used, rather it adapts to accommodate any particular set of policies and content. However, its design does enable the use of classroom-based, informative assessment to –inform self-assessment –inform teaching, and –inform judgements over time.

Alta fundamentals It creates pupil records comprising outcomes from both automated assessments and moderated teacher judgements –all subjects offer opportunities for both types of assessment, and all subjects benefit from the use of both types to enable comprehensive assessment. The Alta System is pupil-centric –information moves with the pupil from class to class, school to school.

CCEA-Alta Assessment Package for Mathematics Martin Montgomery Business Manager CCEA

Background Project began in year period of continuous development and trialling –Curriculum driven –Teacher driven –Learner driven Continuous improvement – bipartisan effort Maturing application Reached operational capability Business case

Criteria for approval of any assessment package 1.Support and enhance teacher assessment 2.Engage and develop the learner 3.Aligned with the Northern Ireland Curriculum 4.Promote best practice in the teaching and assessment of mathematics 5.Produce valid and reliable outcomes 6.Provide easily accessible and useful information

Criteria for approval of any assessment package 7.Meet statutory requirements 8.User friendly for teachers and pupils 9.Reduce the assessment workload 10.ICT based with secure 24/7 access 11.Rigorously evaluated 12.Value for money

Rigorously evaluated Internally Evaluated annually by CCEA Research & Statistics Most recently is June 2006 (150 teachers) Externally Harries and Barmby - Durham University McAlister – Stranmillis College Elsewhere Scotland – SQA, D & G, West Lothian England – UCLES and RM Isle of Man Findings Consistently very positive Issues become part of the development programme

Summary of findings - June 2006 the training was appropriate (98%); the system easy to use (92%); manageability for teachers and pupils very acceptable (94%); levels of engagement, motivation and enjoyment were all high (average 95%); that the system promoted formative assessment (95%) and supported teaching and learning (98%); the system developed mathematical skills (91%) and had a significant impact on the pupils’ ability (79%) while enabling self-assessment (87%).

Formative Assessment using the ALTA system Mary McAlister Stranmillis University College

“ It is by good teaching and learning that standards rise, not by summative or short- term measures to boost attainment – as has been proven by the Black and Wiliam research trawl.” Clarke (2001)

A literature-based theoretical model identified five characteristics for effective formative assessment. For this study, these characteristics were applied to the CCEA-Alta mathematics system to evaluate its formative use by teachers and pupils in Northern Ireland.

The characteristics applied were - sharing assessment criteria and relating these explicitly to learning intentions giving constructive and useful feedback related to Alta information reflecting on Alta assessment data using effective self-assessment strategies to improve performance, and integrating the responses of teachers and their pupils to the Alta system so that pupils know what and how to improve

A further refinement was the introduction of “levels of engagement” as high, mid or low, indicating the depth of interaction of teachers and pupils with the system. The different levels of engagement emerged as the complexity of the formative process became clear. Again, Alta specific indicators were developed.

Summary of Findings CCEA-Alta mathematics offer opportunities for teachers and pupils to use formative assessment in new ways. The potential to use CCEA-Alta maths to promote and support effective, formative assessment has been acknowledged by all of the head teachers, class teachers and pupils interviewed in this study.

Findings For the teachers and pupils involved in this study - there was clear interdependence of engagement, with teachers at high, mid and low levels of engagement being matched exactly to their pupils’ levels of engagement most were engaged at the mid to high level in all five characteristics the highly engaged teachers reduced their administration tasks which helped complement and support their teaching the highly engaged teachers reported that the CCEA- Alta information provided a good basis for parent– teacher discussions and through this they encouraged further successes most reported use of the package had increased motivation

Findings (continued) most reported the package offered new opportunities to develop their ICT skills all found the system simple, intuitive and enjoyable to use most suggested that there was a good variety of questions and response types most found using the system challenging and stimulating the highly engaged teachers and pupils reported that the provision of frequent and short assessments enhanced pupil learning most highly engaged teachers had adjusted their teaching practices to meet individual needs

Commentary on effective use of Alta systems potential benefits engagement class management other factors

Potential benefits assessments linked to curriculum suitable test selectors rich feedback for individual questions, eg replay and for full assessments, eg coloured profiles, drill down supports individual, focused and personalised teaching, learning and assessment results can be interpreted easily and tracked and monitored at an individual, class and school progress level and used to identify strengths and diagnose weaknesses for individuals and groups replay facility supports self-assessment data enable teachers to help pupils to improve performance framework helps pupils develop self-confidence and encourages pupils to take responsibility for their own learning

Engagement integrate Alta assessments into everyday planning and into the wider teaching, learning and assessment plans understand how Alta assessments can be used to judge learning intentions share criteria to prepare for assessments reflect on Alta results at an individual and class level and judge mathematical performance from the overall Alta profiles share feedback from Alta data to identify what and how to improve performance use self–assessment strategies to identify strengths and weaknesses from the Alta results and use replay facility to overcome mistakes and identify misconceptions plan next steps, take action to improve, and return to Alta results to check progress relate plans to the teaching, learning and assessment cycle using Alta information to revise plans in the light of results

Class management teachers and pupils to cooperate closely to maximise the potential benefits of the system teachers to be aware of pupil progress and to respond promptly to results after assessments so that strengths can be identified and weaknesses diagnosed quickly prompt action by teachers to plan remediation and so increase chance of improvements pupils to take responsibility for checking their own results and to be taught to identify their own strengths and weaknesses pupils to check errors initially through the replay facility and then through discussion with the teacher to identify any problems that need to be rectified before moving on organising computers so that the teacher can be available to support pupils if required and give feedback as soon as possible to individuals and class

Notes on deployment of computers computers in the classroom is the best solution enabling pupils to work individually and to be assessed when appropriate for them exclusive access to a computer suite to use Alta at fixed times tends to undermine spontaneity, the continuous nature of the formative process and the focus on individual progress however flexible access to a computer suite can be useful as part of a support programme for formative assessment but the main work needs to be classroom based

Other factors more classroom-based computers train and support teachers to –recognise the potential benefits of the formative use of Alta –deploy effective formative assessment strategies to maximise the achievable benefits –engage with the system at a high level more help to link learning intentions and teaching plans to the success criteria and the assessments provided by the Alta system criteria should be written in a more child friendly way more direct and explicit links should be made between the individual end-of-test results, displayed as ticks and crosses, and the full colour coded profiles which give a pictorial display of achievement mobilise those primary school managers who are interested and knowledgeable about formative assessment processes to introduce Alta and to disseminate information and support good practice in the formative use of Alta in their schools and in cluster groups

Current projects Northern Ireland –work closely with CCEA, and –solution is deployed in 200+ schools Scotland –work with SQA, and –pilots in 6 LAs involving 30+ schools Isle of Man –pilot in 6 schools

Future ?

Demonstration