1 Managing InCAS Assessments: Information Seminar for Assessment Coordinators May/June 2010

2 Learning Intentions Assessment Coordinators will understand: Current information from the Department of Education regarding InCAS and reporting to parents; The 5 key processes involved in managing InCAS Assessments; The effective use of InCAS data; InCAS software developments and the updated CEM website; InCAS evaluation findings; and CCEA help and support. Will have the opportunity to: Complete data analysis activities Share best practice.

3 Overview of the Day amInCAS in Northern Ireland amCoffee Break amManaging InCAS - the key processes pmLunch 1.30 pmManaging InCAS (contd.) 2.30 pmInCAS evaluation & developments CCEA help & support 3.00 pmReporting to Parents

4 Introduction to InCAS Interactive Computerised Assessment System: is a computer-based diagnostic assessment which is adaptive focuses on the pupil as an individual selects the initial question based on the pupils age presents further questions in response to the pupils answers Ref: Information for Teachers-2009 p.9

5 InCAS Assessments Statutory InCAS Assessments – 1.Reading 1.1Word Recognition 1.2 Word Decoding 1.3Comprehension 2.General Maths –Number 1 (counting, informal arithmetic, partitioning & place value, fractions and decimals) –Number 2 (sorting, patterns, formal arithmetic, problem solving and algebra) –Data (Handling Data) –MSS (Measures, Shape and Space) Ref: Information for Teachers-2009 p.13

6 InCAS Assessments Optional InCAS Assessments – Spelling Mental Arithmetic (consists of addition, subtraction, multiplication and division modules) Developed Ability (consists of picture vocabulary and non-verbal ability modules) Attitudes (consists of Attitude to Reading, Maths and to School – it produces a score on a scale from 1-5 / negative to positive) Ref: Information for Teachers-2009 p.13

7 InCAS Assessments Irish Medium Assessments – The Reading (Irish Medium) and Spelling (Irish Medium) assessments have been developed specifically and standardised with Irish Medium pupils. The General Maths (Irish Medium) and Mental Arithmetic (Irish Medium) assessments are translated and have been standardised for all pupils. All assessments available except Developed Ability Ref: Information for Teachers-2009 p.13

8 InCAS Assessments in N.I Increase % Reading 881 Schools Pupils 880 Schools Pupils 886 Schools Pupils272 General Maths 888 Schools Pupils 889 Schools Pupils 886 Schools Pupils265 Developed Ability 119 Schools 3037 Pupils 234 Schools Pupils 345 Schools Pupils917 Mental Arithmetic 290 Schools 7248 Pupils 346 Schools Pupils 345 Schools Pupils358 Spelling 450 Schools Pupils 443 Schools Pupils 399 Schools Pupils194 Attitudes 127 Schools 2500 Pupils 174 Schools 7843 Pupils 227 Schools Pupils686

9 General Maths Assessment Example RRWRRWWWW 8.39 yrs 7yrs 10m Marina 5.36 yrs General Maths Age 6yrs 10m WWRWWRW WRRRWRRRRRWWW WRRRWWRRWW 5.89 yrs7.79 yrs5.89 yrs 7.98 yrs6.06 yrs 7.78 yrs5.59 yrs

10 Quality Assurance Mechanisms CCEA Checks with CEM Ensures InCAS question banks are appropriate to NI Pupils and relevant to Curriculum. Question items mapped to the CCEA draft levels of progression. InCAS assessments are standardised for Northern Ireland Pupils. Trialling of new questions will be separated from formal use of the assessments from Test reliability and validity of the assessments reviewed every year. Conducts evaluation with schools/teachers to determine whether their InCAS outcomes match their professional judgement. Regular meetings with CEM re developments to software and website.

11 Quality Assurance Mechanisms Developments to the software/website agreed based on data analysis and CCEAs evaluations with schools. CEM has to provide their risk register and quality assurance procedures for each new version of the software. Queries from Schools CCEA investigates all queries from schools in relation to pupil outcomes. If CCEA is unable to resolve a query this is referred to CEM for explanation of outcomes.

12 DE Information Circular Number: 2009/14 Subject: InCAS 30 th September 2009 DE website

13 Purpose of InCAS Assessments InCAS is a diagnostic assessment tool designed to support schools in identifying pupils strengths and areas where they are having difficulties InCAS assessments are not for any purpose related to the transfer of pupils from primary to post-primary school. InCAS outcomes … not collected centrally. As with all data, schools will need to ensure that they have appropriate arrangements in place for its safe storage. Paragraphs 4,5,7, DE Circular 2009/14

/11 school year Information for parents Written age-equivalent 2010/11 InCAS outcome for Reading and General Maths, set alongside the pupils chronological age; Written age-equivalent InCAS outcomes for all other available years for Reading and General Maths, set alongside the pupils chronological age; All available information should also be provided in a graphical format, where two years of InCAS outcomes are available; Irish Medium Reading reporting requirements same as for other InCAS outcomes;

15 Parent meeting..for schools to set the InCAS outcomes in an educational context as a basis for explaining to parents their childs strengths and any difficulties … and the actions the school intends to take… …help parents understand what they can do to support their childs learning. Para. 12 DE Circular 2009/14

16 InCAS Exemptions Pupils assessed as having severe learning difficulties. Pupils exempted from any of the requirements of the Education (Northern Ireland) Order New Irish Medium Primary School or unit does not have to carry out the InCAS in the school year in which it first becomes grant-aided. Para. 17 DE Circular 2009/14

Year 4 -7 Pupils A parent meeting will take place by the end of the autumn term. Parents will receive an Annual Report before the end of the school year. InCAS computer- based diagnostic assessment and outcomes reported to parents in writing (autumn term). This includes pupils with Moderate Learning Difficulties (MLD) Ref: Information for Teachers-2009 p.7

18 Session 2 Managing InCAS Key Processes

19 Using InCAS: Key processes Creating the Assessments Completing the Assessments Sending the Assessments for Marking Accessing and Understanding Feedback Using the Information

20 InCAS Challenges Activity Individually identify 3 challenges to completing the 5 key processes associated with the InCAS Assessments. Discuss these with colleagues at your table and compile a list of key challenges.

21 School Preparation Booklet Organize a meeting with relevant staff early in the Autumn term Nominate one person to have responsibility for conducting the Preparation of the InCAS software Consider whether you intend to use InCAS with Year groups outside of 4, 5, 6 & 7 Consider use of Non-statutory InCAS assessments Consider the computer resources in your school Ensure that InCAS software is installed on the required computers

22 Information for Parents Suggestions: Emphasise the diagnostic support for pupils learning purpose of InCAS assessments Show Inky Explain sample graphical data Explain any changes to assessments Outline any trialling of items/new assessments Distribute Parent Leaflet

23 Key Processes CREATING THE ASSESSMENTS

24 Preparation of the InCAS software Moving Data from SIMS into InCAS: Part 1: Working in SIMS.net Part 2:Working in InCAS

25 Admin Menu Ref: Information for Teachers-2009 p.15

26 Creating Assessments Edit/View Biographical Data (p.17) Edit a Pupils Details (p.18) Add a New Pupil (p.18) Create an Assessment: Entire Class (p.19) Create an Assessment: Individual Pupil/s (p.20) Access/Print Passwords: All Pupils Using InCAS (p.21) Ref: Information for Teachers-2009

27 Key Processes COMPLETING THE ASSESSMENTS

28 Completing the Assessments Build the InCAS assessments into your timetables; Take account of when you intend to give the age- equivalent outcomes in writing to parents and hold parent meetings; Ensure all relevant staff are familiar with: –the September 2009 Information for Teachers manual; and –CEM Remediation Guidance (Literacy & Numeracy documents).

29 Initial Teacher Preparation Ensure that pupils: are familiar with the computer/laptop that they will be using for the assessments, especially if they will be using a touchpad understand the assessments they will be completing and have had an opportunity to view the Inky Pupil Demo to help prepare for the assessments available from have sufficient time to complete the assessment/s. Once an assessment is started it cannot be paused have pencil and paper to complete the General Maths assessment Ref: Information for Teachers-2009 p.22

30 Teacher Preparation Summary Assessment Preparation Teachers should check that: they have the password list for the Class all pupils are listed and their details are correct wireless laptops (if used) are within range of the wireless network connection laptop batteries are sufficiently charged the sound is working on all the computers/laptops. there are a sufficient number of working headphones and that these are plugged in firmlyalso ensuring the sound is not set too loud. Ref: Information for Teachers-2009 p.22

31 Teacher Preparation Summary Remember pupils should not: be given help to answer the questions during the assessments use calculators, number lines etc. during maths assessments use pencil and paper for the Mental Arithmetic assessment leave the computer before completing the assessment repeat assessments Ref: Information for Teachers-2009 p.22

32 Key Processes SENDING THE ASSESSMENTS FOR MARKING

33 Assessment Marking As each pupil completes an InCAS assessment their data will be saved automatically onto your school system For ease, schools should ideally wait until all pupils have finished InCAS before sending assessments for marking. Once your assessments have been uploaded, your data will be analysed and charts and tables produced. It is recommended that you allow hours for this to occur. You will need your schools unique User Name and Password to complete this process. Ref: Information for Teachers-2009 p.22

34 Key Processes ACCESSING AND UNDERSTANDING INCAS FEEDBACK

35 Types of Feedback Ref: Information for Teachers-2009 p.27

36 Understanding Data Reading Assessment: Minimum age equivalent outcome is 4 years. If a pupils combined Word Recognition and Word Decoding outcomes are low they will not be given a Comprehension passage - in this case the Reading Age will be based on their Word Recognition, Word Decoding and the minimum Comprehension outcome. General Maths Assessment: Minimum age equivalent outcome is 3 years. Along with an overall General Maths outcome pupils will receive a breakdown for Number, Measures Shape & Space and Handling Data - if they have answered a sufficient number of questions in a consistent manner. If not - this will be indicated with a # symbol in the feedback. Ref: Information for Teachers-2009 p.28

37 Missing Outcomes Issue: Pupils have completed assessments but their outcomes are not in the feedback. Check that the: Assessments have been sent for marking once they were done; Whole process has been completed e.g. you must transfer and upload the assessments; and The Transfer File with the highest number in brackets has been sent.

38 * Look out for asterisks * You may see an * when a pupil has not completed an assessment You may see an * if a pupil has not been given a Comprehension assessment because they performed poorly in either Word Decoding, Word Recognition or both.

39 * Look out for asterisks * You should never see an asterisk for Reading if there are outcomes for Word Recognition, Word Decoding or Comprehension If a pupil has a Reading outcome you should never see an asterisk for their Word Recognition or Word Decoding If a pupil has an average or high Word Recognition and Word Decoding outcomes you should never see an asterisk under their Comprehension You should never report no Reading outcome to a parent These errors can usually be fixed if you phone the InCAS and Annual Report Helpdesk

40 Standard Feedback Ref: Information for Teachers-2009 p.29

41 Standard Feedback Ref: Information for Teachers-2009 p.29

42 4 Levels of Feedback Available Ref: Information for Teachers-2009 p.30

43 Scores Chart Achievement General Maths Modules Ref: Information for Teachers-2009 p.31

44 Scores Chart Attitudes Attitudes

45 Longitude Chart Ref: Information for Teachers-2009 p.32

46 Pupil Scores Table Ref: Information for Teachers-2009 p.33

47 Class Scores Table Order by :GenMaths Age Equivalent Scores (Yrs:Mths) Achievement Age Name(Yrs:Mths)ReadingGenMaths Kim K9:011:612:2 Joe C9:19:210:4 Sophia R8:118:69:11 Zane C8:98:109:9 Anna M9:17:69:5 Chanelle A8:117:59:0 Molly K9:28:49:0 Alex B8:8<4:08:6 Araminta J8:107:57:11 Ref: Information for Teachers-2009 p.33

48 Class Difference Table Order by :Comprehension Age Difference (Yrs:Mths) Reading Modules AgeWord Name(Yrs:Mths)RecogDecodingComprehSpelling William A9:0+1:5+4:2+3:1+1:8 Hayley T8:10+2:2+5:8+2:9+3:11 Kim K9:0+1:4+5:6+2:3* Zane C8:9+0:10-0:1+0:2+1:4 Joe C9:1-1:3+2:70:0* Emma H8:7+1:10+1:3-0:40:0 Sophia R8:11+0:9-1:5-0:4+0:11 Molly K9:2-0:8-1:3-0:5+0:3 Oliver F9:1-0:10-0:2-0:10+0:2 Araminta J8:10-0:3-1:4-1:2+0:7 Ref: Information for Teachers-2009 p.34

49 Age Comparison Chart Ref: Information for Teachers-2009 p.35

50 Interpreting Age – Comparison Charts Using the example school data from Year 6 on your table: 1.Consider the performance of the class against the (green) reference line for the statutory Reading assessment. 2.Consider the Reading module breakdown for Year 6. 3.Do outcomes from optional assessments provide you with additional information?

51 Standardised Scores Table MentalDeveloped NameReadingGen MothsArithmeticAbility Chanelle A * William A124** 122 Alex B5798* 61 Nuala B81 * * Joe C100112* * Zane C Oliver F93** 95 Emma H * Araminta J * Jake J66<55* * Kim K121132* * Molly K * Anna M * Mikey P6860* * Ref: Information for Teachers-2009 p.37

52 Pupil Progress Chart Ref: Information for Teachers-2009 p.38

53 Key Processes USING THE INFORMATION Making Effective Use of InCAS Data

54

55 Every School a Good School Issues Around Effective Use of Data Self-evaluation is only of benefit if it leads to action on the part of those involved. A further important strand is the availability and effective use of data at: –system, –school and –individual teacher level. Does the school make effective use of data as an evidence base to help evaluate performance, identify areas for improvement, and assist with target-setting and assessment for learning?

56 Using InCAS Outcomes Pupil Level Monitoring individual pupil progress – to ensure every pupil is reaching his or her full potential. Class Level Help teachers plan their teaching during the school year to meet the needs of pupils in their classrooms. School Level Self-evaluation Development planning Target-setting.

57 Activity: Reflecting on Feedback In small groups, please look at the data for pupils Zane C and Alex B. Using the information from all available charts what comments do you have?

58 DATA: Action Planning Which InCAS chart(s) or table(s) could teachers use to identify pupils achievement in preparation for the meeting in the autumn term with parents? Which InCAS chart(s) or table(s) could be used by a class teacher to help identify under-achieving children? Which InCAS chart(s) or table(s) could teachers use as a basis for planning targeted/ differentiated support for individuals and groups of pupils within a class? Which InCAS chart(s) or table(s) could you use to monitor the effectiveness of initiatives and strategies within a particular class? Which InCAS chart(s) or table(s) could you use to inform a review at whole school level to identify trends in performance, areas of under-achievement etc Which InCAS chart(s) or table(s) could be used by senior management to inform discussions with ETI?

59 Best Practice Suggestions Action for curriculum managers Raise awareness of data systems and their potential. Promote training in how to use the software and support on how to use the outcomes and share data with colleagues. Find ways to ensure staff have sufficient time to analyse data at a meaningful level. Encourage sharing of good practice. Encourage the appointment of dedicated coordinators to drive the process of interpretation and action. effective-ways-194

60 Best Practice Suggestions Action for class teachers: Highlight specific weaknesses for individual pupils. Identify weaknesses in topics for the class as a whole. Inform accurate curricular targets for individual pupils. Provide evidence to support decisions as to where best to focus resources and teaching. Tailor teaching to the needs of targeted groups. Identify individuals and groups for additional support. Encourage pupils to take ownership of their learning goals. effective-ways-194

61 Session 3 InCAS Evaluation Findings and Developments

62 Evaluation – a cyclical process With Principals, Teachers, Pupils, Parents, Educational Psychologists, CASS officers … - management, - training - use of data - software Action on Evaluation Recommendations and ongoing software developments In the autumn of 2009: Reading & General Maths assessments; Outcomes reported to parents in writing; Parent meetings offered (Yr 4-7) Action 09/10 InCAS Software available Evaluations

63 Evaluation Reports This presentation will refer to 3 separate evaluations reports which available in full on They are: *The specific reports will be noted in the footer of the relevant slides. TitleDateNumber of respondents Throughout this presentation these respondents will be referred to as: InCAS & Annual Report Evaluation of the Year 4 & 7 Teacher Training facilitated face to face November Face to face teachers Evaluation of the Primary Principal Information Seminars on InCAS and The Annual Report January Principals InCAS & Annual Report Evaluation of the Year 4 & 7 Teacher Training facilitated online February Online teachers

64 Developments to InCAS 2010/11 Assessment Developments: New trial items will be included in software, but do not contribute to assessment outcome. 2 New Comprehensions in English language version are "trial" items in InCAS: –Pupils who have an outcome between 9.0 and 9.55 in first comprehension passage will receive an additional comprehension The Master Key in remaining quiz time. –Pupils who have an outcome above 13.5 in first comprehension passage will receive an additional comprehension Phileas Fogg in remaining quiz time. 7 Picture Vocab and 1 Word Recognition item which are not new, having already been present in the software, have been shifted into the trial group due to insufficient data having been acquired on them for the new standardisations. For detailed information see document Developments to InCAS which is available at

65 Developments to InCAS Irish Medium Developments: An Irish Medium Attitudes assessment is now available. Schools can choose to conduct the English Reading as well as the Reading (Irish Medium) assessments and receive associated feedback for both. Developments/Adaptations for pupils with Special Education Needs: The pale green highlight (comprehension passage) is now a more prominent green as is the yellow highlight for choosing answers. Auditory changes – the boing noise when selecting responses has been toned down.

66 SEN Focus Groups Throughout March 2010 the Research and Statistics Unit in CCEA conducted a series of research events with teachers of pupils who have: Dyslexia; Speech and Language Difficulties; Social Emotional and Behavioural Difficulties Attention Deficit Disorder Aspergers; and Autism.

67 Advice to the DE Department of Education CCEA Questionnaires – June 2010 Focus Groups – March 2010 Dyslexia SEBD ADD Speech and Language Autism Aspergers

68 InCAS and NFER Comparison Objectives: Compare individual pupil standardised outcomes for NFER English and Maths with those generated by the InCAS Reading and General Maths assessments in the autumn term of 2009; and Determine whether differences in outcomes exist, and if so whether the differences are statistically significant. Sample: 16 primary schools providing 2,722 matched pupil results for Maths and 2,681 for English.

69 Initial Findings Similar mean outcomes for all year groups in NFER Maths and InCAS General Maths and Yrs 5 and 7 in NFER English and InCAS Reading. Strong or moderately positive correlation for Yrs 5, 6 and 7 in NFER Maths and InCAS General Maths and Yrs 4, 5 and 7 in NFER English and InCAS Reading. No significant difference in outcomes for Yr 7 in NFER Maths and InCAS General Maths or Yrs 5 and 7 in NFER English and InCAS Reading. SEN status was not a major contributing factor for large differences in outcomes in any year. Please note: These findings are based on a small sample size. More research may be necessary.

70 Comparing InCAS and NFER 16 primary schools took part providing 2,722 matched pupil results for Maths and 2,681 for English. Purpose was to: Compare individual pupil standardised outcomes for NFER English and Maths with those generated by the InCAS Reading and General Maths assessments in the autumn term of 2009; and Determine whether differences in outcomes exist, and if so whether the differences are statistically significant.

71 Initial Findings Overall we have identified a good correlation between NFER and InCAS standardised scores with some variations in outcomes of both assessments. SEN status was not a major contributing factor for large differences in outcomes in any year. If you would like to be considered for participation in any future research of this kind we would be happy to keep a record of your contact details.

72 Session 4 CCEA Help and Support

73 CCEA Support & Resources InCAS and Annual Report Resource Pack InCAS SEN Guidance for Teachers 2010 Parent Leaflet InCAS preparation letter (Principals Letter 10/11) CCEA Helpdesk (telephone support) On-line materials at InCAS and Annual Report online learning opportunity

74 Other Support – CEM InCAS Booklets Available for download from: or Diagnosing and Remediating Literacy Problems: Using InCAS Software Diagnosing and Remediating Mathematica l Difficulties: Using InCAS Software InCAS Feedback Guidance 2010

75 Irish Medium Support Early literacy achievement in Irish-Medium schools and early interventions: Responding to information gained from INCAS and from ÁML (Le Gabrielle Nig Uidhir) Forbairt na Litearthachta agus straitéisí luath- idirghabhála i scoileanna lán-Ghaeilge: Ag freagairt do shonraí INCAS agus ÁML Recommended Strategies for Remediation of Reading Difficulties (Irish Medium) Edited by Seán Mac Corraidh

76 Session 5 Reporting to Parents

77 DE Information Circular Number: 2009/15 PUPIL REPORTING REGULATIONS DE website

78 Pupil Reporting Regulations 2009 The 2009 Regulations set out the requirements for reporting by schools to parents and : (a) the information to be provided annually; (b) additional information to be provided at the end of key stages 1, 2 and 3; and (c) additional information to be provided at the end of key stage 2; at the end of key stage 4, and at the end of sixth form. Para.3 DE Circular DE Circular 2009/15

79 Annual Reporting For pupils in Foundation Stage to Key Stage 4 schools are required to report annually to parents by 30th June. Report format is not prescribed in Regulations but must provide clear and meaningful information for parents about their childs progress. If a pupil in any year has been exempted from any part of the curriculum or assessment arrangements the Annual Report should indicate this. Irish-Medium schools can report to parents in Irish or English.

80 Annual Report Content Schools are required to provide parents with an assessment of their childs progress in relation to: Communication (taking account of their achievement in the Language & Literacy area of learning); Using Mathematics (taking account of their achievement in the Mathematics and Numeracy area of learning): and

81 Annual Report Content Brief particulars of the childs achievement in: Each of the areas of learning relevant to pupils in the key stage concerned; Other skills (thinking skills and personal capabilities); Using ICT; Religious education (optional); The pupils interests and strengths; Focus for development. Para. 7 DE Circular DE Circular 2009/15

82 Annual Report to Parents Essential Content: School Name Statutory minimum content Pupil accreditation: achievement of a qualification, award or certificate, including a credit towards a qualification. Fixed/Specific headings Signed and dated by Principal

83 Annual Report to Parents Flexible: Order of Headings Reporting Other Skills within Areas of Learning Optional Content can be re-named or removed Box sizes Font style and size Length of report School Logo Outcomes e.g. class tests

84 Annual Report Options Annual Reports may be: word-processed; completed using Profiles 7 / Assessment software; or Handwritten. Report against the statutory headings using your own comments based on your professional judgement.

85 DE Information Circular Number: 2010/06 Transitional Assessment Arrangements from 2009/10 29 th April 2010 DE website

86 The new Order The Education (Levels of Progression for Key Stages 1, 2 and 3) (Transitional) Order (Northern Ireland) 2010 (S.R No. 135) came into operation on 30 April 2010 Purpose Provides in law for a transitional period – until new Levels of Progression are introduced – the present assessment arrangements would continue. In providing for the current level descriptions to be used for assessment purposes the Order refers to these as Levels of Progression.

87 End of Key Stage assessment and reporting to parents Present assessment arrangements at end of Key Stage 1 and 2 continue against the existing levels in Language and Literacy (English and /or Irish as appropriate) and Mathematics and Numeracy (Para 1). Schools report to parents for each pupil the: Level achieved and the expected level; % of pupils in the final year of that Key Stage in the school attaining: –each level; –expected level or above; –working towards the level; and, –exempted from assessment in that cross-curricular skill. Where a child has been exempted from any part of the assessment arrangements this information must be provided to parents (Para 12).

88 New Levels of Progression The Department of Education states that they will not introduce the new Levels of Progression until schools are provided with the training, support and guidance which they need. It is their intention that they will be introduced during the 2011/12 school year at the earliest and that those for Using ICT a year later than those for Communication and Using Mathematics. For more information

89 Evaluation Please complete the evaluation. This is important as your comments are valued and will inform future improvements. Thank you