TerraNova Score Interpretation Post-Test Data Analysis Webinar Christi Linton

Goals for our Webinar Data Interpretation Understanding the types of information provided by the TerraNova assessment How to identify areas for improvement Utilizing our existing resources Linking Data to Instruction Tips for Communicating TerraNova data to parents

The Purpose of Standardized Achievement Tests Reinforces educational decisions that have a positive impact on individual students and their parents, groups or classes of students, schools and communities Aid teachers and administrators in improving instructional plans, to keep parents informed about the progress being made by their children, and to inform the community of general progress being made by a school or school system

Instructional Program Planning Curriculum Analysis Administrative Planning and Direction Evaluation of Student Progress Needs Assessment Understanding and Using Test Data The Purpose of Standardized Tests “Assessment should promote learning, not simply measure it.” Jay McTighe

Types of Information Provided by Achievement Tests TerraNova Scores: – Accurate Norm-Referenced Information – Relevant Criterion-Referenced Information – Standards-Based Performance Information Inview Scores: – Cognitive Scores

Norm-Referenced Test Scores Norm-referenced – Test questions are primarily selected based on their ability to distinguish specific performance levels matching the “Bell Curve” – The test is given to a nationally representative group of students of all performance levels – “Norms” provide a way to understand how a student’s test performance ranks/compares to similar students nationwide Common Score Types – Grade Equivalent (GE) – Scale Score (SS) – National Percentile (NP) – Normal Curve Equivalent (NCE) – National Stanine

Scale Scores (SS)  Designed to measure student achievement from elementary school through high school  Expected to increase with grade level  TerraNova content forms a developmental continuum through which students move as they develop new capabilities  Scale score shows where the students are on this continuum K Math Scale Scores

National Percentile Scores (NP) Represents the percentage of students in the national norm group whose score was at or below a student’s score A student whose NP is 65 performed better than 65% of those students who took the test when it was normed Important Reminders: – NP scores range from 1 to 99 – The national norm of NP is 50; i.e., 50% of students scored at or below a given student’s score – NP scores are NOT percent correct scores – NP scores SHOULD NOT be added or subtracted because they are not an equal-interval scale score

National Percentile Scale – NP Scale 50 th NP is where national middle is Half the Nation is below 50 Half the Nation is above 50

Normal Curve Equivalent (NCE) Scores NCEs were developed to evaluate student growth NCE scores indicate the relative ranking in comparison to the national group used to establish norm scores on a test NCE scores: – denote relative ranking (1-99) – have an average or norm of 50 – are NOT percent correct scores – can be confused with NP scores – are on an equal interval scale (equal distance between score points) – can be compared and averaged for group comparison or measure growth/gain

National Stanines Standard scores based on a scale of nine equal units that range from a high of 9 to a low of 1 – Stanines 1-3: Below Average – Stanines 4-6: Average – Stanines 7-9: Above Average

Comparing the Scales Source: Page 292 of Teacher’s Guide Q1 Q2 Q3 Q4 Harder to grow Focus on Q3 First NP = 50-75

How are Data Related ? √ Q1 Q2 Q3 Q4 1. Q1 = NP Q2 = NP Q3 = NP Q4 = NP National Quartile Distribution

 If a student is in the 50 th National Percentile each year – he is still growing! K What Does Average Growth Look Like?

Norm-Referenced Data Grade Equivalent Scores (GE) – Created to provide meaningful interpretation First Number represents the grade Second number represents the month 5.3=Fifth-grade, third month – Does not reflect student progression in grade GE scores are rank-order scores only – Advantages Easy to understand – Disadvantages Easily misunderstood and frequently misinterpreted. How would you interpret a GE score of 4.8 obtained by a second grade student on a second grade reading test?

Criterion-Referenced Testing Describes the student’s acquisition of specific skills and areas of knowledge. Describes the student’s acquisition of specific skills and areas of knowledge. TerraNova is organized in terms of commonly taught curriculum objectives TerraNova is organized in terms of commonly taught curriculum objectives Scores based on content knowledge of specific “sub-skills” Scores based on content knowledge of specific “sub-skills” – Score Type: Objective Performance Index (OPI)

Mastery Scores – OPI An Objective Performance Index (OPI) is an estimate of the number of items the student could be expected to answer correctly if he/she had taken 100 items measuring that objective.  Scores based on content knowledge of specific “sub-skills”  Diagnoses strengths/weaknesses High Mastery (Above 75) ( Strong grasp of the concept) Moderate Mastery (50-75) (Moderate grasp of concept; not yet at grade level expectations) Low Mastery (Below 49)

Understanding OPI Scores Position on the Moderate Mastery Range – Farther to the right = approaching Mastery Curriculum and instruction are supporting near grade level performance (on average) Polish/refine curriculum and instruction – Left or middle = needs more work and support Curriculum and/or instruction needs review and possibly revision

What is the line through the circle mean? Confidence band: Takes into consideration the Standard Error of Measurement (SEM) Compensates for variations in performance if the student was to take the test again and again If the student took the test again, it is highly likely that his/her score would be somewhere on this line

OPI Mastery -Grade 5

Performance Level Information Pathway to Proficiency – Level 1: “Starting Out” (Grades 1-2) “Step 1” (Grades 3-12) – Level 2: “Progressing” – Level 3: “Nearing Proficiency” – Level 4: “Proficient” – Level 5: “Advanced” Performance Level Groupings: – Grades 1-2 (Primary) – Grades 3-5 (Elementary) – Grades 6-8 (Middle School) – Grades 9-12 (High School) Scores based on the demonstration of ability compared to a set of standards (Standard Setting: Bookmarking Procedure)

OPI Mastery

Inview Testing (Cognitive Abilities) Based on students’ performance on InView Age-dependent standardized score – Students’ grade levels are not considered CSI considers overall cognitive ability – No subscores per se – Mean (average) score for the group – Range identifies high/low score in the group CSI scale – Mean = 100 – Standard Deviation = 16 – “Average Range” = 84 to 116

Guide to Interpreting the Cognitive Skills Index (CSI) Do not treat CSI as an IQ score – True full scale IQ scores are based on individual tests with multiple subtests administered by a highly qualified person Relate to the Anticipated Achievement scores – Are students working within/above/below their CSI? – Can be useful in identifying students who need more support

Using Inview Anticipated Achievement Scores Look for a difference between the Obtained Normal Curve and the Anticipated Normal Curve – If Positive, students overachieved – If Negative, students underachieved Obtained = 75 Anticipated = 70 Difference = +5 Obtained = 75 Anticipated = 80 Difference = - 5 Look at what type of learner this student is: Verbal? Non-verbal? Instructional Strategy is working for this type of learner

What data do we get from TerraNova 3 and InView? Score TypeMeasure GrowthRankMastery Anticipate Achievement Student 1. Scale Score (SS) 2. Norm Curve Equivalent (NCE) 3. National Stanine (NS) 1. National Percentile Rank (NP) 2. National Percentile Grade/Age (NPG, NPA) 3. Grade Equivalent (GE) 4. Quartiles 5. Local Quartiles 1. Objective Performance Index (OPI) 2. Mastery Category 1. Anticipated Scale Score 2. Anticipated NCE 3. Anticipated NP 4. Anticipated GE 5. Anticipated NS Summary Grade, Class, School 1. Mean Scale Score (MSS) 2. Mean NCE (MNCE) 1. Median National Percentile (MDNP) 2. NP of the Mean Norm Curve Equivalent (NP of the Mean NCE)

Measuring Growth Score TypeDefinitionPurposeNotes SSScale Score 1. Measure Growth over time 2. Used as basis for all Norm Referenced scores (NCE, NP, GE) 1. Equal Interval All students have one 3. SS can be averaged NCE Norm Curve Equivalent 1. Measure Growth over time 2. Compare performance between subtests on same test 3. Comparing growth between tests from different publishers 1. Equal Interval All students have one 3. NCE can be averaged and converted into stanines or percentiles Stanine National Stanine 1. Measures growth 2. Less precise as SS or NCE 1. Equal interval Convert to NCE for more precise growth

Ranking Students Score TypeDefinitionPurposeNotes NP National Percentile Rank 1. Ranks student performance against the national norm (nation). 1. Scale is bell curve – not equal interval 2. Cannot compare NP across subject areas 3. It is not % correct NP of the MNCE National Percentile of the Mean Norm Curve Equivalent 1. Shows where the average student is ranked MDNP Median National Percentile 1. Shows where my middle student is Quartile National Quartile 1. Shows where students are ranked vs. national distribution of 25% in each Quartile 1. Q1 = NP Q2 = NP Q3 = NP Q4 = NP 76-99

What will we do with this data? Examine data to determine areas of strengths and weakness Relate those areas to Standards Determine what to do instructionally to address areas of student weakness in our daily/weekly/monthly instruction – May need to enhance or change your current instructional plan – May need to work with groups of students differently

Action Steps: Analyze strengths and needs by objectives (student, class, or by school) – High Mastery: (Enrichment activities) – Moderate Mastery: (Instruction needed—if majority of class is in moderate and low---need more whole group instruction) – Low Mastery: (Remediation and extra support) Use data to open instructional conversations (parents, teachers, school dialogue) Utilize Resources: – Worksheets/handouts – Teacher’s Guides to TerraNova (item analysis, content descriptions, sample test examples) –

How do I use this information in my classroom? Read the full descriptions of the objectives so you understand the skill that was assessed Use them in conjunction with other data you have about students to focus your classroom instruction on what students do not know and what they cannot do If you teach in a subject area in which the skills are utilized, try to focus on developing the skill through your specialized content

Teacher’s Guide to TerraNova, The Third Edition

Example for Math: “Computation and Numerical Estimation” means students demonstrate that they: – know and use correct computation procedures – solve real-world problems e.g. dealing with money – apply estimation strategies – determine (evaluate) reasonableness of results  How can you help students develop these skills in the context of what you teach? Computation Computation in context Estimation Computation with money Recognize when to estimate Determine reasonableness Estimation with money

Sample Items-Computation and Numerical Estimation

Focus on “Best Practices” for a Standards-Based learning classroom Clearly communicate standards to students and parents Clearly align your assessments and instruction to the standards Use assessment results to adjust/focus your instruction Re-think your grading and other management structures Plan for intervention, for different learning styles, and for transfer of learning

Handout/Resources

Tips for Communicating Data with Parents Our Attitude: Parents….. – Want what is best for their child – Are the child’s first and most important teachers – Are equal partners and collaborate with the school Our Information at a Parent-Teacher Conference: – Clear and straight forward – Supported by samples of classroom work or tests Steps: – (1) Share strengths—positive statements (student’s strengths as a learner) – (2) Share challenges---constructive information (where improvements can be made; why improvement is needed—share work samples and expectations for the grade level – (3) Create action plan---specific next steps: What will the school and teacher do How the parents can help at home What the student can do

Sample Template for a Home Report

Need Additional Help? Utilize your TerraNova Resources: – Norms Book – Technical Manual – Beyond the Numbers booklet – Teacher’s Guide to TerraNova – Scoring Guides – Test Directions for Teachers – Test Coordinator’s Handbook – Classroom Connections to TerraNova – Guidelines for Inclusive Test Administrations – Your local assessment consultant – Thank You! Christi Linton (301)