Presentation on theme: "Assessment – An Integral Part of RTI"— Presentation transcript:
1Assessment – An Integral Part of RTI Weighing cows won’t make ‘em fatter …just as, assessing children will not increase student learning!Sandy’s Slide Introduction:WelcomeThe purpose of this module is to share information about theregulations and the role of assessment in a 3 tier model .“ While assessment won’t increase student learning it can serveto inform us about student instructional needs and helpteachers target instruction to those needs.”Now, we will begin with a review of module 1 .
2Module 1 Model Tier 1 Framework Review Lori and Julia
3Purpose of Training Modules RTI is not a program; RTI is a processState provides a framework and processThrough regulationsThrough professional developmentThrough technical assistanceDistrict/School develops unique implementationRTI implementation must take into consideration unique characteristics of local cultureRTI should build on existing systems/initiativesDistrict LeadershipDiversity in contribution/input/skillsDistribution in workloadEveryone is knowledgeable and supportiveShare framework and provide guidance for school implementationLori and JuliaDistrict Leadership Team – regular meetings and assignments
4? ? Tier 1 ? ? ? All Students, All Staff, ? All Settings, All Year ? Lori and JuliaEvery student always receives Tier I instruction. Tier II and Tier 3instruction will be given in addition to the Tier I instruction.(Be sure to explain that students do not leave Tier 1 when they move to Tier 2 or 3)….Tier 1 is the core standards’ aligned instruction that ALL students receive.It is a common misconception that Tier 2 and 3 students physically “move”out of tier 1 ….They do not.Address questions. Refer to handout of Tier 1 Components and QualityInstruction Example and Non-example.
5Most IMPORTANT RTI Component Why Focus on Tier 1?Federal Law and State Regulations must ensurestudent has received quality instructionneed for intervention is not due to poor/inconsistent/ or lack of core instructionResearch/Previous RTI ImplementationSpend Sufficient time on Tier 1 – Quality Instruction - before implementation of InterventionsMost IMPORTANT RTI ComponentLori and Julia
6RTI Framework Where have we been? Where are we going? Established District Leadership TeamOverview of RTINeeds Assessment of District and School Level ImplementationTier 1 FrameworkWhere are we going?Assessment FrameworkAssessments (December)Data Management (December)Data Analysis (February)Team Problem Solving (February)Interventions (April)SLD Determination (TBD)Secondary Focus ( )Lori and Julia
7Assessment In The 3 Tier Model RTI Module IIDelaware Department of EducationSandy’s slideActual Beginning of the Assessment Section.
8Purposes of Assessment in RTI To inform instructionTo provide early interventionTo monitor progress at the student, class, school, and district levelsTo evaluate instructional programs/strategiesSandy’s slide
9Objectives for the DayT o identify ways to implement a comprehensive assessment planTo develop an understanding of screening, progress monitoring, diagnostic, and outcome assessmentsTo identify the critical components of reading, math, early childhood, and behavior as related to assessmentDiscuss the objectives. Speak to the goal.Districts/schools will leave with an understanding of acomprehensive assessment plan and determine where they arein the process of establishing such a plan -- what they havein place and what they still need to do.Use Anticipation Guide to assess background of informationand promote table discussion.
10Objectives for the Day (Continued) To evaluate current assessment toolsTo review available assessment toolsTo develop an understanding of a data management systemSandy’s Slide
12Questions? Sandy’s Slide We will have a parking lot for questions that you have today. Pleasewrite questions on an index card on your table. The coordinators willCollect them when they see you hold them in the air.We will answer as many questions as we can. If we feel the questionis complex, we will take it back to DOE for additional clarificationand send you a response in the near future.
13It is useless if we do not use it to guide our actions. Assessment is the collection of data to make decisions. (Salvia & Ysseldyke, 1997)It is useless if we do not use it to guide our actions.The terms assessment and evaluation are used interchangeably,but it can be helpful in navigating the implementation of RTI tothink of how these terms are different.If you think of assessment as the process of collectinginformation, it becomes easier to convey to teachers the needfor standardization, reliability, validity, and using differentassessments for different purposes.This leads to thinking about evaluation as the process of usingthe results of data to make decisions (i.e., information collectedthrough assessment). We use assessments to make decisions and toevaluate students’ progress, effectiveness of instruction, and effectiveness ofan intervention.We give assessments to help us evaluate. The assessment data helps us tomodify instruction , curriculum changes, assist in daily planning, and make groupdecisions. Evaluate students’ progress as they go through the tiers.Again, this isn’t just about assessment (or giving tests), it is about USINGthe data to make educationally sound decisions.
14Three Tiered Model Increasing Support ~5% ~15% ~80% of Students Tier III:Students not responding to Tier I or II interventions – Sustained Intensive Small Group & Individual InterventionsPossible Special Education Identification for non-responders of Tier III interventionsThree Tiered Model~5%Tier II:Students not responding to Tier I efforts – group interventions,Specialized Research-based InterventionsIncreasing Support~15%Tier I:Classroom/All Students -Core Class InstructionpossibleSpecial Services** “Remember this is what instruction looks like in the 3 tier model”**Quickly review this slide with participants.If the core is effectively taught with fidelity and the curriculum isdifferentiated with academic and behavioral supports for all students,80% or more of all students will be successful with the stronginstruction provided at the Tier I level.Now we are going to look at curriculum and assessment because not onlydo they work together, but they are essential to ensure that all children learn.~80% of Students
15Assessment in a 3 Tier RTI Model Tier I /Core InstructionUniversal ScreeningProgress MonitoringDiagnosticOutcomeWe want to emphasize once again that Tier 1 is for all studentsall the time for both instruction and assessment.Tier I includes the following assessments:Universal Screening Assessments provide an initial indication of which studentsare entering the school year “at risk” .Additional assessments after the universal screening given in Tier I will depend onthe child’s score on the screening assessment.Progress Monitoring Assessments are given periodically to determine whetherstudents are making adequate progress.Diagnostic Information includes the gathering of information that can helpguide interventions for students who are experiencing difficulty learning corecontent, for example learning to read.Outcome Assessments are given at the end of the year to assess what studentslearned in core content area.
16Assessment in 3 Tier RTI Model According to the RTI regulations, when a student is in Tier II, progressmonitoring occurs every week and a formal diagnostic assessment may begiven to gather more specific information.Assessments in Tier III are primarily the same as those in Tier IIexcept we must dig deeper. Remember, these are students that have notresponded enough to the instruction that we are providing.This would probably require us to do a more formal diagnostic assessmentsuch as the DAR (Diagnostic Assessment of Reading ??)
17Comprehensive Assessment Plan Universal ScreeningProgress MonitoringDiagnostic TestingOutcome TestingData Management System1. A comprehensive assessment plan is a critical element of an effectiveschool-level plan for preventing instructional difficulties across corecontent areas.2. These components should be included in a comprehensiveassessment plan: universal screening, progressmonitoring, diagnostic testing, outcome testing, along with a datamanagement system.3. For interventions to work, we must have a coherent assessment system, onethat employs the various types of assessment in concert.4. Let’s review the Purposes of Assessment:Screen for students who will need additional instructional supportDiagnose students’ instructional needsMonitor progress of students over timeEvaluate outcomes at key points in timeKeep in mind that you must always use multiple sources for assessment inmaking decisions. This will give us a better understanding of a child’s progressor lack of progress.
18Where Should the Assessment Go? UniversalScreeningProgress MonitoringDiagnosticOutcomeOur goal for this activity is for districts will work in their groupsto place their assessments in the most appropriate category on thesorting chart on card stock located on their table.Directions:Participants will use the matrix of assessments in their folders to writethe names of the assessments that they presently use on post it notes.Then the group will decide which category is appropriate for eachassessment. There should be discussion at each table regarding thereasons the assessments should be placed in a particular category.This is really a pre-assessment for you. After we have described each of the fourassessment types, we will ask you to re-evaluate your sort based on newinformation that we have provided regarding the characteristics and purposesof each of the four assessments.
19Which Assessment Term?The assessment should measure the critical skills/components of the subject area it is assessing.The assessment should yield similar scores if students were tested on a different day, by a different tester, or on a minimally different set of items.WHITE BOARD ACTIVITY – Please read the definition on the screen. At your table, discusswhether this is the definition of validity or reliability. Now write the term that you feel is definedby the definition on your white board and hold it up. This is the definition of validity.Why do you need to understand these terms?In order to be most useful in school settings, assessments of any kind must meet certain criteria .Validity and reliability are two of the most essential considerations in the selection of assessments.Let’s address validity first. Of utmost importance is that the assessments measure the critical skills ineach subject area, and this concern addresses validity. To be valid, assessments must measure what they areintended to measure. In the case of reading, it is the 5 major components asidentified by the National Reading Report...phonemic awareness, phonics, fluency, vocabulary andcomprehension. Predictive validity is another very important type of validity .Tests with predicative validityare designed to predict future performance or success. Dibels, in reading, has predictive validity. This is anessential feature to have because we want to know if students are on the track to reading success.Pull in the definition or reliability. Note the definition of reliability. Reliability means that anassessment yields similar, consistent scores each time it is administered. Obviously this is animportant feature. To have reliability, we would expect similar scores if students were tested ona different day, by a different tester, or on a minimally different set of items. Assessments are trulyvaluable when they obtain a reliable score. A reliability co-efficient should be as close to “1” as possible –a co-efficient of .9 is considered reliable. You are more apt to get this with an individually administeredtest. Group administered tests generally have lower reliability co-efficients. Turn to a partner and discusswhy this may be so. (Indiv– more engaging for the child, tester can identify more problems, child producessomething/Grp -efficient) You will need to remember that a good assessment is BOTH valid and reliable.When we think of validity and reliability, most often we are thinking of published assessments.However, included in your packet, is a handout created by Kristin Ritchey of the U. Of DEthat provides the procedures necessary for developing your own assessment tool.It is a complex process that will include designing validity, reliability studies as well asdetermining norms and setting benchmark goals.
20Which Assessment Term?The appraisal of student progress by using materials and procedures directly from the curriculum taught.A simple set of procedures for repeated measurement of student growth toward long range instructional goals.WHITE BOARD ACTIVITY – Let’s use the white boards again. Is this the definition forcurriculum-based assessment or curriculum-based measurement? Talk with your friends andwrite your answer. This is the definition of curriculum-based assessment. The key here is thephrase “directly from the curriculum taught”. In curriculum-based assessments, probes aredeveloped on the books or material that make up the child’s curriculum. Therefore, it is agreat way to see how well a child is performing on the materials that are being used forinstruction. Often the student is assessed across several levels of the curriculum and performancecriteria are established to determined acceptable levels of student mastery.Turn to your partner and try to think of some examples of curriculum-based assessments.(fluency probes from a specific core program; end of the level or theme anthology tests, runningrecords, word recognition tests from the core) PULL IN THE NEXT DEFINITION: This isthe definition of curriculum-based measurement. In contrast this is not based on the curriculumper se, but uses progress monitoring content that is constant from one measurement period tothe next , AND the assessment is based on the critical skills of the content taught. Inreading the essential components are assessed. Examples of CBMs are DIBELS andAims –Web . CBMs can be used for screening and progress monitoring and may providesome diagnostic information as well. You may wonder why you need to know these terms.It is important because you are using CBAs right now. However, CBMs may be new to someof you, and curriculum-based measurement has become a general term that is frequently usedas an umbrella term for progress monitoring tests.
21Which Assessment Term?A specific criterion level of skills specified as an indication of an acceptable proficiency or masteryA test used to determine the overall developmental level of a child with respect to other studentsWhite Board Activity -- Is this the definition of criterion-referenced assessment or anorm-referenced? Let’s discuss and use our white boards again. This defines acriterion–referenced assessment and the give-away is the term “criterion”.In a criterion-referenced test results are compared against an establishedstandard. It is a goal to achieve. Let’s brainstorm some examples of criterion-referencedtests at your table. (DSTP, DIBELS, DE driving test)NOW SHOW THE NEXT DEFINITION: This is the definition of a norm-referencedassessment. Norm-referenced tests compare one child’s performance with what might benormally expected of other children. Hence the word norm. Examples are the Iowa Test ofBasic Skills, Gates-McGinity, and the NAEP. These are tests which give uspercentile ranks, stanines, scaled scores, and normal curve equivalence information. .Now let’s mix it up a little. PULL IN THE TRUE/FALSE QUESTION. Is this a trueor a false statement? Use your whiteboards. Yes, it is true. A criterion-referenced test mayalso be norm-referenced. An example of this is DIBELS.
22Which Assessment Term?An assessment that provides information about student progress in order to make mid- course corrections or improvements to instruction.The final assessment, usually quantitative in practice, of the degree to which the goals and objectives of a program have been attained.White Board Activity: Is this the definition of formative or summative?Use your white boards to answer. Yes, it is the definition of formative. You use itto inform instructional decisions. What are some examples of formativeassessments? Discuss this at your table. (Examples: theme test, weekly spellingtest, phonics inventories, spelling inventories)BRING IN THE SECOND DEFINTION: In contrast to formative, this is thedefinition of a summative or outcome assessment. It “sums” up the degree towhich goals have been attained by a student. Let’s discuss some examples.(DSTP, SATs, final examinations)Look at your assessments on the category card and see if you can distinguishbetween the summative and formative assessments at your table.Why do you need to understand these terms? When choosing assessments, youneed to know your PURPOSE. Is this assessment going to provide me withinformation to help me teach in my needs-based group or is it an achievement testthat tells me what a child has or has not learned, but gives me little informationon how to proceed with instruction. All of these terms are defined for you in yourglossary. Familiarity with these terms and what they mean will probably be helpfulas you develop your assessment plan.
23Tier I: Universal Screening (NASDSE, 2005) Universal Screening (of ALL students) occurs at least three times per year.Beginning, middle, endProcedures must identify which students are:proficient in the target skill,developing the target skill, anddeficient in the target skill.Basic question to be answered:Should student be judged “at risk”?Information on this slide was taken from a report by NASDSE, which is the NationalAssociation of State Directors of Special Education.Regulations – Universal Tier I instruction screening for reading andmathematics shall be conducted at least 3 times each regular school yearat routinely and fairly spaced intervals. The first screening should beconducted within 2 weeks of the beginning of the school year, orwithin 2 weeks of the child’s entry into school because some students will grow over the summer,Some will lose ground, and some children are new to the school.Universal screening is given to all children so that early identification of struggling studentscan be made both academically and behaviorally.The test must have predictive validity – be predictive of reading acquisition andlater reading achievement.-In reading we know that reading trajectories are established early and thatstudents on a low trajectory tend to stay on that trajectory and fall further and further behind.The basic question for a screening measure is whether or not the studentshould be judged as “at risk.”The later children are identified as needing support, the more difficult it is to catch up!Screening measures may be either a criterion-referenced, such as DIBELS which providesinstructional levels, or normative comparison , such as in our regulations below the 40th percentile–Criterion-referenced is preferred because it addresses the critical skills.Universal screening is considered formative data.
24Tier I Universal Screening Criteria Efficient – brief, accurate, inexpensiveGenerally administered individuallyMultiple probesClearly defined procedures for administering and scoringBroad Index – measures the Big IdeasSchool-wideValid and ReliableScreening is the first step in identifying children who may be at riskfor difficulties and needing additional support.These students are then considered for a more in-depth assessment,such as monitoring their progress during the next six weeks and /orcollecting diagnostic information.For a screening measure to be useful, it should satisfy three criteria(Jenkins, 2003)It needs to identify students who require further assessment bymeasuring the Big Ideas. Big Ideas are predictive of reading acquisition andlater reading achievement. They are something we can do something about;something we can teach and something that improves the outcome if taught.It is better to err on the side of over-identifying than under identifying at-riskstudents.2. It needs to be practical – efficient, easy to score, easy to understand, and quick to administer.3. It needs to generate positive outcomes (accurately identifies students withoutconsuming resources that could be put to better use)Criterion measures are preferred because they give more accurateinformation about performance over relevant skills.
25Please take out your Take Away window. The universal screening Progress MonitoringDiagnostic InformationOutcome / SummativeData Management SystemPlease take out your Take Away window. The universal screeningpane is in the top left hand corner. Please take a moment toself reflect about the purpose of universal screening andat least 3 insights that you have gained into this assessment.After you have done your individual reflection, please turn to yourPartner and discuss what you have written.Now as a district, please take a moment to reflect on your currentassessments under universal screening.Relative to the information that you have just heard, what isyour thinking on the universal screening tool you are currently using?Any new insights? Does it meet the criteria just described?
26Progress Monitoring Criteria Measures rate of growth toward an observable, measurable, and targeted goalMeasures small increments of growthHas multiple formsProgress monitoring determines through frequent measurement if students are makingadequate progress or need more intensive support to accelerate learning.The regulations state in TIER I that children who score above the 25th percentile butbelow the 40th percentile shall have progress monitoring toward end of the yearbenchmarks at least once every 2 weeks until progress monitoring consistentlydemonstrates that the child is on a trajectory to meet end of year benchmarks.So as you consider available tools for progress monitoring, you need to keep the abovefeatures (on the slide) in mind.It must measure growth in small increments over time towardsa targeted – benchmark goal.It is generally a curriculum based measurement. ACBM measures the critical components that have been identified as the necessaryelements to achieve proficiency in a content area and which are covered acrossthe school year.Alternate forms or probes which measure the same skills and arecomparable in difficulty must be available so that we are able to administer themrepeatedly.
27Progress Monitoring Criteria (Continued) Is efficientIs individually administeredIs graphed and viewed regularlyIs comparable across studentsProgress monitoring determines through frequent measurement if students are makingadequate progress or need more intensive support to accelerate learning.The regulations state in TIER I that children who score above the 25th percentile butbelow the 40th percentile shall have progress monitoring toward end of the yearbenchmarks at least once every 2 weeks until progress monitoring consistentlydemonstrates that the child is on a trajectory to meet end of year benchmarks.So as you consider available tools for progress monitoring, you need to keep the abovefeatures (on the slide) in mind.It must measure growth in small increments over time towardsa targeted – benchmark goal.It is generally a curriculum based measurement. ACBM measures the critical components that have been identified as the necessaryelements to achieve proficiency in a content area and which are covered acrossthe school year.Alternate forms or probes which measure the same skills and arecomparable in difficulty must be available so that we are able to administer themrepeatedly.
28Grade-Level Progress Monitoring Criteria Kindergarten – letter recognition, phonemic awareness, alphabetic principleFirst Grade –alphabetic principle, oral reading fluencySecond Grade – alphabetic principle, oral reading fluencyThird Grade – oral reading fluencyIn Reading First we use DIBELS as our progress monitoring tool, and it measuresthe five components as identified by the National Reading Panel – phonemicawareness, phonics, fluency, vocabulary and comprehension. The componentswhich are monitored change as the grade levels change. In kindergarten,benchmarks are set for phonemic awareness skills – initial sound fluency istargeted in the fall and winter and phoneme segmentation in spring. Letter namingfluency is also addressed as is word use fluency. In first grade, phonemesegmentation and letter naming continue to be assessed; however, the targetedskill in fall and winter is nonsense word fluency which is the phonetic componentand in the spring, oral reading fluency becomes the targeted skill. Word usefluency is also assessed. In second and third grade, the targeted component isoral reading fluency, even though NWF continues to be assessed in fall ofsecond grade. And, again, word use fluency is assessed. So DIBELS measuresthe appropriate, targeted skills along a developmental continuum to help studentson pace for acquiring proficiency in reading comprehension.
29Trend lineorAim line?We would like to share a DIBELS report but first we need to think about theseterms. We would like for you to use your white board and markers again. As atable decide which of these terms is described by the following definition.A line on a graph which connects the student’s initial data point with thebenchmark or goal point of a targeted skill for a specific student.(Answer: Aim line). Student progress is monitored inrelationship to the aim line.(Answer: Trend line). The trend line indicates the student’s actual progressAgain Show trend line in relationship to aim line (.Is he progressing onthe same trajectory as the aim-line, above the aim-line, below the aimline)
30DIBELS REPORTSUsing a graph for an individual student, a trajectory, aim-lineand trend line will be discussed. The slope of the line willalso be discussed.Describe the end points of the aim line. Describe the green circle and purplesquare. Mention the child is doing well.This child is on the trajectory to meet end of year grade level benchmarks.We assess benchmark students 3 times a year.
31DIBELS REPORTSThe Student Summary report displays Karen’s performance on benchmarkskills across grades and time periods and as well as provides item-level detailsall in a single view. As we can see here, Karen has performed below benchmarkon three occasions, her progress can be seen here along a trajectory toward hergoal of benchmark.Three consecutive data points below the aim line indicates a need for change ininstruction.
32right, read what your table mate has written, Universal ScreeningProgress MonitoringDiagnostic InformationOutcome / SummativeData Management SystemPlease take out your Take Away window. The progress monitoring pane is in thetop right hand corner. Please take a moment to self reflect about the purpose ofprogress monitoring and at least 3 insights that you have gained into thisassessment. After you have done your individual reflection, please leave iton the table. I will ask you to stand and move two to three seats to yourright, read what your table mate has written,and star any thoughts with which you agree, and add one or more of yourthoughts to that person window.– Something that you heard, but perhaps yourtable mate did not.Now as a district, please take a moment to reflect on your current progressmonitoring assessments. relative to the information that you have just heard,What is your thinking on the progress monitoring tool you are currently using?Any new insights?Return to your seat
33Diagnostic Information Knowledge about a child’s skills and abilities that is useful in planning instructionCan be derived from student work, teacher observations, or other tests, as well as diagnostic testsAccording to the regulations, children in Tier 1 who score above the 25th percentile,but not at benchmark on any instructional screening, must receive differentiated,needs- based instruction. This means students must receive targeted instructionin the critical skills at the child’s instructional level.This instruction is very specific, explicit and systematic.The purpose of diagnostic information is to provide information to direct thetargeted instruction in the needs-based groups. We can not assume that a child has comprehensionProblems because he does not score well on a comprehension test. It could be a decoding problem, alanguage problem, or a sight word problem.Any information gathered about the child’s knowledge and skill in the componentsof the subject area is diagnostic information.In reading, this diagnostic information may be gleaned frominformal assessments such as phonics screeners, spelling inventories, or highfrequency word surveys.In contrast to diagnostic information, diagnostic testsare generally lengthier, but also provide information to drive needs-basedinstruction. Remember, important instructional information can come from sourcesother than formal diagnostic tests.Therefore, it is important to consider whether the administration of a diagnostictest will provide additional information for instruction. Instructional timeis more valuable than the time vested in administering a diagnostic test if itsimply duplicates information that we already know or that we can get from an efficientinformal test.There must always be a purpose for assessment.
34Diagnostic Information Universal ScreeningProgress MonitoringDiagnostic InformationOutcome / SummativeData ManagementSystemPlease take out your Take Away window. The Diagnostic Information/Assessment pane is in the lower left hand corner. Please take a moment to selfreflect about the purpose of Diagnostic Information/Assessment and at least3 insights that you have gained into this assessment. After you have done yourindividual reflection, please leave it on the table. I will ask you to stand andmove 2 (3) seats to your left, please read what your table mate has written,star any thoughts with which you agree and add one of your thoughts tothat persons window – something that you heard but perhaps your tablemate did not. Now as a district, please take a moment to reflect onyour current assessments. Relative to the information that you have just heard,what is your current thinking on the Diagnostic Information tool you arecurrently using? Any new insights?RETURN TO YOUR SEATS
35Outcome Assessment Outcome assessments are important because they give school leaders and teachers feedback about the overall effectiveness of their curriculum.Frequently group administeredProvides summative data – gives end of the year information on the child’smastery of critical skills. Does not provide information for ongoing instructionpurposes.These are often “high stakes” test (AYP ratings, accountability)An outcome test or a year-end achievement test might be used to measuregrowth in a broad area.Summative assessment – provides an evaluation of mastery of standards for thepurpose of reporting or accountability.As part of a comprehensive plan, outcome assessments should be administeredevery year from kindergarten through third grade.Longitudinal studies of reading have shown that students are much more likelyto meet grade level standards in reading at the end of third grade if they havemet those standards in each preceding year.Outcome test at the end of grade K-2 are useful to school leaders to ensurethat instruction in each grade is sufficiently powerful to keep most studentson track for successful performance when they take important readingaccountability measures at the end of third grade. (DSTP, achievement test, Iowa Test of BasicSkills.
36Diagnostic Information Universal ScreeningProgress MonitoringDiagnostic InformationOutcome / SummativeData ManagementSystemPlease take out your Take Away window. The Diagnostic Information/Assessment pane is in the lower left hand corner. Please take a moment to selfreflect about the purpose of Diagnostic Information/Assessment and at least3 insights that you have gained into this assessment. After you have done yourindividual reflection, please leave it on the table. I will ask you to stand andmove 2 (3) seats to your left, please read what your table mate has written,star any thoughts with which you agree and add one of your thoughts tothat persons window – something that you heard but perhaps your tablemate did not. Now as a district, please take a moment to reflect onyour current assessments. Relative to the information that you have just heard,what is your current thinking on the Diagnostic Information tool you arecurrently using? Any new insights?RETURN TO YOUR SEATS
37Tier II Weekly Progress Monitoring Diagnostic assessments may need to be considered.In Tier II, the Regulations state that interventions shall be delivered and progress shallbe monitored weekly against established benchmarks. So progressmonitoring is increasing.The Regulations also state that if, after 6 weeks of Tier II intervention, a child has notmade progress toward benchmarks or has made progress,but is not on a trajectory to meet end-of-year benchmarks,an IST shall meet to review the child’s program.The team may decide that the child’s instruction is being adequately modifiedand the trend line is closing the gap on the aim-line on our progress monitoringreport and that the child needs another 6 weeks of instructionOR the team may decide that the instruction is not meeting the child’s needs –the trend line is not closing the gap on the aim-line on the progressmonitoring report. The main purpose of progress monitoring in Tier II is todetermine whether the intervention is successful in helping students learn at anappropriate rate.We may also find that we will need more information – and might consider off grade levelprogress monitoring, or more in-depth information on a broader scalesomething beyond spelling inventories, phonics screeners or word recognitionsurveys that were mentioned earlier and which should have already been given.You might need to give a diagnostic assessment.
38Tier II Diagnostic Assessments Provide in-depth, reliable assessment of component skillsAre relatively lengthyAre given when there is a clear expectation that it will provide new, reliable information about a child’s difficulties to inform more powerful instructionFormal diagnostic assessments as we have mentioned are generally lengthy in natureand assess the component skills in depth. They are used when the more informal diagnosticinformation such as spelling inventories, phonics screeners, HFW surveys etc.does not provide enough information to address the child’s needs. Diagnosticassessments are not meant to be given to every student but rather, only whentruly needed as they take longer to give and therefore, remove the student fromvaluable instruction for a longer period of time. Informal, diagnostic information,such as the specific sound recognition or letter naming, may be teacher createdor from a specific curriculum. Formal diagnostic assessments are generallypublished valid and reliable assessments. There are many reading diagnosticassessments available: DAR,( Diagnostic Assessment of Reading)ERDA (Early Reading Diagnostic Assessment ). PPVT (Peabody Picture Vocabulary Test)Fox in a Box to name a few.Remember we not recommending specific tests to be purchased.Keep in mind that if quality instruction is occurring in TIER I and informal diagnosticinformation is being utilized to inform instruction, the need for a formal diagnosticassessment should lessen.In choosing a diagnostic tool make sure you are getting additional information to informInstruction. Does this tool answer the following question?What information does this diagnostic give me that can be used to informinstruction tomorrow in my small needs based groups?
39Progress Monitoring Diagnostic Testing Tier III According to the regulations, progress monitoring is monitored weekly against theestablished standards in Tier 3 just as in Tier 2. In addition the RTI procedures aredesigned so that the child receives the appropriate level of targeted instruction inaddition to their core instruction in the general classroom setting. Students are receivingmore targeted and intensive interventions in terms of frequency and duration based onthe child’s progress against the benchmark as measured through theweekly progress monitoring.It is more likely that further diagnostic assessment may also be necessary at this level,but only if additional information to target instruction can begleaned from the diagnostic.
40A Comprehensive Plan Universal Screening Progress Monitoring Diagnostic Information and AssessmentsOutcome AssessmentData Management SystemSo lets summarize the characteristics of a comprehensive assessment plan. For interventions tobe successful, a coherent assessment system must be in place.The various types of assessments work together. The results of the assessmentsare used to inform the planning, and this is called “assessment-driven” instruction.If the universal screening does not reveal a problem, core instruction continues.Remember differentiation of instruction is a core component of Tier Iinstruction and is a component of quality instruction that needs to routinelyoccur for all students. If with quality instruction and differentiation ofinstruction, the student is not progressing, then a diagnostic is administered to targetneeds based instruction.These assessments give us information to identify the specific needs to be addressedin needs-based group instruction.As the differentiated instruction proceeds, progress monitoring is continued anda decision is made as to whether additional instruction should continue.The outcome assessments are usually administered to measure important readingoutcomes, such as reading comprehension and provide important informationto the teachers and school leaders about the overall effectiveness of the reading program.They also provide an evaluation of the child’s mastery of standards for thepurpose of reporting or accountability. A data management system allows youto organize and effectively utilize the assessment data and is necessary to effectively utilize data in planningand modifying instruction. With this in mind we would like you torevisit your sorting activity. Think about changes you might want to make.Do you think that you placed your assessments in the correct categories now thatyou are aware of their individual purposes? Are they appropriate for the purpose?
41Taking StockTake out the the Taking Stock of Assessment handout from your folder. Referto the sorting chart that we did before. Please list your reading assessments onhandout. The purpose of today was to show you the components of a goodassessment plan, we have shared with you the importantcharacteristics of each of the 4 different types of assessments.At this point we would like you to actually take stock of the assessments thatyou presently are using. Perhaps they many of the necessary features are alreadyplace.At this time let’s go through an example using something familiar tomany of us DIBELS. Note that DIBELS is available for grades K-5. Can wecircle screening? The answer would be yes , because we use it three times ayear to identify students that are “at risk” while addressing the critical readingskills at each benchmark for each grade level. What else can we circle in thisColumn? Answers: Progress monitoring because it aligns with the screening andhas multiple probes and sensitive to small changes in growth. We could alsoCircle diagnostic because we can get some diagnostic information especially ingrades K-1. In Reading First Schools the federal allows us to use it as an outcomemeasure in addition to the DSTP. Validity and reliability information is availableAt the UN of Oregon website by way of a technical report. Data Management Plan systemfor DIBELS can be found at the UN of Oregon website and Wireless Generation.We have prepared for you some guiding questions to help you evaluate your assessmentto help you determine your assessment needs so that you have a coherent integrated plan.
43RTITransforming Our Vision of How to Increase the Mathematics Proficiency of All Our Children
44Effective Math Assessment Tools How do the characteristics of mathematical proficiency shape the design of effective screening and progress monitoring tools?What are the elements of effective math assessment?
46Mathematical Reasoning Proficiency Adaptive reasoning (Reasoning) –capacity for logical thought, reflection, explanation, and justification.AnalyzeCompare/ContrastMake an InferenceEvaluateClassify* In order to access reasoning the item must be “novel”
47Pilots for Progress Monitoring America’s ChoicemCLASS:MathKeymath3
48Effective Mathematics Assessment Although there is a purpose for assessments that compare the mathematical skills of a student or group of students to others in the district, state, nation, or world, these assessments are not created to facilitate an individual student’s learning.
49Working Inside the Black Box: Assessment for Learning in the Classroom A follow up study to the original article by William and Black
50Effective use of formative assessment has shown a direct correlation to student learning and include the following characteristics:The learning targets are shared clearly with students from the beginning of the learningClassroom assessments accurately measure achievement of the important learning targetsStudents are given continuous, descriptive feedback that includes evidence about what they currently do understand and what they still need to work onStudents are involved in the assessment, record keeping, and communication of learningStudents understand how to close the gap between the goal and where they currently “are” (Black & William, 1988;Stiggins in DuFour et al., 2005)
51Can a Universal Screening Tool be formative in design yet still set benchmarks that help to identify students in need of intervention?
52Vision for the FutureStatewide Universal Screening Tool with Embedded Formative Assessments Identifies and Informs Suggests Interventions Continuous Fine Tuning of our Collective Knowledge
53Identify the Focal Points for each Grade Initial Data Collection: Curriculum BasedFormative AssessmentTasksTeacherInvolvementSelectionof embeddedAssessment TasksIdentify theFocal Pointsfor each GradeBenchmarks ofStudent UnderstandingAndMathematical ProficiencyStudent Work SampleswithSuggested InterventionsUnpacking of embeddedAssessments to createUniversal Screening ToolIdentificationof students belowBenchmarks
54Vision for the Future Phase One Current Assessment Practice Data Quarterly AssessmentsTransfer TasksLead Teacher TeamsGrade Level Representation (Initial Focus Grade K-5)Nomination ProcessBegin identification of key tasks and student work dataState/District Support
56Early Childhood Assessment and Progress Monitoring Verna Thompson
57What happens at Tier 1 ALL young children have access to: Evidence-based curriculum for ALL areas of developmentEffective teaching strategies & learning opportunities for ALL childrenUniversal “probing” of key skills in ALL areasAssessing acquisition of key skills in ALL areasAligned to:
58What Happens at Tier 1Results of progress monitoring of development show:Most children are making progressCurriculum planning is effectiveLearning opportunities are meaningfulMost children are not making sufficient progressModification of teaching strategies for whole classSome children are not making sufficient progress“Early intervening” for individual children
59Recognition and Response Guidelines for Progress Monitoring and Probing Where: In a variety of natural settings & routinesWho: By Informed caregivers (teachers, parents, teams)What: Collection of multi-sources of data in ALL areasCurriculum based assessment aligned with EarlyLearning FoundationsObservationsParent InformationWork SamplesChecklistsWhen: Ongoing assessment of skills in ALL areasPeriodic “Probing” of ALL Key SkillsIdeally same tool is used for Monitoring and ProbingResults inform curriculum, teaching strategies, planned learning opportunities for children
60Guidelines for Selecting Authentic Assessment Tools Authentic Assessment measures should have the following characteristics:Curriculum-basedDesigned to be used multiple timesEasy to scoreSensitive to individual differencesProvide information on both level and rate of growth in key areas of learningRelated to long term learning goals in curriculumAligned to Delaware Early Learning FoundationsIdeally, single measure should be used for authentic assessment and probing for key skills
61Authentic Assessment Tools for early Childhood Curriculum-based tools(aligned to Early Learning Foundations)Creative Curriculum Developmental AssessmentHigh Scope CORCarolina CurriculumChecklists linked to Early Learning Foundations(Collected from multi-sources of information)RORSWork SamplingDevelopmental Checklist Birth to Five
62Early Childhood in Delaware Head Start and Early Childhood Assistance Programs use authentic assessmentEarly Learning Foundations being revisedWill include format for assessing childrenChild Care Licensing Regulations revisedCenters required to assess childrenTraining on authentic assessment is in planningABCD Grant – Public HealthScreening initiative
63ABCD Grant Assuring Better Child Health and Development Collaboration with Medicaid, DPH, DE-AAP, Autism Society of DEDevelopmental Screening for all children at well-visits with a standardized screening toolAges & Stages, PEDS9 mos, 12 mos, 18 mos, 24 mos, 36 mosBeing piloted in two community practices in DEWill be replicated statewide with Medicaid policyPlans include training early childhood providersEffort by the National Association of State Heath Policy
64Recognition and Response Pilot Pilot States presently implementing R&R:ConnecticutArizonaMarylandFlorida
65What Can Delaware School Districts Do Now? Advocate for QUALITY Early Childhood programsDevelop RELATIONSHIPS with families and programs in communityDevelop PARTNERSHIPS with community programsCommunicate with programs in communitySuggest checklists aligned to Early Learning FoundationsProvide training on using assessment checklistsUse AUTHENTIC ASSESSMENT to monitor children’s progressNOT responsible for screening all children from birth in community
66Using Recognition and Response in Kindergarten Kindergarten Screening:Consider Pre Kindergarten experiencesChild was in an environment that provided opportunitiesKindergarten is the first learning opportunityAnalyze screening results
67Activity At your table: Select one area of learning from Delaware Early Learning Foundations2. Observe children in video clip3. Document learning opportunities you observed in that area of learning4. Discuss what you observed with the group
69RTI and Behavior/ Mental Health Do we have to “do RTI” for behavior and social-emotional issues???Does it make any sense to “do RTI” for behavior and social emotional issues?NOTE: on this slide emphasize the “does it make sense” portion … do we have to do it is secondary.Do we have to “do RTI” Short answer: NOWith respect to regulations for special education, there is no requirement for the three tier model to be in place for behavior. So, to be clear, schools are NOT REQUIRED to do anything with respect to identification of behavior issues at present.Of course, children with behavior and social emotional difficulties will be part of the schoolwide academic screenings and routine monitoring that you will be doing. So, you may turn up some kids in need of behavior and social-emotional interventions as you do your academic screenings (e.g., those who find it hard to participate in the routine screenings).RULE OUT for LD … need to rule out behaviorDoes it make sense? Short answer: Yes!As you are developing your three tiered model for academics, it makes sense to consider how behavior fits into your model.Many students who are struggling with behavior will improve as you attend to their academic difficulties through differentiated instruction.Others may need specific attention to behavior issues, in addition to their academic concerns.Still others may be ok academically but have behavioral or social emotional problems.All of these situations can be addressed through a three tiered approach to behavior, fully integrated with your three tiered model for academics.No!Yes!
70RTI and Behavior/ Mental Health Are we already “doing RTI” for behavior and social emotional issues?Short answer: quite possibly! At least in some aspects….Many of you in the audience are from districts who have PBS in your schools. Some are using Responsive Classrooms, and others are using a model (whether it be formal or informal) that attends specifically to teaching children behavioral expectations and social-emotional competencies in a systematic, school-wide manner.Quite Possibly!
71RTI and Behavior/Mental Health Tier 1:Schoolwide teaching of behavioral expectations and social emotional competenciesUsing schoolwide data to identify and address problemsIf your school is already attending to making sure all students know what is expected, the expectations are actively taught and kids’ efforts to meet the expectations are acknowledged (not necessarily with tangible reinforcers), you are already accomplishing some of the goals of the universal or tier one level.This next part is focus of slide:If you go a step further and systematically examine how successful the program is in accomplishing the goal of a safe, supportive school environment in which kids know how to get along with one another, and identifying where the program is less than optimally effective (e.g., in certain locations of the school, at specific times of day or activities, with specific students), then you are already accomplishing the remainder of the tier one goals.You use this information to continue to improve the schoolwide program, provide assistance to teachers who want it, and identify kids who may need more attention.
72Universal Screening and Behavior/ Mental Health What is already happening?Monitoring office discipline referralsTeacher referral to school-based problem solving teamThere are some differences between the academic and behavioral programs that you will be developing.With behavior, you may or may not choose to do routine screening for difficulties (with academics, of course, this is required).But, again, let’s look at what you may already be doing that will help you move quickly to assist kids in need.Office Discipline Referrals (ODRs) can be a source of identifying kids who may be in need of additional supports. As a school you can choose how low or high to set your standard for when a child would be judged in need of more specialized interventionTeacher referral: most schools have some kind of school-based problem solving teams who receive requests for assistance regarding behavior. While we don’t usually think about this as a universal screening, the “teacher as test” model is a good one! Teachers know which kids are demonstrating atypical behavior. However, this method may not be systematic enough to meet your needs.As you are setting up your three tiered model for academics, it is important to create a unified system of problem solving. DON’T have separate teams for behavior and academic problem-solving. If your school is large, several grade level teams as the initial problem solving step for BOTH academics and behavior will make more sense.
73Universal Screening and Behavior/ Mental Health Are we ready to increase our attention to universal screening for behavior and social emotional issues?Turning to things you probably are not doing….The methods discussed next are recommended for consideration ONLY once your schoolwide system (tier one) is firmly in place and functioning well. In addition, you probably will want to wait until your Tier 2 and 3 systems are functioning well for both behavior and academics.Why?Because these methods are about identifying kids who might otherwise “slip through the cracks” (especially kids with internalizing difficulties like depression and anxiety). You don’t want to identify kids before you are ready to provide effective services to them once identified.An exception: if you have a well-developed set of intervention options (perhaps through a wellness center of community-based counseling center), you may want to begin sooner.Maybe!
74Universal Screening and Behavior/ Mental Health Purpose: to identify youth who have high risk for developing behavioral or mental health problemsConducted on a schoolwide basisTypically involves several levels of assessment to avoid over- or under-identification of studentsTime will not permit a detailed discussion of methods today.The Positive Behavior Supports initiative will be conducting a day long workshop in the spring for those interested in learning more.“Multiple Gating Procedures” are common.
75Universal Screening and Behavior/ Mental Health Multiple Gating ProceduresGate 1: teacher nomination procedureGate 2: teacher rating scale procedureGate 3: observation and/or more detailed rating scalesIdentification of students most at-riskIntervention PlanningIn this kind of approach, at Gate 1, teachers receive information about particular risk factors. Often, these are defined as “externalizing” (e.g., conduct problems, hyperactivity) and “internalizing” (e.g., depression, anxiety).Teachers then identify up to 3 students in their classes who exhibit the risk factors just defined. This is gate 1. You can expect an over-identification of at risk children through this procedure (six students per classroom is A LOT but at this stage you are trying not to miss anyone).Immediately following, teachers complete a BRIEF rating scale on the students he/she just nominated. These results are compared to normative data (may be national or local norms) and students scoring in the atypical range are identified. This is gate 2.These first two gates are designed to be accomplished in a single schoolwide meeting in about an hour. Again, there may be children identified who not need interventions (although fewer than at Gate 1), so you move to gate 3.Gate 3 can take a variety of forms but often involves systematic observation and more detailed rating scales (some of which will be familiar to you: Achenbach, BASC, etc.).This procedure is designed to yield identification of the most at-risk kids. As part of Gate 3, you will figure out two things:1) Does this student appear to need intervention?2) If yes, what kind of intervention makes sense? (i.e., develop an intervention plan).
76Universal Screening and Behavior/ Mental Health Self-report proceduresGate 1: schoolwide screeningGate 2: follow up interviewGate 3: diagnostic interviewAt middle and high school levels, especially if you are interested in internalizing behaviors like anxiety and depression, self report is a more accurate identification method than teacher or even parent report.There are a variety of methods available to do this (see handout of resources).
77Universal Screening and Behavior/ Mental Health What does progress monitoring look like?The same for academic progressMore individualized around specific behavioral or social-emotional concernsOnce you have identified kids at risk and are applying interventions, you will be doing progress monitoring specific to those interventions.Keeping our eye on the prize of academic progress, kids with social emotional and behavioral problems stay within the regular academic progress monitoring system, so you will know if academics are improving or deteriorating.Kids with both academic and behavioral/social-emotional issues will also be receiving differentiated instruction, receiving more intensive services within the tiers for academics as needed, but you will also be keeping watch on the effectiveness of interventions targeted specifically toward the behavioral issues in many cases.
78Tools for progress monitoring: Daily Report CardsTools for progress monitoring:php/tbrc/tbrc.phpSee handoutYou can create very child-specific behavior ratings that can be used as progress monitors. These can be combined with chart dog to create graphs.
79Universal Screening and Behavior/ Mental Health Read more about it….Distribute handout
81DATA MANAGEMENT FREEWARE http://www.jimwrightonline.com RTI: Graph Data for Visual Analysis-Charts and graphs transform progress-monitoring data into visual displays.-Time-series graphs in particular are widely used-A positive trend-line demonstrates when the student is responding well to intervention.I. Excel Graphs Made Easy-Download pre-formatted Excel spreadsheets-Enter data and create time-series graphs for common academic measures-Designed by Dr. Jim McDougal and student colleagues Karrie Clark and Jacklyn Wilson from SUNY College at Oswego.II. Generate Time-Series Graphs On-Line-Enter your student data into the on-line application ChartDog-Create time-series graphs-Plot trend linesIII. Paper Charts-Chart your data by hand-Collection of blank time-series charts that you can download and print
82DATA MANAGEMENT COMMERCIAL Existing Grading Program & ExcelSoftwareWeb basedHandheldQualitative DataExamples:(demos)(archived web casts)
83Diagnostic Information Universal ScreeningProgress MonitoringDiagnostic InformationOutcome / SummativeData ManagementSystemPlease take out your Take Away window. The Diagnostic Information/Assessment pane is in the lower left hand corner. Please take a moment to selfreflect about the purpose of Diagnostic Information/Assessment and at least3 insights that you have gained into this assessment. After you have done yourindividual reflection, please leave it on the table. I will ask you to stand andmove 2 (3) seats to your left, please read what your table mate has written,star any thoughts with which you agree and add one of your thoughts tothat persons window – something that you heard but perhaps your tablemate did not. Now as a district, please take a moment to reflect onyour current assessments. Relative to the information that you have just heard,what is your current thinking on the Diagnostic Information tool you arecurrently using? Any new insights?RETURN TO YOUR SEATS
84A Look at the Real World…… Implementation of a ComprehensiveData PlanPresented by Capital School DistrictPam HererraColleen RinkerMichele Waite
85A Comprehensive Framework for Reading Instruction HEADMaking informed decisionsbased on dataComprehensiveAssessmentSystemLiteracy Support Team(LST)Instructional Support Team(IST)Whole Body is Tier 1 instructionHigh quality, effective instruction for all studentsBackbone-Assessment SystemLST/IST Go hand in handHeart-studentsClothes are Tiers 2 and 3 interventions and they change based on needsGrade Level MeetingsGrade Level Planning
86Comprehensive Assessment Matrix for Reading Organization of the assessment systemPurposes for assessmentsTimelines for assessments and schedulesUsing an Assessment TeamImportance of professional development related to administering assessmentsRole of district in implementation1)Show assessment matrixes/ discuss importance of using a variety of assessments that give you the most valuable information so informed decisions can be made about students. Discuss reasons why as we moved to DIBELS that also kept GATES and GRADE. We made small adjustments each year to the plan-as we added all schools to using DIBELS we kept GATES and GRADE because at those levels were the only consistent measure. Will continue to re-evaluate whether or not we will continue. EASE OF IMPLEMENTATION IS THE KEY. Showing the importance of having the data and USING the data to make instructional decisions is INTEGRAL to the success of implementation.WE MUST CHANGE OUR MINDSET-INSTEAD OF VIEWING ASSESSMENT AND INSTRUCTION AS TWO SEPARATE THINGS WE MUST DEMONSTRATE THAT ASSESSMENT IS INSTRUCTION. That is why the measures we select must be curriculum based. This is a very hard perception to change! If don’t find out what our students know or don’t know, we are teaching “a program” not students. Chances are, we will have to re-teach. It just makes more sense to take the time to find out what they know so we can better match instruction to needs.2)Discuss use of an Assessment Team for benchmarks.3)Importance of professional development for using and administering the assessments for reliability. Paper pencil and palms. Having teachers practice progress monitoring at grade level meetings.4)Provide time and support for schools during implementation/infrastructure necessary//help with purchase of materials and training for staff/additional technology support for palms and database reporting/Literacy Support Meetings/Grade Level meetings/Walkthroughs/Professional Development (ALL revolves around our data) Show book club list, most schools doing Differentiated Instruction this year plus another related to data at school level. CHANGE HAPPENS AT THE CLASSROOM LEVEL
87School Level Implementation Literacy Support Team Meetings and connections to ISTUse of assessment dataModel LessonsFollow up with teachersData notebook for principalImportance of principal involvement at all levels
88Implementation at the School Level Data notebooks for each teacherAdditional assessments and purposesDiagnostic assessments (chart)Infrastructure necessaryGrade level meetings and planning based on dataProfessional Development
89Your Role Before and After the RTI Initiative Topics to ConsiderDE RTI RegulationsSchedulesProfessional DevelopmentAccountabilityMaterials & ResourcesData Collection & ReviewParticipants will move to another room and discuss their roles in theimplementation of a effective comprehensive assessment plan forRTI in their districtsGuiding Questions for this activityHow will your role change?How will you role remain the same?What are some of the challenges that you might encounter?How will you support your colleagues in the implementation of RTI?
90Taking StockIntroduce the homework assignment by handing out the Taking Stock ofAssessments worksheets ( see Kathy’s form titled Taking Stock ofAssessments)