2. Indicator 7: Measuring Preschool Outcomes Entry Data Collection Using the COS Process Sarah Geldart – MA ESE Sarah.Geldart@doe.mass.eduSarah.Geldart@doe.mass.edu 781.338.3364 Additional Contact: Lauren Viviani – MA 619 Coordinator lviviani@doe.mass.edulviviani@doe.mass.edu 781.338.3372

3. Agenda  Overview of State Performance Plan/Annual Performance Report (SPP/APR)  Indicator 7  Completing the Child Outcomes Summary Process  Identifying entry ratings 2

4. SPP/APR Information  17 indicators of performance or compliance in special education  State identified targets for performance indicators  MA ESE set targets for Indicator #7 based on data & feedback from stakeholders 3

5. Indicator 7: Preschool Outcomes Percent of preschool children aged 3 through 5 with IEPs who demonstrate improved: A.Positive social-emotional skills (including social relationships); B.Acquisition and use of knowledge and skills (including early language/communication and early literacy); and C.Use of appropriate behaviors to meet their needs (20 U.S.C. 1416(a)(3)(A)) 4

6. Outcome A: Children Have Positive Social Relationships Involves:  Relating with adults  Relating with other children  For older children- following rules related to groups or interacting with others Includes areas like:  Attachment/separation/ autonomy  Expressing emotions and feelings  Learning rules and expectations  Social interactions and play 5

7. Outcome B: Children Acquire & Use Knowledge & Skills Involves:  Thinking, reasoning, remembering, problem-solving  Using symbols and language  Understanding physical and social worlds Includes:  Early concepts – symbols, pictures, numbers, classification, spatial relationships  Imitation  Object permanence  Expressive language and communication  Early literacy 6

8. Outcome C: Children Take Appropriate Action to Meet Their Needs Involves:  Taking care of basic needs  Getting from place to place  Using tools  In older children, contributing to their own health and safety Includes:  Integrating motor skills to complete tasks  Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility)  Acting on the world to get what one wants 7

9. Indicator 7: Preschool Outcomes  Five Progress Categories for Each Outcome  The percent of children who: a)did not improve functioning b)improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers c)improved functioning to a level nearer to same-aged peers but did not reach it d)improved functioning to reach a level comparable to same-aged peers e)maintained functioning at a level comparable to same- aged peers 8

10. Summary Statements  OSEP has taken the five categories and collapsed them into two summary statements  Summary Statement 1: Of those children who entered the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they exited the program.  Summary Statement 2: The percent of children who were functioning within age expectations in each Outcome by the time they exited the program. 9

11. The Child Outcomes Summary Process Presentation developed with the assistance of the Early Childhood Technical Assistance Center

12. What is the Child Outcomes Summary (COS) Process?  A team process for reviewing child assessment data from different sources culminating in the ‘rating’ of a child’s functioning on a scale of 1-7  Considers the child’s functioning across situations and settings  At home, in a play group, at the library, in child care, at community events, in preschool, etc.  Compares child’s functioning to same age peers 11

13. Why was the process developed?  For federal reporting on child outcomes  No method to synthesize child outcomes data from multiple sources  Different programs would be using different assessment instruments, and outcomes data would need to be aggregated across programs 12

14. Purposes of the COS  It is not an assessment tool  It uses information from multiple sources, including assessment tools, observations, and family interview to get a global sense of how the child is doing at one point in time 13

15. Features of the COS  7-point rating scale  Rating is based on the child’s functioning:  What the child does across settings and situations  Compared with what is expected given the child’s age  Ratings look at the child in two ways  How is the child functioning compared to age- expected functioning?  Has the child gained at least one new skill in this area? 14

16. Why two COS questions?  Progress data for federal reporting are calculated from the COS ratings (entry versus exit) plus the yes/no progress question  The yes/no progress question differentiates between children who made absolutely NO progress or regressed -- and children who gained at least one new skill 15

17. Timing of the COS Process  In order to ensure that the entry ratings represent a accurate snapshot of a child’s current functioning, please complete the COS Process within 4-6 weeks of a student entering a program or beginning services and within 4-6 weeks of a student exiting a program or ending services. 16

18. The Child Outcomes Summary (COS) Process Children ages 3-5 start special education services (Aug. 1 – Dec. 31) Data is collected about functioning across settings and situations (including from families) The COS team meets to discuss functioning, complete the COS Form, and assign a rating for each outcome (ideally within 4-6 weeks of starting services) January-Entry data from the COS Forms is entered into the data collection spreadsheet and submitted to ESE June– Exit data is collected as students exit the program and reported annually to ESE in June 17

19. Sources of Information for the COS Process 18

20. Summary ratings are based on… Types of Information  Curriculum-based assessments (e.g., HELP)  Norm-referenced assessments (e.g., BDI-2)  Developmental screenings (e.g., Ages & Stages)  Observation and report Sources of Information  Routines Based Interview (family information)  Service providers  Therapists  Physicians  Child care providers  Teachers  People familiar with the child in all of the settings and situations he’s in 19

21. Gathering Information from Families  Family input about the child’s functioning is critical  Family members see the child in situations that professionals do not  Need to ask family members about what the child does at home  The team will need a way to learn what family members know about the child  There is no expectation that family members will be able to determine whether what they are seeing is age appropriate 20

22. Essential Knowledge for Completing the COS Among them, COS team members must:  Know about the child’s functioning across settings and situations  Understand age-expected child development  Understand the content of the three child outcomes  Know how to use the rating scale  Understand age expectations for child functioning within the child’s culture 21

23. Important point  It is not necessary that all team members be knowledgeable in all 5 areas  Especially, there is no expectation that parents understand the rating scale or typical child development…  But the professionals have to! 22

24. Why age anchor? To answer the two COS questions: a. To what extent does this child show age- appropriate functioning, across a variety of settings and situations, on this outcome? (Rating: 1-7) b.Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No) Age Anchoring Resources: http://ectacenter.org/eco/pages/training_resources.asp#ChildDevelopment 23

25. The Child Outcomes Summary (COS) Process Children ages 3-5 start special education services (Aug. 1 – Dec. 31, 2014) Data is collected about functioning across settings and situations (including from families) The COS team meets to discuss functioning, complete the COS Form, and assign a rating for each outcome (ideally within 4-6 weeks of starting services) January 2015 – Entry data from the COS Forms is entered into the data collection spreadsheet and submitted to ESE June 2015-June 2017 – Exit data is collected as students exit the program and reported annually to ESE in June 24

26. The Form  Cover page + three outcome pages  On each outcome page:  Two questions per outcome  Space to document the basis for the rating  Accessing the form: http://ectacenter.org/eco/pages/outcomes.asp# COSFormandInstructions25

27. COS Form – Cover Page 26

28. COS Form- Page 2 27

29. COS Form – Page 2 Continued 28

30. COS Form – Page 2 Continued 29 This question will be answered when completing the exit rating.

31. The Process for Answering Questions 1a, 2a, 3a For each outcome… 1.Discuss the child’s current functioning in this outcome area across settings and situations 2.Identify areas where the child’s functioning is age appropriate 3.If not all functioning is age appropriate, identify areas where the child’s functioning reflects immediate foundational skills 4.Decide which rating best describes the child’s current functioning 30

32. Selecting a Rating

33. Looking at Development… 32 Foundational Skills Immediate Foundational Skills Age- Expected Skills

34. 7 – Completely  Child shows functioning expected for his or her age in all or almost all everyday situations that are part of the child’s life  Functioning is considered appropriate for his or her age  No one has any concerns about the child’s functioning in this outcome area 33

35. 6 – Between completely and somewhat  Child’s functioning generally is considered appropriate for his or her age but there are some significant concerns about the child’s functioning in this outcome area  These concerns are substantial enough to suggest monitoring or possible additional support  Although age-appropriate, the child’s functioning may border on not keeping pace with age expectations 34

36. 5 – Somewhat  Child shows functioning expected for his or her age some of the time and/or in some settings and situations  Child’s functioning is a mix of age-appropriate and not age-appropriate behaviors and skills  Child’s functioning might be described as like that of a slightly younger child 35

37. 4 – Between somewhat and nearly  Child shows occasional age-appropriate functioning across settings and situations  More functioning is not age-appropriate than age-appropriate 36

38. 3 – Nearly  Child does not yet show functioning expected of a child of his or her age in any situation  Child uses immediate foundational skills, most or all of the time across settings and situations  Immediate foundational skills are the skills upon which to build age-appropriate functioning  Functioning might be described as like that of a younger child 37

39. 2 – Between nearly and not yet  Child occasionally uses immediate foundational skills across settings and situations  More functioning reflects skills that are not immediate foundational than are immediate foundational 38

40. 1 – Not yet  Child does not yet show functioning expected of a child his or her age in any situation  Child’s functioning does not yet include immediate foundational skills upon which to build age- appropriate functioning  Child functioning reflects skills that developmentally come before immediate foundational skills  Child’s functioning might be described as like that of a much younger child 39

41. 40 EntryExit Age-expected development

42. 41 http://ectacenter.org/eco/assets/pdfs/Decision_Tree.pdf

43. Checking for Quality - Ratings  Ratings on all three outcomes should be reported for every child  Ratings are needed in all areas even if:  No one has concerns about a child’s development.  A child has delays in one or two outcome areas, but not in all three outcome areas 42

44. Checking for Quality - Documenting the Rating  On the form, documentation needs to include:  What evidence led to the selected rating, evidence of …..  Age expected functioning?  Immediate foundational skills  Skills and behaviors that will lead to immediate foundational skills  Who participated in the conversation and the decision  Documentation provides a record of the rationale for the rating decision 43

45. Why is it important to document the rating?  Evidence can be reviewed to see whether people are using the system properly (i.e., rating similar children in the same ways)  Documentation helps identify needs for future training and technical assistance  Documentation may be useful for new team members reviewing the file 44

46. Checking for Quality - The Progress Questions (1b, 2b, 3b)  Apply only if a Child Outcomes Summary Form has been completed previously  Do not complete at entry  Has the child shown any new skills or behaviors in the outcome area since the last rating? (Yes or No)  Small steps of progress count!  Most will select “Yes” 45

47. Checking for Quality – Completeness of the Form  All information is completed (i.e. name, DOB, people involved, family input)  Evidence for all ratings, that support each of the ratings  Ratings complete  Progress question answered at exit 46

48. Children ages 3-5 start special education services (Aug. 1 – Dec. 31) Data is collected about functioning across settings and situations (including from families) The COS team meets to discuss functioning, complete the COSF, and assign a rating for each outcome (ideally within 4-6 weeks of starting services) January– Entry data from the COSFs is entered into the data collection spreadsheet and submitted to ESE June– Exit data is collected as students exit the program and reported annually to ESE in June 47 The Child Outcomes Summary (COS) Process

49. Resources: Early Childhood Technical Assistance Center  Assessment Crosswalks  ECTA Center has “crosswalked” assessment tools to the Outcomes  Crosswalks show which sections of assessment are related to each Outcome  The number of items addressing an Outcome does not necessarily mean that the assessment captures functioning across settings  The COSF & information on how to complete it  Other resources on Early Childhood Outcomes www.ectacenter.org 48

50. Resources: Early Childhood Technical Assistance Center  Outcomes Measurement PD Resources:  http://ectacenter.org/eco/pages/training_resources.asp http://ectacenter.org/eco/pages/training_resources.asp  Overview of the COS Process from ECTA:  http://ectacenter.org/eco/assets/docs/COSF_overview.rtf http://ectacenter.org/eco/assets/docs/COSF_overview.rtf  The Child Outcomes Step-by-Step Video:  http://ectacenter.org/eco/pages/videos.asp http://ectacenter.org/eco/pages/videos.asp  Outcome Rating Definitions:  http://ectacenter.org/eco/assets/pdfs/Definitions_Outcome_Ratings.pdf http://ectacenter.org/eco/assets/pdfs/Definitions_Outcome_Ratings.pdf  The Ratings Decision Tree:  http://ectacenter.org/eco/assets/docs/Decision_Tree.doc http://ectacenter.org/eco/assets/docs/Decision_Tree.doc 49

51. Contact Information for Indicator 7 Primary Contact: Sarah Geldart  Sarah.geldart@doe.mass.edu – 781.338.3364 Sarah.geldart@doe.mass.edu Additional Contact: Lauren Viviani  lviviani@doe.mass.edu – 781.338.3372 lviviani@doe.mass.edu 50