Child Outcomes Measurement: Getting to Quality Data

Child Outcomes Measurement: Getting to Quality Data
2009 Data Summit ECO Child Outcomes Measurement: Getting to Quality Data Lauren Barton and Robin Rooney ECO and NECTAC Arkansas July, 2009

Objectives Review the child outcomes data requirement
Refresh our understanding of the COSF process Learn strategies for COSF quality assurance Start thinking about child outcomes data

Keeping our eye on the prize: High quality services for children and families that will lead to good outcomes.

Goal of early childhood special education
“…To enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community.” (from Early Childhood Outcomes Center,

Driving Force for Data Comes from the Federal Level
Government Performance and Results Act (GPRA) Program Assessment Rating Tool (PART) Individuals with Disabilities Education Act (IDEA)

State of the Nation: Federal reporting requirements

Reporting Schedule Data in reported to OSEP in progress categories every February Data cover the previous federal fiscal year For example, data on all children who exited July 1, 2008 through June 30, 2009 will be reported in February 2010 Data are considered ‘baseline’ in 2010 States must set targets Summary statements used for target setting States must report data to public by school district/local program in 2011

OSEP reporting categories
2009 Data Summit ECO OSEP reporting categories Percentage of children who: Did not improve functioning Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers Improved functioning to a level nearer to same-aged peers but did not reach it Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers When OSEP says ‘improved,’ they want to know ‘what kind of progress’ children have made. These are the 5 categories of progress states must report. 3 outcomes x 5 “measures” = 15 numbers 8

Part C and Preschool. Average Percentage of Children in Each Category
Part C and Preschool Average Percentage of Children in Each Category Outcome 1: Social/Emotional 2009 Data Summit ECO This is an average of the percentages states reported for Outcome 1 – for Part C and preschool – as reported Feb 09. When using this presentation with local programs it is useful to add slides or handouts with the local program’s progress data which can then be compared with state and national bar charts. 9

Part C and Preschool Average Percentage of Children in Each Category Outcome 2: Knowledge/Skills
2009 Data Summit ECO This is an average of the percentages states reported for Outcome 2 – for Part C and 619 – as reported Feb 09. When using this presentation with local programs it is useful to add slides or handouts with the local program’s progress data which can then be compared with state and national bar charts. 10

2009 Data Summit ECO Part C and Preschool Average Percentage of Children in Each Category Outcome 3: Getting Needs Met This is an average of the percentages states reported for Outcome 3 – for Part C and 619 – as reported Feb 09. When using this presentation with local programs it is useful to add slides or handouts with the local program’s progress data which can then be compared with state and national bar charts. 11

It’s too early to put meaning on the data
2009 Data Summit ECO Caution: It’s too early to put meaning on the data Most states are focusing on improving the quality of the data collection The bar charts show some of the earliest progress data states have submitted to OSEP. This is meant to pique participant interest in the numbers and how they may look when aggregated. But no one really considers these early data to be highly accurate. So it’s too soon to attach meaning. 12

What we are learning nationally about child outcomes measurement

How are states collecting child outcomes data?
2009 Data Summit ECO How are states collecting child outcomes data? Possible state approaches to collection of child data Child Outcomes Summary Form (COSF) About 70% of state Part C programs About 60% of state 619 programs Single assessment statewide Publishers’ online assessment systems Other approaches According to the Annual Performance Reports submitted to OSEP in February 2009, most states were using the COSF. Some states have mandated that all local programs use the same tool for outcomes measurement. Some require everyone to use the BDI, others the AEPS, etc. A few states are working with the publishers of assessment tools to develop electronic systems so that local providers enter data into the system and the computer generates a report. 14

What we’re learning about child outcomes measurement
The process of training for child outcomes data collection has uncovered other areas of significant need related to professional development.

Providers need to know more about:
Assessment How to gather assessment data to reflect functioning across settings and situations, especially how to gather child functioning information from families Understanding the results of the assessment Sharing assessment results sensitively and honestly with families

Functional outcomes What are they? How do they differ from outcomes organized around domains? What do they mean for how professionals from different disciplines operate as a team?

Typical child development What are the functional expectations for children at different ages with regard to each of the 3 goal statements? How to involve families in the outcomes measurement process Working as a team

What we are learning nationally about COSF implementation
2009 Data Summit ECO What we are learning nationally about COSF implementation Challenges: Limited staff development time Learning age-expected child development Meeting as a team Shift toward thinking functionally 19

2009 Data Summit ECO What we are learning nationally about COSF implementation Strategies: Collaboration with other programs – preschool and Part C Focus groups scheduled throughout the year for teachers and providers Training module to include DVD, individual assistance if needed 20

2009 Data Summit ECO What we are learning nationally about COSF implementation Strategies: Training modules available on-line COSF training integral part of staff development Incorporating staff feedback into plans for staff development 21

2009 Data Summit ECO What we are learning nationally about COSF implementation Strategies: Use of Early Learning Guidelines Input from child care, general education and preschool teachers Phone conferences Scheduling meetings to coincide with IFSP/IEP team meetings Shared folder on program site that is password protected 22

2009 Data Summit ECO What we are learning nationally about COSF implementation Strategies: Establishing core competencies Shift training and practice to incorporate functional assessments and writing functional IFSP/IEP goals Topical monthly teleconferences for training and feedback Select a tool that helps develop functional IFSP/IEP goals Practice, practice, practice! 23

2009 Data Summit ECO What we are learning nationally about COSF implementation Benefits: Opportunities to collaborate with other staff, other programs Improved IFSP/IEP goals Improved communication with families about child’s functioning 24

2009 Data Summit ECO What we are learning nationally about COSF implementation Benefits: Stronger collaboration between Part C and preschool (especially when both programs are using the COSF) IEPs/IFSPs written with more functional goals At the end of this presentation, it may be helpful to hold a discussion of COSF implementation issues local programs are facing. A discussion worksheet can be found at this link 25

Arkansas Updates

OSEP progress categories

OSEP reporting categories
2009 Data Summit ECO OSEP reporting categories Percentage of children who: Did not improve functioning Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers Improved functioning to a level nearer to same-aged peers but did not reach it Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers When OSEP says ‘improved,’ they want to know ‘what kind of progress’ children have made. These are the 5 categories of progress states must report. 3 outcomes x 5 “measures” = 15 numbers 28

Getting to progress categories from the COSF ratings
2009 Data Summit ECO Getting to progress categories from the COSF ratings 29

Functioning 2009 Data Summit ECO
Developmental trajectories are a good way to think about how the numbers we report to OSEP come from the COSF data. A child’s developmental trajectory can be plotted from entry to exit using a points on the 7-point scale. On the graph displayed, the vertical axis shows the child’s level of functioning, corresponding to the 7 points on the rating scale. The dotted line separates overall age appropriate (6-7) from less than age appropriate (1-5). The horizontal axis corresponds to the child’s age in months. 30

2009 Data Summit ECO To plot a child’s developmental trajectory, we start by plotting his level of functioning at entry. In this example, the child’s entry rating was at a level 3. Because we will just plot entry and exit, we can indicate the entry rating on the left end of the horizontal axis. We will plot the exit rating at the right end of the horizontal axis. Entry 31

Entry Exit 2009 Data Summit ECO
At the time of exit, this child’s functioning at a level 4 in the outcome area. We can plot the exit rating toward the right end of the horizontal axis. Entry Exit 32

Draw a line to show the child’s developmental trajectory. This child came into the program at a level 3 and left at a level 4 in this particular outcome area. The exit rating is below the dotted line, indicating that he did not catch up with his peers and did not leave the program at age level. However, the upward, positive slope indicates that the child made progress. Entry Exit 33

2009 Data Summit ECO Key Point The OSEP categories describe types of progress children can make between entry and exit Two COSF ratings (entry and exit) are needed to calculate what OSEP category describes a child progress Now, to apply the developmental trajectory concept to OSEP reporting – think about the 5 categories of data as different types of progress a child can make when comparing his status at entry and his status at exit. The COSF ratings at entry and exit are the 2 data points needed to determine the type of progress made. 34

2009 Data Summit ECO How changes in ratings on the COSF correspond to reporting categories a - e e. % of children who maintain functioning at a level comparable to same-aged peers Rated 6 or 7 at entry; AND Rated 6 or 7 at exit Let’s look at one progress category at a time, starting with category ‘e’ – “maintained functioning at a level comparable to same-aged peers.” A child will be reported in this category if he enters Part C services at age level (COSF rating of 6 or 7) and exits Part C services at age level (COSF rating of 6 or 7). 35

This is the developmental trajectory for a child who will be reported in category ‘e.’ He entered services with a rating of 7 in a particular outcome area and exited services with a rating of 7. He maintained age appropriate functioning. Notice the positive, upward slope of the trajectory – which indicates that he made progress in this area. Remember that a child must make progress in order to maintain functioning at any level. This is because the child is older at exit than he was at entry, and we are comparing his functioning to age expectations in order to determine the rating. More skills are expected as a child gets older. So to maintain a level of 7 (or any level) the child would have to have added new skills. If a child looked the very same at exit as he did at entry, the rating would actually go down on the 7-point scale. In other words, a child who was age appropriate when he was 1 year old, but didn’t acquire any new skills by the time he exited, would actually look like a much younger child upon exit – and would receive a much lower rating. Entry Exit 36

This is another example of a developmental trajectory for a child who is reported in category ‘e.’ Even though he entered services with a lower rating than at exit, he was still in the realm of ‘overall age appropriate’ and therefore “maintained functioning at a level comparable to same-aged peers.” Entry Exit 37

One more developmental trajectory that corresponds with the ‘e’ category. Even though the child’s rating was lower at exit (went from 7 to 6), 6 is still in the realm of overall age appropriate and therefore it is considered that the child “maintained functioning at a level comparable to same-aged peers.” Notice that the trajectory has a slight upward, positive slope – indicating that the child did make some progress. If he had not made any progress at all between entry and exit, he would have looked like a much younger child – and his exit rating would have been lower… Entry Exit 38

Rated 5 or lower at entry; AND
2009 Data Summit ECO How changes in ratings on the COSF correspond to reporting categories a - e Rated 5 or lower at entry; AND Rated 6 or 7 at exit d. % of children who improve functioning to reach a level comparable to same-aged peers Category ‘d’ includes the children who “improved functioning to reach a level comparable to same-aged peers.” These are children who entered services below age level, but exited services AT age level. These are the children who ‘catch up.’ 39

The child with this developmental trajectory entered services with a COSF rating of 5 in an outcome area. His exit rating of 6 or 7 indicates the child is now showing age expected functioning. He caught up. Entry Exit 40

Rated higher at exit than entry; AND
2009 Data Summit ECO How changes in ratings on the COSF correspond to reporting categories a - e c. % of children who improved functioning to a level nearer to same aged peers, but did not reach it Rated higher at exit than entry; AND Rated 5 or below at exit Children are reported in the “c” progress category if the “improved functioning to a level nearer to same-aged peers but did not reach it.” The children reported in this category entered services below age level and exit services at a higher rating – but not at a level 6 or 7. They narrowed the gap, but did not close it. 41

The developmental trajectory for a child in the ‘c’ category may have entered services with a rating of 3, as in this example, and exited services with a rating of 4. The upward slope of the trajectory shows that the child made progress, even though he didn’t move beyond the dotted line into age-appropriate functioning. Entry Exit 42

Here is another example with a child who entered with a rating of 3 and left with a rating of 5, showing a very steep growth rate. Entry Exit 43

Rated 5 or lower at entry; AND Rated the same or lower at exit; AND
2009 Data Summit ECO How changes in ratings on the COSF correspond to reporting categories a - e b. % of children who improved functioning, but not sufficient to move nearer to same aged peers Rated 5 or lower at entry; AND Rated the same or lower at exit; AND “Yes” on the progress question (b) Children reported in the ‘b’ category “improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers.” They entered and exited services below age level and did not move up on the scale. Their exit rating may actually be lower than the entry rating. What determines that the child should be reported in progress category ‘b’ (rather than category ‘a,’ which means NO progress) is the yes/no progress question on the COSF. The question asks if the child made any progress, including the acquisition of even one new skill between the time of entry and exit. If the answer is yes, the child will be reported to OSEP in the ‘b’ progress category. 44

The child whose trajectory looks like this one would be reported in ‘b.’ The child entered services with a rating of 5 and left with a rating of 5. The answer to the yes/no progress question was ‘yes,’ the child had acquired at least one new skill. When you plot the trajectory, note the slope - the line is not flat. The child has improved functioning. Entry Exit 45

This child’s rating actually went down between entry and exit. That means the child’s development did not keep up with age expectations. The yes/no progress question was answered ‘yes,’ however, indicating that the child acquired at least one new skill. Note that this trajectory also has a slightly upward slope. Entry Exit 46

Another example – the child entered services with a rating of 3 in a particular outcome. At exit, he was also given a rating of 3. Remember that you always compare the child’s development to age expectations. As the child gets older, more is expected. To receive a rating of 3 at exit means that he still showed immediate foundational skills relative to an older child. Maintaining the same rating actually implies progress. The yes/no question should be answered ‘yes’ and the child reported in the ‘b’ category of progress. Entry Exit 47

One more example – even in maintaining a rating of 1, the child may have acquired a new skill and could be reported in the ‘b’ category. Note the slope - this child is showing improved functioning. Entry Exit 48

2009 Data Summit ECO How changes in ratings on the COSF correspond to reporting categories a - e a. % of children who did not improve functioning Rated lower at exit than entry; OR Rated 1 at both entry and exit; AND Scored “No” on the progress question (b) Children reported in the ‘a’ progress category are those who “did not improve functioning.” These are the children who exit services at a lower rating than when they entered and the yes/no progress question indicated that they did NOT acquire even one new skill. 49

In this example of a child’s developmental trajectory, he entered services with a rating of 3 and exited with a rating of 2 in this particular outcome area. The yes/no progress question was answered ‘no.’ Notice the downward slope of the trajectory as plotted on this chart. Entry Exit 50

The child who enters and exits with a rating of 1 may also be reported in the ‘a’ category – IF the yes/no progress question is answered ‘no.’ Entry Exit 51

2009 Data Summit ECO The ECO Calculator can be used to translate COSF entry and exit ratings to the 5 progress categories for federal reporting ECO developed a calculator in Excel for states and programs to use to translate the COSF entry and exit ratings into the progress categories for federal reporting. Find the calculator (tutor) at this link -- 52

2009 Data Summit ECO These are the progress data AR 619 submitted to OSEP in February 2009. 53

2009 Data Summit ECO These are the progress data AR preschool submitted to OSEP in February 2009. 54

2009 Data Summit ECO These are the progress data AR preschool submitted to OSEP in February 2009. 55

2009 Data Summit ECO These are the progress data AR Part Csubmitted to OSEP in February 2009. 56

2009 Data Summit ECO These are the progress data AR Part Csubmitted to OSEP in February 2009. 57

2009 Data Summit ECO These are the progress data AR Part C submitted to OSEP in February 2009. 58

Next steps: Target setting

Summary statements for target setting
2009 Data Summit ECO Summary statements for target setting States set targets for two sets of progress data These sets of progress data are referred to as ‘summary statements’ Summary statements were developed so that states would not have to set 15 targets for child outcomes! OSEP decided not to have states set goals (or ‘targets’) for each of the 5 progress categories. Because there are 3 outcomes, setting goals for each category would mean 15 targets just for this indicator on each state’s Annual Performance Report! Instead, OSEP collected several rounds of input and feedback from stakeholders and worked with ECO to define two ‘summary statements’ for setting targets. The next two slides describe the summary statements. 60

2009 Data Summit ECO Summary Statement 1 Of those children who entered the program below age expectations in the Outcome Area, the percent who substantially increased their rate of growth by the time they exit the program. Progress categories (c + d) (a + b + c + d) The first summary statement is about children who made ‘more than expected’ progress while they were in the program. Children who make ‘more than expected’ progress are those who move up on the COSF 7-point rating scale. They include children who ‘catch up’ to age level (which is progress category d) and also children who do not catch up, but move closer to age level by the time they leave the program (which is progress category c). For this summary statement, states will set targets for the percentage of children who leave the program having caught up, or having moved closer, to age level. 61

2009 Data Summit ECO Summary Statement 2 The percent of children who are functioning within age expectations in the Outcome Area by the time they exit the program. Progress categories d + e The second summary statement is about children who leave the program at age level. Children who leave the program at age level are those who move up to a 6 or 7 on the COSF 7-point rating scale (which is progress category d) or who maintain a level of 6 or 7 for entry and exit COSF ratings (which is progress category e). For this summary statement, states will set targets for the percentage of children who leave the program having caught up to or maintained age level. 62

Understanding the Outcomes and the COSF: A Quick Review

Essential knowledge for completing the Child Outcomes Summary Form
Between them, team members must: Know about the child’s functioning across settings and situations Understand age-expected child development Understand the content of the three child outcomes Know how to use the rating scale Understand age expectations for child functioning within the child’s culture

Outcomes Jeopardy Biting $100 $100 $100 $200 $200 $200 $300 $300 $300
2009 Data Summit ECO Outcomes Jeopardy Pointing to the cabinet for cereal Reading the letter “S” on the Stop sign Washes hands before lunch Biting Plays by himself in the classroom Plays with rhyming words Building a castle from blocks with a friend Problems sleeping Sharing a cookie at lunchtime $100 $100 $100 $200 $200 $200 Note: you must be in ‘slide show’ mode for this to work. You should click on each box and the ‘door’ slides away. Answer key: (1) 100—Outcome 3; (2)100—Outcome 2; (3) 100—Outcome 3. (1)200—Outcome 1, 2 or 3, depending upon how biting is affecting this child’s functioning (Outcome 1 if no one will play because he/she bites, Outcome 2 if he or she is always in time out for biting and misses the learning activities, Outcome 3 if it’s associated with feeding issues); (2)200—Outcome 1; (3)200—Outcome 2. (1)300—Outcome 1 and 2 (playing with a friend=Outcome 1 and representational play=Outcome 2); (2)300—Outcome 1, 2 or 3, depending upon how lack of sleep affects this child’s functioning (Outcome 1 if he/she is too sleepy to interact with others, Outcome 2 if he/she is too sleepy to participate in learning activities, Outcome 3 if he or she is too sleepy to get his or her needs met, (3)300—Outcome 1 and 3 (sharing=Outcome 1 and participating in lunch=Outcome 3). $300 $300 $300 65

Children have positive social relationships
Involves: Relating with adults Relating with other children For older children, following rules related to groups or interacting with others Includes areas like: Attachment/separation/autonomy Expressing emotions and feelings Learning rules and expectations Social interactions and play

Children acquire and use knowledge and skills
Involves Thinking Reasoning Remembering Problem solving Using symbols and language Understanding physical and social worlds Includes: Early concepts—symbols, pictures, numbers Imitation Object permanence Expressive language and communication Early literacy

Children take appropriate action to meet their needs
Involves: Taking care of basic needs Getting from place to place Using tools (e.g., fork, toothbrush, crayon) In older children, contributing to their own health and safety Includes: Integrating motor skills to complete tasks Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility) Acting on the world to get what one wants

Rating Scale Jeopardy $100 $100 $100 $200 $200 $200 $300 $300 $300
2009 Data Summit ECO Rating Scale Jeopardy Age appropriate functioning – no concerns Mix of age appropriate and not age appropriate functioning No age appropriate functioning – not yet showing immediate foundational skills Some age appropriate functioning but very little No age appropriate functioning – lots of immediate foundational skills Age appropriate functioning – some concerns Rarely shows age appropriate functioning No age appropriate functioning – some immediate foundational skills Age appropriate functioning $100 $100 $100 $200 $200 $200 Note: you must be in ‘slide show’ mode for this to work. You should click on each box and the ‘door’ slides away. Answer key: (1) 100—rating=7, (2)100—rating=5, (3) 100—rating=1. (1)200—rating=4, (2)200—rating=3, (3)200—rating=6. (1)300—rating=4, (2)300—rating=2, (3)300—rating= 6-7 $300 $300 $300 69

7 – Completely Child shows functioning expected for his or her age in all or almost all everyday situations that are part of the child’s life Functioning is considered appropriate for his or her age No one has any concerns about the child’s functioning in this outcome area

6 – Between completely and somewhat
Child’s functioning generally is considered appropriate for his or her age but there are some significant concerns about the child’s functioning in this outcome area These concerns are substantial enough to suggest monitoring or possible additional support Although age-appropriate, the child’s functioning may border on not keeping pace with age expectations

5 – Somewhat Child shows functioning expected for his or her age some of the time and/or in some settings and situations Child’s functioning is a mix of age-appropriate and not age-appropriate behaviors and skills Child’s functioning might be described as like that of a slightly younger child

4 – Between a 5 and a 3 Child shows occasional age-appropriate functioning across settings and situations More functioning is not age-appropriate than age-appropriate.

3 – Nearly Child does not yet show functioning expected of a child of his or her age in any situation Child uses immediate foundational skills, most or all of the time across settings and situations Immediate foundational skills are the skills upon which to build age-appropriate functioning Functioning might be described as like that of a younger child

2 – Between 3 and 1 Child occasionally uses immediate foundational skills across settings and situations More functioning reflects skills that are not immediate foundational than are immediate foundational

2009 Data Summit ECO 1 – Not yet Child does not yet show functioning expected of a child his or her age in any situation Child’s functioning does not yet include immediate foundational skills upon which to build age-appropriate functioning Child functioning reflects skills that developmentally come before immediate foundational skills Child’s functioning might be described as like that of a much younger child After going through the ratings on these slides, or at the end of this presentation, it is helpful to walk through a child example. This can be a child presented by a participant, going through the assessment data and discussing level of functioning, etc. in order to come up with a rating. See instructions for holding a COSF rating discussion as part of a training activity at this link: 76

Using data for program improvement = EIA
Evidence Inference Action

Evidence Evidence refers to the numbers, such as
2009 Data Summit ECO Evidence Evidence refers to the numbers, such as “45% of children in category b” The numbers are not debatable 78

Inference How do you interpret the #s?
2009 Data Summit ECO Inference How do you interpret the #s? What can you conclude from the #s? Does evidence mean good news? Bad news? News we can’t interpret? To reach an inference, sometimes we analyze data in other ways (ask for more evidence) 79

2009 Data Summit ECO Inference Inference is debatable -- even reasonable people can reach different conclusions Stakeholders can help with putting meaning on the numbers Early on, the inference may be more a question of the quality of the data 80

Explaining variation Who has good outcomes = Do outcomes vary by
2009 Data Summit ECO Explaining variation Who has good outcomes = Do outcomes vary by Region of the state? Amount of services received? Type of services received? Age at entry to service? Level of functioning at entry? Family outcomes? Education level of parent?

Action Given the inference from the numbers, what should be done?
2009 Data Summit ECO Action Given the inference from the numbers, what should be done? Recommendations or action steps Action can be debatable – and often is Another role for stakeholders Again, early on the action might have to do with improving the quality of the data 82

Promoting quality data through data analysis
2009 Data Summit ECO Promoting quality data through data analysis Examine the data for inconsistencies If/when you find something strange, what might help explain it? Is the variation because of a program data? Or because of bad data? (at this point in the implementation process, data quality issues are likely!) Quality data means that the data are valid and accurate. Look at your data by program and determine whether what you see matches with what you expected to see. If something looks strange, discuss what program or family characteristics might explain the differences. At this point in the data collection, however, it’s likely that the variation means the data are not accurate. Looking at the data across programs will help pinpoint where the issues might lie and who needs more training in what areas of the COSF process. 83

The validity of your data is questionable if…
The overall pattern in the data looks ‘strange’ Compared to what you expect Compared to other data Compared to similar states/regions/agencies

COSF Ratings – Outcome 1 Entry data (fake data)
2009 Data Summit ECO COSF Ratings – Outcome 1 Entry data (fake data) Rating Statewide numbers 1 30 2 42 3 51 4 60 5 10 6 7 Suppose you looked at your statewide data by rating at entry. Do these patterns match what you would expect to see? Does anything surprise you? Are you surprised that no one was given a rating of 7 at entry? Are you surprised that the most popular rating is 4? What might explain the patterns that you see? 85

2009 Data Summit ECO COSF Ratings – Outcome 1 Entry data (fake data) Rating Statewide numbers and % 1 30 (15%) 2 42 (20%) 3 51 (25%) 4 60 (30%) 5 10 (5%) 6 7 0 (0%) Here is the same data, but by percentage rather than number. How does the pattern look now? Do you prefer looking at the data by percentage? Does it make any more sense or less sense? 86

Frequency on Outcome 1 – Statewide (fake data)
2009 Data Summit ECO Frequency on Outcome 1 – Statewide (fake data) Another way to look at the data – with a bar chart. What does this bar chart communicate to you? 87

2009 Data Summit ECO COSF Ratings – Outcome 1 Entry data (fake data) Rating Agency 1 Agency 2 Agency 3 Agency 4 1 3 2 4 5 6 7 It is useful to compare data across groups. These groups could be local agencies in the state. Does any group’s data look strange? Is it odd that one agency had only one child who entered services at age level (agency 1). What might explain the differences? 88

2009 Data Summit ECO COSF Ratings – Outcome 1 Entry data (fake data) Rating Group 1 Group 2 Group 3 Group 4 1 15% 5% 10% 2 20% 3 25% 4 30% 5 6 7 0% Here are the same data, but with percentages. Do you think these are easier to compare? Does any group’s data look strange? What might explain the differences? 89

Questions to ask when looking at data
2009 Data Summit ECO Questions to ask when looking at data Do the data make sense? Am I surprised? Do I believe the data? Some of it? All of it? If the data are reasonable (or when they become reasonable), what might they tell us? When we believe the data, how can we use it for program improvement? Right now we are talking about analyzing data for the purposes of improving data quality. At some point, when staff are using the COSF reliably and with meaning, we can look at the data for program improvement purposes. THEN when we see differences we may conclude that some programs have poorer outcomes for children and need some help. We will also conclude that some programs have great outcomes for children – and we will want to learn from them! 90

In the end, data are not just numbers
In the end, data are not just numbers. Our data can help us make a difference in the everyday lives of children and their families.

Child Outcomes Measurement: Getting to Quality Data

Similar presentations

Presentation on theme: "Child Outcomes Measurement: Getting to Quality Data"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Child Outcomes Measurement: Getting to Quality Data

Similar presentations

Presentation on theme: "Child Outcomes Measurement: Getting to Quality Data"— Presentation transcript:

Similar presentations

About project

Feedback