1. Measuring Child and Family Outcomes
Session One
Linking Assessment to Functional Outcomes and Accountability
Anne Brager, MS, RN
Program Supervisor
Frederick County Infants and Toddlers Program
Adapted from ECO Center

2. Good Outcomes Begin with Good Assessment!
What are Outcomes?
Outcomes are the benefits that children and families experience as a result of early intervention
The three child outcomes, as measured for accountability purposes are different than IFSP outcomes in that:
OSEP’s three child outcomes for accountability apply to all children and reflect global functioning in broad areas of development (social-emotional, knowledge and skills, getting needs met)
IFSP outcomes are specific to an individual child, based on his or her individual needs

3. What is Assessment?
“Early childhood assessment is a flexible, collaborative decision-making process in which teams of parents and professionals repeatedly revise their judgments and reach consensus about the changing developmental, educational, medical and mental health services needs of young children and their families.”
(Bagnato and Neisworth, 1991)
Quoted in DEC Recommended Practices, 2005

4. Assessment is…
“Assessment is a generic term that refers to the process of gathering information for decision-making.”
Mary McLean, 2004

5. Purposes of Assessment
In the early intervention system
assessment serves several purposes:
Identification of children with special needs (including screening, diagnosis, and determination of eligibility for services)
Planning instruction or intervention
Monitoring of the child’s level of knowledge or progress
Program evaluation and accountability

6. Accountability???
Accountability involves embedding ongoing data collection into program practices because the need to examine the effectiveness of the services being provided is ongoing
Given the number of factors that influence the provision of services (e.g., staff changes, reimbursement formulas), it is not reasonable to assume that a specific program that was effective at one point in time will always be effective

7. Assessment for Accountability
Policymakers and funders are raising legitimate questions about which programs are effective as they struggle with how to allocate resources
The Government Performance and Results Act (GPRA) means Part C Programs need to provide evidence that program participants demonstrate positive outcomes
An important issue for assessment is that the Office for Special Education Programs (OSEP) decided not to require states to use the same assessment tool. Each state is free to chose their own

8. Assessment as a Process
It is important to distinguish between
assessment as a process and
assessment as a single instrument or test!
Assessment as a process involves multiple ways of collecting information about a child’s performance, including, for example, observation, documentation of the child’s work, checklists, rating scales, portfolios and norm-referenced or curriculum based tests

9. Variation in What Tools are Used
So far these are the tools we know that
states are using for accountability:
Of 28 states who listed specific assessment instruments:
HELP-15 states
BDI/BDI-2-13 states
AEPS-11 states
Creative Curriculum-6 states
ELAP-6 states
Not yet determined-23 states

10. Variation in How Assessments Have Been Used
In the past, there has been much variation
in assessment procedures in early
intervention programs across the country:
In some programs, the only formal assessment has been for eligibility
Other states completed assessment annually
Some states have always completed assessment at exit

11. Views on Assessment There have been major changes in last 15
years in how assessment of young
children is viewed by the early childhood
community:
Old position: Do not test little kids
New position: Ongoing assessment is part of a high quality early childhood program

12. New Thoughts on Assessment
“The best way to understand the development of children is to observe their behavior in natural settings while they are interacting with familiar adults over prolonged periods of time”
--Uri Bronfenbrenner

13. What Changed our Thinking
The purpose of assessment was redefined
in the last decade:
It’s no longer about sorting, labeling, using to deny access to services
It’s now about getting a rich picture of what children can do and can’t do and using that information to help them acquire new skills
It’s now about monitoring progress
New and different tools became available for general
early childhood: Curriculum-based assessments were developed, e.g., Creative Curriculum, Work Sampling,
etc.

14. What Changed Our Thinking
Assessment had always been seen as a process with multiple purposes
Distinctions have been made between good and bad uses of assessment with young children
Good assessment practices are now widely recognized as a key component of high-quality early childhood programs
Good uses are now promoted!
For more information: National Association for Education of Young Children (NAEYC) web site (Position statement on Curriculum, Assessment and Evaluation)

15. NAEYC Position Statement
Policymakers, early childhood professionals and other
stakeholders in young children’s lives have a shared
responsibility to:
Make ethical, appropriate, valid and reliable assessment a central part of all early childhood programs.
To assess young children’s strengths, progress,
and needs, use assessment methods that are
developmentally appropriate
culturally and linguistically responsive
tied to children’s daily activities
supported by professional development
inclusive of families

16. NAEYC Position Statement continued…
Connected to specific, beneficial purposes:
Making sound decisions about teaching and learning
Identifying significant concerns that may require focused intervention for individual children
Helping programs improve their educational and developmental interventions

17. DEC Recommended Practices for Assessment
The Recommended Practices emerged from
focus groups and are supported by literature
from the Division of Early Childhood (DEC):
Parents and professionals who participated in these focus groups identified repeatedly two concerns
Principle stakeholders, parents and family members must play a vital and indispensible role in assessment from beginning to end
Assessment methods and materials must accommodated children’s developmental and disability specific characteristics

18. DEC Recommended Practices for Assessment
The Recommended Practices are organized
around five statements:
Professionals and families collaborate in planning and implementing assessment
Assessment is individualized and appropriate for the child and family
Assessment provides useful information for intervention
Professionals share information in respectful and useful ways
Professionals meet legal and procedural requirements and Recommended Practice guidelines

19. Assessment for Accountability
The assessment process we use to determine eligibility
and program planning can be used for accountability!
The three child outcomes:
Children have positive social-emotional skills (including social relationships)
Children acquire and use knowledge and skills (including early language/communication [and early literacy])
Children use appropriate behaviors to meet their needs
Assessment tools can inform us about children’s functioning in each of the three outcome areas
Challenge: There is no assessment tool that assesses the three outcomes directly

20. Additional Challenges
Young children are unreliable test takers
Because of their limited ability to understand and follow the directions of the test
They may not be able to control their behavior
They have short attention spans
And are easily distracted
What young children demonstrate in a test situation may not be an accurate representation of what they are capable of doing on a daily basis

21. Types of Tests Standardized vs curriculum based
By definition, standardized tests must be administered the same way to all test takers.
The use of a consistent approach during test administration enhances reliability between administrators and confidence that children are responding to the same tasks.
Standardized tests usually provide normative data to compare the child’s functioning with that of other children of the same age.

22. Standardized Tools Few standardized assessment tools
provide detailed guidelines for appropriate
accommodations during testing to
effectively measure capabilities among
the broad range of children who have
special needs.” McLean, 2005

23. Curriculum-based Assessments
Bagnato and Neisworth (1991) define curriculum-based assessment as a “form of criterion-referenced assessment wherein curricular objectives act as the criteria for the identification of instructional targets and for the assessment of status and progress”
Some of the newer assessment tools emphasize critical skills to facilitate the development of fuctional behavior

24. Currently Available Assessment Tools
Most assessment tools are organized around developmental domains and are based on sequences of skills that build on each other
This organization may pose difficulties for accurate assessments of functioning in young children with disabilities because delayed skills in one domain can influence assessment of or functioning in other domains
Each assessment tool sees children through its own lens
Each lens is slightly different
Lens are not right or wrong

25. Variations Among States
States are continuing to make key decisions related to their accountability systems, so any information about state choices must be seen as time-limited and interpreted cautiously
In many state, some assessment-related decisions are being made by local programs, so there is variation within the states
The information available thus far suggests that much of the data on child outcomes will be collected with curriculum-based tools

26. Variations Among States
Most states are building their accountability systems around assessments that were already being administered for other purposes (such as eligibility determination or program planning)
Few have adopted a new statewide measure solely for collecting data for accountability purposes

27. Maryland’s Top Five
Following a survey distributed to all local Program Directors in the state, Maryland state Department of Education (MSDE) identified the 5 most commonly used assessment instruments in Maryland.
Results from at least one of these assessment instruments (or other instrument if more appropriate), in conjunction with information obtained from other possible sources, including record review, interview, observation, and screening, will be used in providing the information required to complete the Child Outcomes Summary Form (COSF).
Ideally, the same tool would be used at entry and exit. However, use the tool that is most appropriate for the child.

28. Maryland’s Top Five The five most common assessment measures are:
Early Learning Accomplishment Profile (E-LAP)
Early Intervention Developmental Profile (EIDP)
Hawaii Early Learning Profile (HELP)
Battelle Developmental Inventory II
Ages and Stages

29. Key Questions for Accountability:
How much and what information will the tool provide about the child’s current level of functioning and the attainment of the 3 child outcomes?
Would providers intentionally or unintentionally give children higher scores than warranted, knowing that conclusions about program effectiveness would be based on the information?

30. Assessment Tool Lens
Each assessment tool carries its own organizing framework
Most are organized around domains but children integrate skills when performing functional activities
No right or wrong lens:
How can what the assessment tool provides be used to develop functional outcomes for young children?
How can what the assessment tool provides be
used as information about the achievement of
the three outcomes?

31. Critical Assumptions Related to the 3 Child Outcomes
Achievement of the outcomes is age based, children of different ages will demonstrate achievement in different ways
There are many pathways to competence for children with atypical development (using sign language, wheelchair). This seems obvious but can get lost when an assessment tool uses a different lens.

32. Remember This
Flexibility is required in using assessment tool results to make decisions about a child’s functioning on the outcomes
Team needs to decide what information from an assessment tool is relevant to understanding child’s functioning

33. More DEC Recommended Practices for Assessment
Assessment provides useful information
for intervention!
Assessment results are immediately useful for planning program goals and objectives
Good assessment practices assess not only immediate mastery of skill, but also whether child can demonstrate skill consistently across settings and with other people
Good assessment practices also assess the presence and extent of atypical child behavior that may be a barrier to intervention and progress

34. Using the Assessment Results for Outcomes
What is an Outcome:
Outcomes are the benefits that children and families experience as a result of early intervention
The three child outcomes, as measured for accountability purposes are different than IFSP outcomes in that:
The three child outcomes reflect global functioning in broad areas of development (social-emotional, knowledge and skills, getting needs met)
IFSP outcomes are specific to an individual child, based on his or her individual needs

35. What Makes an Outcome Functional? Functional Outcomes:
Let’s Look at Using Assessment Information to Develop Functional IFSP Outcomes
What Makes an Outcome Functional?
Functional Outcomes:
Refer to things that are
meaningful to the child in the
context of everyday living
Refer to an integrated series
of behaviors of skills that
allows for the child to achieve
the important everyday goals

36. Meaningful Outcomes
Meaningful outcomes promote a child’s functioning in three key foundations of early development - social interaction, mastery over environment, and engagement in learning - in ways specific to each family and child
(McWilliam, 2002; National Research Council, 2000).

37. Writing IFSP Family Centered, Measureable, Functional Outcomes
Family-centered outcomes are written to address:
The desires and goals a family has for them and their child
Created with the family after assessing what activities are meaningful to individual family members.
Includes not only activities of daily living but also the family's ability to participate in cultural and social experiences they hold valuable

38. Functional Outcomes
“Function” refers to those activities identified by the family that support the development of the child’s physical, social and psychological well being, i.e. the ability to feed oneself, to hold a toy, to communicate, or to play
Early intervention teams do not “treat” the primary diagnosis, i.e. cerebral palsy, autism, but instead treat the functional
delay or disability

39. Determining Appropriate IFSP Functional Outcomes
A few questions to consider:
What are the activities the family would like to do that are difficult?
Has the family given up on any family outings, experiences or chores?
What kinds of things would the family like for the child to do that would make life easier for them or more fun for the child?
Read more at MD’s IFSP Tutorial: Assisting Families to Select Functional Outcomes

40. Measureable IFSP Outcomes
Outcomes should be written so that all members of the team will understand when the outcome has been met.
Outcomes should address one of the following areas:
They should enhance the family’s ability to care for or to engage in activity with their child
They should enhance the child’s ability to participate in functional activities (feeding, dressing, moving in his environment, communicating, playing, etc.)
They should expand on activity settings in which the child already participates
Read more at MD’s IFSP Tutorial:
Measurable Criteria: How we Know we have Achieved Outcomes

41. Effective Strategies on an IFSP Identify How Functional Outcomes Will be Achieved
Each strategy should address:
Individualization- Do strategies build on child and family interests?
Context- Do strategies build on familiar places, people, and routines?
Mastery- Do strategies ensure generalization of a child’s actions and interactions in multiple settings and tasks?
Collaboration- Do strategies specify who will do what?
Read more at MD’s IFSP Tutorial:
Effective Strategies Identify How Functional Outcomes will be Achieved

42. Making a Difference! The results of improved assessment and
individual IFSP outcomes development
and services is two-fold:
First and foremost, it ultimately leads to enhanced services for children and families
Which, as a result, can’t help but positively impact Maryland’s outcome measurement results!

43. More to Come… Next week: Explore Maryland’s approach to
gathering child outcome data to
measure OSEP’s Child Outcomes
The role of the Child
Outcomes Summary Form
Explore other state’s approaches