1. Aligning Outcomes Measurement With Early Learning Guidelines/Standards Jennifer Olson (OR) Birth-5 Ruth Littlefield (NH) Birth-5 April 2006 – Albuquerque, NM

2. Questions to Guide the Session: What was the context in the state that led to the choice to align with the Early Learning Standards/Guidelines? How has the alignment with standards/guidelines influenced your selection of assessment tools? How will you report back on standards/guidelines and/or outcomes to local programs and to OSEP? What other implications have you seen as a result of deciding to align with your standards/guidelines? What do you feel are the pros and cons for the decision to align with your state standards/ guidelines?

3. 3 Oregon Early Childhood Foundations and Assessment Project OREGON DEPARTMENT OF EDUCATION, PORTLAND STATE UNIVERSITY, UNIVERSITY OF OREGON AND EDUCATORS FROM OREGON EARLY INTERVENTION/ EARLY CHILDHOOD SPECIAL EDUCATION, PRE-KINDERGARTEN/HEAD START AND CHILD CARE PROGRAMS

4. 4 PROJECT CONTACT INFORMATION Portland State University Joel Arick:503-725-4251, arickj@pdx.edu arickj@pdx.edu Ruth Falco:503-725-4486, falcor@pdx.edu falcor@pdx.edu Helen Young: 503-725-5207, youngh@pdx.edu youngh@pdx.edu Oregon Department of Education Nancy Johnson-Dorn: 503-378-3600 X2339, nancy.johnson- dorn@state.or.us nancy.johnson- dorn@state.or.usnancy.johnson- dorn@state.or.us Nancy Latini: 503-378-3600 X2361, nancy.latini@state.or.us nancy.latini@state.or.us Jennifer Olson: 503-378-3600, x2338, jennifer.olson@state.or.us jennifer.olson@state.or.us

5. 5 New Hampshire Early Learning Curriculum Guidelines and Preschool Child Outcomes for Young Children with Disabilities Ages 3-5 New Hampshire Department of Education/Bureau of Special Education/Preschool Special Education Office

6. 6 Question 1. What was the context in the state that led to the choice to align with the Early Learning Standards/Guidelines?

7. 7 Context for Alignment – Federal Requirements National focus on accountability, standards-based reform, & measurement of child outcomes (GPRA, 1993; IDEA, 1997 & 2004; NCLB, 2001) National focus on accountability, standards-based reform, & measurement of child outcomes (GPRA, 1993; IDEA, 1997 & 2004; NCLB, 2001) Good Start, Grow Smart (2002) required research-based, voluntary state guidelines for literacy, language, pre-reading, and numeracy, indicating what young children birth to 5 should know & be able to do prior to kindergarten Good Start, Grow Smart (2002) required research-based, voluntary state guidelines for literacy, language, pre-reading, and numeracy, indicating what young children birth to 5 should know & be able to do prior to kindergarten

8. 8 Context for Alignment – ODE Initiatives Oregon Department of Education (ODE) initiated development of child-based outcome standards (expectations for what children should know and be able to do prior to kindergarten) Oregon Department of Education (ODE) initiated development of child-based outcome standards (expectations for what children should know and be able to do prior to kindergarten) Guidelines in standards development Guidelines in standards development Link preschool outcome standards to statewide, Oregon K-12 Standards Link preschool outcome standards to statewide, Oregon K-12 Standards Focus on standards for all children, 0 – 5, including children in Part C, Part B 619, OPK, Head Start, & child care programs Focus on standards for all children, 0 – 5, including children in Part C, Part B 619, OPK, Head Start, & child care programs Add domains essential to early childhood to the domains in the K-12 standards (e.g., Approaches to Learning and Social Emotional) Add domains essential to early childhood to the domains in the K-12 standards (e.g., Approaches to Learning and Social Emotional)

9. 9 Context for Alignment Oregon Department of Education (ODE) formed advisory group of over 100 early childhood professionals to develop initial draft of Oregon Early Childhood Foundations (birth to 5) that align with Oregon Department of Education (ODE) formed advisory group of over 100 early childhood professionals to develop initial draft of Oregon Early Childhood Foundations (birth to 5) that align with K – 12 standards Oregon Early Childhood Foundations reviewed (Scott-Little, 2005) Oregon Early Childhood Foundations reviewed (Scott-Little, 2005) Revised Foundations drafted (ODE, 2005) Revised Foundations drafted (ODE, 2005)

10. 10 Context for Alignment Oregon Early Childhood Foundations developed in 8 domains Oregon Early Childhood Foundations developed in 8 domains Approaches to Learning Social Emotional Development Language and Literacy Development MathematicsScience Social Studies Physical Education and Health The Arts

11. 11 Context for Alignment – Legislature Requirements Needed easily understood, aggregated child outcome data to present to Oregon state legislators to support funding for Early Intervention/Early Childhood Special Education (EI/ECSE) and Oregon Pre-Kindergarten (OPK) Programs/Head Start Needed easily understood, aggregated child outcome data to present to Oregon state legislators to support funding for Early Intervention/Early Childhood Special Education (EI/ECSE) and Oregon Pre-Kindergarten (OPK) Programs/Head Start Oregon legislature charged Oregon Department of Education (ODE) with developing statewide assessment for EI/ECSE and OPK/Head Start Oregon legislature charged Oregon Department of Education (ODE) with developing statewide assessment for EI/ECSE and OPK/Head Start This Oregon Early Childhood (EC) Assessment needed to be linked to standards for school-aged children and provide the ability to monitor overall progress of children from pre-K through school age This Oregon Early Childhood (EC) Assessment needed to be linked to standards for school-aged children and provide the ability to monitor overall progress of children from pre-K through school age

12. 12 Context for Alignment in NH Preschool Special Educator’s Questions Preschool Special Educator’s Questions What is the general preschool curriculum? What is the general preschool curriculum? What standards/guidelines exist for us to follow? What standards/guidelines exist for us to follow?

13. 13 Context for Alignment in NH Creation of the Early Learning Curriculum Guidelines (ELCG) Project Creation of the Early Learning Curriculum Guidelines (ELCG) Project 619 funded 619 funded Create informed dialog across early childhood community Create informed dialog across early childhood community Strengthen connections between preschool special education and K-12 system Strengthen connections between preschool special education and K-12 system Develop shared language and frames of reference for communication Develop shared language and frames of reference for communication Enhance educational and developmental outcomes for young children with disabilities Enhance educational and developmental outcomes for young children with disabilities Support access to general early childhood curriculum and appropriate preschool activities for young children with disabilities Support access to general early childhood curriculum and appropriate preschool activities for young children with disabilities

14. 14 Context for Alignment in NH NH Department of Health & Human Services: Child Development charge to create early learning standards for children birth through 5 NH Department of Health & Human Services: Child Development charge to create early learning standards for children birth through 5 Domain Categories Domain Categories Indicators of progress Indicators of progress GSEG Grant: NH Cornerstones Project GSEG Grant: NH Cornerstones Project Subcommittee on preschool outcomes Subcommittee on preschool outcomes

15. 15 Question 2: How has the alignment with standards/guidelines influenced your selection of assessment tools?

16. 16 Alignment with Standards and Assessment Selection – Addressing Federal and State Requirements Office of Special Education Programs (OSEP) requires reporting of Office of Special Education Programs (OSEP) requires reporting of (a) % of children who reach or maintain functioning at level comparable to same-age peers (a) % of children who reach or maintain functioning at level comparable to same-age peers (b) % of children who improve functioning (but were not included in a) (b) % of children who improve functioning (but were not included in a) (c) % of children who do not improve functioning (c) % of children who do not improve functioning State requires State requires Assessment aligned with Early Childhood Foundations Assessment aligned with Early Childhood Foundations Child outcome data the Legislature will easily understand Child outcome data the Legislature will easily understand

17. 17 Alignment with Standards and Assessment Selection – Current Program Practices ODE currently requires EI/ECSE Programs to use one of four different curriculum-based assessments (CBAs) annually for each child ODE currently requires EI/ECSE Programs to use one of four different curriculum-based assessments (CBAs) annually for each child Each OPK/Head Start Program uses one of at least two different curriculum-based/criterion- referenced assessments with all children Each OPK/Head Start Program uses one of at least two different curriculum-based/criterion- referenced assessments with all children ODE continues to value the use of comprehensive CBAs to link assessment to program planning ODE continues to value the use of comprehensive CBAs to link assessment to program planning

18. 18 Alignment of Standards and Assessment Selection – Concerns with CBAs Not feasible to aggregate data from the various CBAs Not feasible to aggregate data from the various CBAs Not feasible to require all EI/ECSE and all OPK/Head Start Programs to use a common CBA Not feasible to require all EI/ECSE and all OPK/Head Start Programs to use a common CBA CBAs CBAs Not designed to provide aggregated data on developmental progress Not designed to provide aggregated data on developmental progress Not designed to compare child progress to that of typical peers Not designed to compare child progress to that of typical peers Previous attempts to use CBA data to share outcomes with Legislature were unsatisfactory Previous attempts to use CBA data to share outcomes with Legislature were unsatisfactory CBAs do not hold a clear relationship to Oregon Early Childhood Foundations and do not include all Foundation Domains or Areas CBAs do not hold a clear relationship to Oregon Early Childhood Foundations and do not include all Foundation Domains or Areas

19. 19 Alignment of Standards with Assessment - Oregon Early Childhood Assessment ODE professionals decided federal and state needs will most effectively be met through development of the Oregon Early Childhood Assessment System, developed to be compatible with current CBAs while aligning with Oregon Early Childhood Foundations and Oregon K-12 Standards ODE professionals decided federal and state needs will most effectively be met through development of the Oregon Early Childhood Assessment System, developed to be compatible with current CBAs while aligning with Oregon Early Childhood Foundations and Oregon K-12 Standards ODE partnered with researchers at Portland State University (PSU) and University of Oregon (UofO) to develop the system ODE partnered with researchers at Portland State University (PSU) and University of Oregon (UofO) to develop the system

20. 20 Alignment of Guidelines with Assessment Tools Tools not yet selected: Assessment Fair on May 18th NH Early Learning Guidelines (Good Start Grow Smart Framework, not developmental skills checklist or curriculum Articulate a common visions and commitment to children Provide a map of what children should know, understand and be able to do Basis for what adults working with young children should know, understand and be able to do

21. 21 Alignment of Guidelines with Assessment Tools Created as a companion document: NH Early Learning Curriculum Guidelines and Preschool Outcomes Guiding Principals Written in educational rather than early childhood terms Mirrors Early Learning Guidelines Preschool Outcomes presented with sub-outcomes Expands on preschool indicators and what you may observe Connects Outcomes and sub-outcomes to Domain areas

22. 22 Question 3. How will you report back on standards and/or outcomes to local programs and to OSEP?

23. 23 Process to Report on Outcomes: Statewide EC Assessment System Portland State University & University of Oregon are collaborating with the Oregon Department of Education to develop an assessment that measures the skills and abilities identified in the Early Childhood Foundations for all programs that serve birth to age 5 children This assessment system will be used to report back to local programs and OSEP

24. 24 Development of Assessment Items Research regarding early childhood development and important skills in each of the foundation domains and areas was reviewed Research regarding early childhood development and important skills in each of the foundation domains and areas was reviewed Numerous research-based assessments (both standardardized and non-standardized) were reviewed to identify important skills across development from birth to 6 years old Numerous research-based assessments (both standardardized and non-standardized) were reviewed to identify important skills across development from birth to 6 years old 128 items across 16 foundation areas were developed based on the research to assess a sample of skills or abilities from the foundation areas 128 items across 16 foundation areas were developed based on the research to assess a sample of skills or abilities from the foundation areas Items were selected that are: Items were selected that are: Observable Observable Functional Functional Developmentally appropriate (Sampling skills from 6 months – Developmentally appropriate (Sampling skills from 6 months – 6 years of age)

25. 25 Criteria Used by Developers to Design the Assessment Developmentally appropriate Developmentally appropriate Conducted in natural environments Conducted in natural environments Activity-based assessment (play, table-time, snack, group) Activity-based assessment (play, table-time, snack, group) Measure progress over time Measure progress over time Aggregate/combine data across children/programs Aggregate/combine data across children/programs Relate to currently used CBAs (e.g., AEPS, HELP, Carolina, Galileo, Creative Curriculum) Relate to currently used CBAs (e.g., AEPS, HELP, Carolina, Galileo, Creative Curriculum) Valid and reliable Valid and reliable Easy for staff to use Easy for staff to use

26. 26 EC Assessment System On-Line assessment system On-Line assessment system Uses combination of Checklist Rating Scale and Direct Observation of child during typical routines Uses combination of Checklist Rating Scale and Direct Observation of child during typical routines Provides users with a student report Provides users with a student report Interactive computer functions make data entry easy and provide immediate information to staff Interactive computer functions make data entry easy and provide immediate information to staff

27. 27 PILOTSTUDY: Demonstration of how data can be reported to local programs and OSEP

28. 28 Pilot Study Pilot Study to “try-out” the assessment was conducted from February 2005 to September 2005 Pilot Study to “try-out” the assessment was conducted from February 2005 to September 2005 Educators gave children a pre-assessment and a post-assessment Educators gave children a pre-assessment and a post-assessment (each child’s pre and post-assessments were 2-4 months apart) (each child’s pre and post-assessments were 2-4 months apart)

29. 29 Pilot Study Purposes To determine if assessment items would measure progress of children over time in several settings To determine if assessment items would measure progress of children over time in several settings To get feedback on assessment items, assessment process, and the on-line system To get feedback on assessment items, assessment process, and the on-line system To conduct preliminary validity and reliability studies To conduct preliminary validity and reliability studies To develop initial process for reporting on child outcomes to OSEP To develop initial process for reporting on child outcomes to OSEP

30. 30 Demographics Pilot Study: Preliminary Data Number of Children Assessed: Child Care (no-disabilities) 239 EI/ECSE 140 Head Start (non-EI/ECSE)108 Total:487

31. 31

32. 32 Internal Consistency Reliability All Subjects in Pilot Study less than 60 months Domain Area Correlation Matrix - Total Pre assessment scores correlated with each domain Approaches to Learning.8480 Creative Arts/Health Status & Practices.7960 English Language Arts.8903 Physical Education.7774 Social Emotional Development.8025 English.7527 Math.7874 Science.7650 Standardized Item Alpha (N=422) Reliability Coefficients – 9 items.9413

33. 33 Demographics: EI/ECSE Disability Type NumberPercent Autism3323.6% Communication117.9% DevelopmentalDelay7352.1% Hearing Impair 75% Orthopedic Imp 96.4% Other Health Im 21.4% Not available 53.6%

34. 34 Federal Outcomes Data for Part C & B: Monitoring Criteria (a) Percent of children who reach or maintain functioning at a level comparable to same-aged peers (a) Percent of children who reach or maintain functioning at a level comparable to same-aged peers (b) Percent of children who improve functioning (not included in a) (b) Percent of children who improve functioning (not included in a) (c) Percent of children who did not improve functioning (c) Percent of children who did not improve functioning

35. 35 Pilot Study Results: A) Comparison with Same-aged Peers Preliminary Aggregate Data Preliminary Aggregate Data Comparable to same-aged peer= within one standard deviation of same age peers (child care subjects) Comparable to same-aged peer= within one standard deviation of same age peers (child care subjects) Pre-post intervention period was 4 months Pre-post intervention period was 4 months * Post data collection was not available for all subjects at this time Percent EI/ECSE Children Comparable to Same-aged Peer AgeGroupPre(n=100)Post*(n=61) 36-47 mths 25%40% 48-59 mths 33%43%

36. 36

37. 37 % of Children in EI/ECSE Making Progress in Foundation Areas in 4 months or less Engagement & Persistence (App To Learning) 29/50 – 58% Reasoning & Problem Solving (App To Learning) 30/41 – 73% Listening & Understanding (Eng Lang Arts) 25/48 – 52% Speaking & Communicating (Eng Lang Arts) 32/53 – 60% Fine Motor (Physical Education) 39/58 – 67% Gross Motor (Physical Education) 23/26 – 88% Cooperation & Self Control (Social Emotional Dev) 25/41 – 61% Social Relationships (Social Emotional Dev) 32/46 – 70% Arts, Movement, Music & Dramatic Play (Cr. Arts) 38/59 – 64% Hygiene, Nutrition & Personal Care (Health) 17/35 – 49%

38. 38 % of Children in EI/ECSE Making Progress in Foundation Areas in 4 months or less (Developmental Level of Items: 36 – 60 months) Phonological Awareness (English) 12/33 – 36% Alphabet Knowledge (English) 27/44 – 61% Numbers & Operations (Math) 23/35 – 66% Patterns & Measurement (Math) 25/34 – 73% Matter (Science) 21/39 – 54% Organisms & Heredity (Science) 26/38 – 68%

39. 39 Teacher Perceptions of Online Assessment System Survey Results (N=48) StatementStronglyDisagreeDisagreeAgreeStronglyAgree I could use the online computer assessment system 2%4%41%52% I could use the 7-step online assessment process 0%7%39%54% I could enter and add new data into the system 0%7%52%41%

40. 40 What We Learned From the Pilot Study All agencies (Preschools/Child Care, Head Start, and EI/ECSE) were supportive of the project All agencies (Preschools/Child Care, Head Start, and EI/ECSE) were supportive of the project The on-line system efficiently allowed the assessment to be implemented across programs The on-line system efficiently allowed the assessment to be implemented across programs All programs had access to the online system All programs had access to the online system In 4 months or less, all children in EI/ECSE, Head Start and Childcare programs showed progress In 4 months or less, all children in EI/ECSE, Head Start and Childcare programs showed progress Child Care/preschool sites were very excited about being included in this project Child Care/preschool sites were very excited about being included in this project Preliminary pilot data appears to validate the use of the assessment to monitor child development Preliminary pilot data appears to validate the use of the assessment to monitor child development

41. 41 NH Planning to Report Back Still under construction Strong stakeholder and state commitment to make this useful at local, state and federal level Working with publishers of Assessment tools to: minimize labor, maximize usefulness of information to staff, provide state level data that can easily be reported to OSEP, be a cost-effective as possible NHSES Statewide special education data system

42. 42 Question 4. What other implications have you seen as a result of deciding to align with your standards?

43. 43 Implications of Alignment of Standards and Assessment Assessment data will be more valued by Oregon Legislature as well as OSEP Assessment data will be more valued by Oregon Legislature as well as OSEP Increased knowledge of quality in EC settings Increased knowledge of quality in EC settings Linking the EC Assessment to K-12 standards and assessments may enable longitudinal progress monitoring Linking the EC Assessment to K-12 standards and assessments may enable longitudinal progress monitoring

44. 44 Implications of Alignment with Guidelines Family input indicates this is useful for parents to see their child’s progress Promotes better quality community child care/preschool options for all children Strengthens the discussion with policy makers in the educational community Reinforces the bridge between Part C and 619

45. 45 Implications of Alignment with Guidelines Brought preschool special education to the child care table in a very positive manner Provides a foundation and framework for the conversations about the two burning questions for preschool special educators: What is the general preschool curriculum? What standards/guidelines exit for us to follow?

46. 46 Question 5. What do you feel are the pros and cons for the decision to align with your state standards?

47. 47 Pros Assessment will meet both Federal and State needs Assessment will meet both Federal and State needs Assessment will address common curriculum goals for infants, toddlers and preschool children across EI/ECSE, OPK/Head Start, and child care Assessment will address common curriculum goals for infants, toddlers and preschool children across EI/ECSE, OPK/Head Start, and child care Assessment will link EI/ECSE, OPK/Head Start, and child care outcomes to outcomes in K-12 school programs Assessment will link EI/ECSE, OPK/Head Start, and child care outcomes to outcomes in K-12 school programs GSEG grant, in addition to State funding, will provide opportunity to further field-test unique measure to meet Federal and State needs GSEG grant, in addition to State funding, will provide opportunity to further field-test unique measure to meet Federal and State needs

48. 48 Cons Perception that assessment is not as credible as other assessments Perception that assessment is not as credible as other assessments Perception that CBM could be used instead of adding another asessment tool. Perception that CBM could be used instead of adding another asessment tool. Development of a valid and reliable measure, aligned with Oregon Early Childhood Foundations, requires time and funding Development of a valid and reliable measure, aligned with Oregon Early Childhood Foundations, requires time and funding

49. 49 EC ASSESSMENT SYSTEM If time or for those interested, we can view the on-line Oregon EC Assessment system If time or for those interested, we can view the on-line Oregon EC Assessment system

50. 50 Pros Resource documents have more stakeholder buy-in Final products are more comprehensive and useful More useful for a wider variety of players (families, early care and special education providers, administrators, etc) One framework for all rather than parallel systems

51. 51 Pros Based on Early Childhood best practices and improvement rather than on compliance and federally reporting Brought Preschool Special Education to the Child Development table in a positive, pro-active way

52. 52 Cons More time to complete