Presentation on theme: "What is Indicator 14 and why is it important?"— Presentation transcript:
1 What is Indicator 14 and why is it important? Dawn A. Rowe, NPSO CoordinatorRhode Island Advanced Transition TrainingProvidence, RIJanuary 26, 2011National Post-School Outcomes Center
2 Session Objectives To learn what I-14 is and it’s utility To learn about federal data collection and reporting efforts for post-school outcomesTo learn about Rhode Island’s data collection and reporting efforts for post-school outcomesTo introduce strategies that have evidence to support positive outcomes for youth with disabilities
3 The National Post-school Outcomes Center [NPSO] A national technical assistance & dissemination center funded by the U.S. Department of Education Office of Special Education Programs. We help State Education Agencies develop practical, yet rigorous data collection systems to describe the further education and competitive employment experiences of youth with disabilities as they transition from high school to adult life.Akzidenz-Grotesk BQ Reg
4 Purpose of IDEATo ensure that all children with disabilities have available to them a free and appropriate public education that emphasizes special education and related services designed to meet their unique needs and prepare them for further education, employment, and independent living.IDEA Regulations §300.1(a)
5 Federal Reporting Requirements Under IDEA, States are required to submit:State Performance Plan (SPP)Annual Performance Report (APR)
6 Federal RequirementsState’s performance plan and annual report are based on 20 Part B indicators 4 specific to secondary transition:1Percent (%) of youth who graduate from high school2Percent (%) of youth who drop out of high school13Percent (%) of youth with IEPs age 16 or above with an IEP that includes specified transition components (e.g., postsecondary goals)14Percent (%) of youth who achieve post-school outcomesAkzidenz-Grotesk BQ Reg
7 Critical Interrelationships for Achieving PSO Staying in school(Indicator 2)Quality IEPs(Indicator 13)Positive post-school outcomes(Indicator 14)Graduating(Indicator 1)Kohler (NSTTAC), 2007
8 Part B SPP/APR Requirements for Indicator 14 Percent of youth who are no longer in secondary school, had IEPs in effect at the time they left school, and were:Enrolled in “higher education”In “competitive employment”Enrolled in “some other postsecondary education or training”In “some other employment”
9 Competitive Employment Other Postsecondary Education or Training PSO Definitionsfull- or part-timecommunity college (2-year program)college/university (4- or more year program)one complete termHigher Educationpay at or above the minimum wagesetting with others who are nondisabled20 hours a weekfor at least 90 days (includes military)Competitive Employmentat least 1 complete termeducation or training program (e.g., Job Corps, adult education, workforce development program, vocational technical school which is less than a 2-year program)Other Postsecondary Education or Trainingpay or been self-employedat least 90 days.This includes working in a family business (e.g., farm, store, fishing, ranching, catering services, etc.)Other Employment
10 Indicator 14 for Federal Reporting Click to edit Master text stylesStates must report a percentage for each A, B, and C in the SPP/APR and to the public. States must also provide the actual numbers for each 1, 2, 3 and 4.States are not required to report those former students who were “doing something other than” or “not engaged” in 1, 2, 3, or 4. However, those leavers will be part of the “total respondents”.
11 Findings from state data collection efforts are used to: NationallyRhode IslandReport at the national, state, and local levelsGuide and improve transition programs for transition age youth with disabilitiesContinue improvement in data collection focusing on improving representativenessDevelop capacity to improve outcomes for specific populations of youth who are not engaged at the same rate as others after high school (e.g., youth with ED, AA, Hispanic) . (FFY 2010 SPP)
12 Collecting Post-school Outcomes Data Who are the data collected on?What data are collected?How are the data collected?When are the data collected?Who collects the data?
13 Who are data collected on? NationallyRhode IslandThose with IEPs who leave high school:With a diploma – regular or modifiedWith a certificateBy aging outBy leaving early /dropping outYouth with IEPs who leave high school by:Graduating with a regular diplomaAge outLeft school early (i.e., dropped out)Akzidenz-Grotesk BQ RegChallenge: Finding leavers one year out of school
14 What data are collected? NationallyRhode IslandIn school:Demographic dataDisabilityGenderRace/ethnicityAgeMethod of exitOne year out:Higher educationCompetitive employmentOther postsecondary education or trainingOther employmentIn school:Demographic dataDisabilityGenderRace/ethnicityAgeMethod of exitStudent ContactsOne year out:Higher educationCompetitive employmentOther postsecondary education or trainingOther employmentAkzidenz-Grotesk BQ Reg
15 How are data collected nationally? Census v. SampleMethod of CollectingEvery district over the life of the SPPADM > 50KRepresentative sample DisabilityGenderRace/ethnicityAgeSurveyPhoneMailedFace-to-faceWeb- or Internet-basedExtant databaseAkzidenz-Grotesk BQ RegChallenge: Finding leavers one year out of school
16 How are data collected in Rhode Island? CensusMethod of CollectingAttempt to contact every leaver in every district over the life of the SPPUse unique identifiersSurveyCombination of Phone andOnline survey (district personnel complete)Akzidenz-Grotesk BQ RegChallenge: Finding leavers one year out of school
17 When are data collected? NationallyRhode IslandApril through SeptemberWhen youth have been out of school for at least one yearSameAkzidenz-Grotesk BQ RegChallenge: Finding leavers one year out of school
18 Who collects these data? NationallyRhode IslandSEA or LEA staffTeachers and support staffAdministratorsOutside contractorUniversitySurvey CenterStudent’s last known case manager (certified special education teacher)LEA Census ClerksSpecial Education Administrators(Regional training provided via WebEx)Akzidenz-Grotesk BQ RegChallenge: Finding leavers one year out of school
19 Results of State’s Efforts: Outcomes for Students with Disabilities as measured by Indicator 14 United States Department of Education, Office of Special Education Programs (2011).Part B State Performance Plan/ Annual Performance Report 2011 Indicator Analyses.
21 Rhode Island’s PSO Data These data represent youth who left school during the 2009-2010 school year. Of the 1579 leavers 1101 (70%) responded to the PSO survey.The responders were found to represent youth across the state based on disability category, gender, and minority status.Dropouts were slightly underrepresented.Of those who responded, 62% reported being enrolled in higher education, competitively employed, enrolled in some other postsecondary education or training program, or other employment since leaving high school.
22 How well do those who responded represent all leavers in 2009-2010? Overall Response Rate: 70%Is our sample size large enough to draw meaningful conclusions?States are asked to describe in the SPP how well their data obtained from those who respond to the survey represent the total leavers in the state. For RI in , of the 1579 total leavers (census), 1101 responded to the survey, for a 70% response rate.Remember, the question we want to answer is how well do those who responded represent all leavers in ? To answer this question we look beyond the response rate and compare the proportion of respondents to leavers on key characteristics: disability, gender, minority status, and method of exit. A difference of +-3% in the proportions indicates an importance difference between those who responded and the total leavers.One category exceeds that 3% difference in proportions: youth of who dropout.Differences greater than ± 3% are important
23 Challenges Finding leavers 1-year out of school Contacting leavers 1-year out of school
24 Rhode Island’s Method of Exit Of the 1100 youth who responded across the state… Facilitator's Notes:(Purpose #2)This chart shows what percentage of state youth who responded to the survey exited school by graduating, dropping out, aging out or other means of exit.If numbers do not total 100%, you should use a chart with columns (i.e., bars), not a pie chart.In most cases, the number of youth who graduate and dropout will not total 100% of the youth who left school in a year for several reasons:Differences in the definitionsDifferences in the data used to calculate graduation and dropout ratesSome youth die, or move out of State, and may not be categorized as either.To reach 100% on the pie chart, you may need to include categories of other methods of exit; for example, youth who reach maximum age or age out, or Other, as defined by the State or district.Numbers in the chart are place holders.Change the numbers in the graph by doing the following:-Double click on a slice of the pie graph to open the Excel file-When the Excel spreadsheet opens:-Click on the tab labeled “Sheet 1”-Change the numbers you want to change-Click on the tab labeled Pie Chart-Close the spreadsheet by clicking outside of the Excel boxData Source: RI PSO SY
25 Rhode Island’s Engagement Rate: Of the 1100 respondents Convert percents to numbersChange dataRI PSO SY
26 Let’s look at examples of how data can be disaggregated. So What??You are probably asking:What do these data tell me?What use are these data?How can these data be used to improve programs for youth with disabilities?Engagement rate alone is insufficient to inform program decisions.The engagement rate should be disaggregated by subgroups of students based on key characteristics, such as demographic data, geographic location, or other relevant information.Let’s look at examples of how data can be disaggregated.
27 Are Males & Females engaged at the same or similar rate? Brainstorm questions to answer about the engagement rates of males and females. Do we do better with certain disability groups or genders?Facilitator's Notes:(Purpose #3)Ask the stakeholder group to brainstorm the questions they would like to answer about the engagement rates of males and females.You may ask people to write questions on sticky notes and read them aloud or display them on chart paper.Place the group’s questions on chart paper. Using the data, answer the questions.Examples of questions:Are males and females competitively employed at approximately the same rate? If not, which group is employed at a higher rate?What do you think contributes to the higher employment rate for some groups?What could the district do to promote competitive employment for groups with a lower employment rate?
28 Percent of Males & Females Engaged Engagement RateFacilitator's Notes:(Purpose #3)Take a couple of minutes to make observations about what’s displayed.Record observations on chart paper.Of the questions the group wants to answer, do the data answer any of the group’s questions?Numbers in the chart are place holders.Change the numbers in the graph by doing the following:Right click on a bar in the graphSelect “edit data”When the Excel spreadsheet opens:Change the numbers you want to changePush the return key on the keyboardClose the spreadsheet28Data Source: RI PSO SY28
29 Are youth with various disabilities engaged at the same or similar rate? Brainstorm questions to answer about the engagement rates of youth with different disabilities. Do we do better with certain disability groups or genders?Facilitator's Notes:(Purpose #3)Ask the stakeholder group to brainstorm the questions they would like to answer about the post-school outcomes for youth of different disability categories.You may ask people to write questions on sticky notes and read them aloud or display them on chart paper.Place the group’s questions on chart paper.Using the data, answer the questions.
30 Engagement Rate by Disability Categories AO = All Other DisabilitiesEngagement RateFacilitator's Notes:(Purpose #3)Take a couple of minutes to make observations about what’s displayed.Record observations on chart paper.Of the questions the group wants to answer, do the data answer any of the group’s questions?Numbers in the chart are place holders.Change the numbers in the graph by doing the following:Right click on a bar in the graphSelect “edit data”When the Excel spreadsheet opens:Change the numbers you want to changePush the return key on the keyboardClose the spreadsheet30Data Source: RI PSO SY30
31 Are youth from various race or ethnic groups engaged at the same or similar rate? Brainstorm questions to answer about the engagement rates of youth from different backgrounds. Do we do better with certain groups of youth?Facilitator's Notes:(Purpose #3)Ask the stakeholder group to brainstorm the questions they would like to answer about the post-school outcomes for youth of different disability categories.You may ask people to write questions on sticky notes and read them aloud or display them on chart paper.Place the group’s questions on chart paper.Using the data, answer the questions.
32 Engagement Rate by Ethnicity Categories AO = All Other DisabilitiesEngagement RateFacilitator's Notes:(Purpose #3)Take a couple of minutes to make observations about what’s displayed.Record observations on chart paper.Of the questions the group wants to answer, do the data answer any of the group’s questions?Numbers in the chart are place holders.Change the numbers in the graph by doing the following:Right click on a bar in the graphSelect “edit data”When the Excel spreadsheet opens:Change the numbers you want to changePush the return key on the keyboardClose the spreadsheet32Data Source: RI PSO SY32
33 How can disaggregated PSO data be used? What are other states doing?
34 How other states are using PSO data to inform transition programs NC revised their PSO survey to collect more programmatic data to examine whether in-school programs and services correlate with better post- school outcomes.MD is sharing PSO data with the state Interagency Council to inform and improve the cross-agency collaborative services provided while youth are in-school.
35 How other states are using PSO data to inform transition programs SD compared outcomes of students who participated in certain transition programs to those who did not to determine program effectiveness.
36 Let’s Look Closer: South Dakota Question: Are youth who participate in 4 state specific in- school programs more likely to be engaged one-year out of school than youth who do not participate in these programs?Programs: Catch The Wave, Youth Leadership Forum, Self-Advocacy, & Project SkillsUsed an odds ratio to measure effect size, describing the strength of association between two dichotomous values: youth engaged (i.e., working or enrolled) versus youth not engaged (i.e. not working or enrolled).
37 OVERALL for 08-09: Odds Ratio of engagement for students who participated in Catch The Wave, Youth Leadership Forum, Self-Advocacy, or Project Skills# engaged# not engagedParticipate in CTW, YLF, Self-Advocacy, or Project Skills17719Did not participate in CTW, YLF, Self-Advocacy, or Project Skills12026177/19120/26 = 9.316/4.615 OR of 2.03n= 342Those who participated in these programs were 2.03 times more likely to be engaged in work or school than those who did not participate in these programs.177/19120/26 = 9.316/4.615 OR of 2.03
38 Empirical EvidenceSouth Dakota has empirical evidence that youth who participated in Catch The Wave, Youth Leadership Forum, Self-Advocacy, or Project Skills were consistently more likely to be positively engaged 1-year out of high school than youth who did not participate in these programs.With sufficient data, you could do something similar to examine whether the programs offered in RI or your district contribute to positive post-school outcomes.
39 The Logic of Using PSO Data for Program Improvement Data CollectionPositive Post-school OutcomesData CollectionData UsePositive Post-school outcomes
40 Thinking about Transition Related Program Improvement In what areas are youth with disabilities doing well?What areas need improvement?What is the state/district doing well?What does the state/district need to improve?Facilitator's Notes:(Purpose #4)Place each question on individual sheets of chart paper. Ask the group to answer each question.Depending on group size, you may want them answer on sticky notes, to allow individual think time, or go one by one around the group.The goal with this slide is to start to look critically at the district or school program and identify how the program contributes to positive post-school outcomes and how it does not.If you have several areas that need improvement, you may need to prioritize the needs. There are several criteria that can be used to prioritize needs. Examples of criteria include:Resource investment (people, money, or equipment/materials): no additional resources, few resources, or significant resourcestime investment: changes can be addressed immediately, in the short-term (defined as ….), or in the long-term (defined as …)Degree of change anticipatedNumber of youth/families affected
41 How do you make the connection between post-school outcomes of youth with disabilities and practices implemented in the district, school, or classroom?
42 Evidence-Based (Special)Education… …is a decision-making process for deciding what to teach based on:frameworks or principlesbest available research evidenceprofessional judgmentstudent needs and valuesA third response has been to look at EBP as a more generic, decision-making process…two examples…1. See TEC, Jan/Feb, 2010, pages 42-53, “Knowledge-to-Action Guides”(Buysse, Wesley, Snyder, & Winton, 2006; Cook, 2010 Detrich, Spencer, & Slocum, 2009;Turnbull et al., 2010; Cook, 2010)
43 Create Individualized Interventions Adapt Interventions Frameworks Provide a Systematic Basis for using Professional Judgment to:Select InterventionsCreate Individualized InterventionsAdapt Interventions(Spencer, 2009)
44 Student-Focused Planning Student Development Family Involvement A Framework for Secondary Transition: The Taxonomy for Transition ProgrammingStudent-Focused PlanningStudent DevelopmentFamily InvolvementInteragency CollaborationProgram StructureProvides a framework to guide secondary transition instructional planning and practices(Kohler, 1996)
45 Need to Consider Inputs Outputs Outcomes OAVSNP, 2010 Keynote Deanne Unruh, Ph.D.,
46 Inputs Outputs Outcomes OAVSNP, 2010 KeynoteWhat resources do we have in place or need?InputsWhat strategies/ evidence-based practices will we use?OutputsWhat student outcomes do we need to improve?OutcomesDeanne Unruh, Ph.D.,
47 Focus on What you Can Control Post-School OutcomesPredictorsPracticesLesson Plans
48 PREDICTORS Post-School Outcomes Independent Living Employment EducationIndependent Living
49 In-School Predictors by Post-School Outcome Area Predictors/OutcomesEducationEmploymentIndependent LivingAcademic/General EducationxCareer AwarenessCommunity ExperiencesExit Exam Requirements/High School Diploma StatusInteragency Collaboration*Occupational CoursesPaid Work Experience*Parental InvolvementSchool Integration*
50 In-School Predictors by Post-School Outcome Area Predictors/OutcomesEducationEmploymentIndependent LivingSelf-Advocacy/Self-Determination*xSelf-Care/Independent LivingSocial SkillsStudent Support*Transition Program*Vocational Education*Work Study*
51 Linking Post-School Outcome Data to Evidence-based Practices and Predictors How do I do this?
52 How do Predictors Apply to Transition Planning and Instruction? OAVSNP, 2010 KeynoteHow do Predictors Apply to Transition Planning and Instruction?Provides practitioners information about secondary transition program characteristics that have been empirically linked to improved post- school success for students with disabilitiesCan be used to:develop and expand programsevaluate existing programsCan help IEP teams design annual IEP goals and transition services that are more likely to help students achieve their stated post-school goalsDeanne Unruh, Ph.D.,
53 Predictors of Post-School Success for Students with Disabilities Youth Transition Program (YTP)Transition planning focused on post-school goalsInstruction in academic, vocational, independent living, and personal social areasPaid job training while in the programAssistance to secure employment or enter postsecondary education upon leaving the programFollow up support for up to 2 years after leaving the programTransition to independence Process System (TIP)Relies on a transition facilitator/specialistYouth-centered and focused on youth engagementTransition plans developed that access various service agencies based on unique needs of individual
54 Sample State’s Engagement Rates Of the 400 youth who responded to the interview/survey across the state…Facilitator's Notes:(Purpose #2)This chart shows what percentage of state youth who responded to the survey were in each engagement category (i.e., higher education, competitive employment, postsecondary education or training, and other employment).If numbers do not total 100%, you should use a chart with columns (i.e., bars), not a pie chart.In most cases, the categories of higher education, competitive employment, other employment, and postsecondary education or training will not total 100% of youth who left school. To reach 100%, you may need to include an additional category representing the remaining percentage of youth surveyed. In this example, we label the category not engaged to represent those youth who are not counted in one of the other categories. The state may prefer to use labels such as Other or Neither. If these categories are used in the State, define who is represented in these categories.For example, Other may include youth who are incarcerated. Neither may include youth who are not engaged in any positive post-school outcome.Numbers in the chart are place holders.Change the numbers in the graph by doing the following:-Double click on a slice of the pie graph to open the Excel file-When the Excel spreadsheet opens:-Click on the tab labeled “Sheet 1”-Change the numbers you want to change-Click on the tab labeled Pie Chart-Close the spreadsheet by clicking outside of the Excel boxData Source: Sample PSO
55 TIP Youth Transition Program Interagency Collaboration Increased Competitive EmploymentTIPYouth Transition ProgramInteragency Collaboration
56 Rhode Island’s Engagement Rate: Of the 1100 respondents Convert percents to numbersChange dataRI PSO SY
57 What is Rhode Island doing that could lead to higher engagement after high school? Community-based Training (e.g., The Sports Exchange)Post-School EngagementRhode Island Transition AcademiesCareer Development ProjectRhode Island Guides for Students in TransitionRhode Island “My Transition” websiteCheck out the new Rhode Island web site for studentsBeing informed is the best way to advocate for the services and programs you may need to effectively transition from high school to employment, post-secondary education & training and community living. This page is divided into sections that may help you navigate the transition process and meet with success in the transition from school to adult lifeUnderstanding the Transition ProcessThe Rhode Island Department of Education works closely with school districts, public adult services agencies, parent organizations, universities and colleges to provide current and useful information. One tool that many young adults have found useful is and on-line tool called Youthhood. At Youthhood you will find tools to assist you in planning for your life after high school. It is a great place to start.The RI Transition Academies are a unique educational program that offers students with disabilities, 18 to 21 years old, the opportunity to complete their high school education on a college campus and/or in various employment and community settings. It is geared toward the student who has completed or nearly completed the academic courses necessary for graduation and would benefit from an additional one or two years to acquire the functional life skills, vocational skills and social skills necessary to be better prepared for adult life. Potential student candidates must be in special education with an Individual Education Plan (IEP), be currently enrolled in school and near completion of their academic program. Candidates must also be eligible for services through the Office of Rehabilitation Services and must have a current application with the Office of Rehabilitation Services.College Resources & planing: This directory helps students who receive supports through Individual EducationPrograms (IEPs), or 504 Plans, to learn about post-secondary education opportunities. This guide will help to:- Plan and prepare for post-secondary education- Identify available supports to aid access to post-secondary education- Find disability-related information about colleges in Rhode IslandEmployment Resources and Planning: Designed for students in transition, this guide walks them through the steps of planning a career and conducting an employment search. The workbook format encourages students to enter their own information for future use. This guide complements School to Career training programs and curricula.Living & particpating in your Community: this guide will help students plan and access the supports they need to live and participate in their communities.Rhode Island Transition CouncilRegional Transition Centers
58 Predictors Post-School Outcomes Evidence-based Practices Lesson Plans “While the evidence-based practices were designed to teach students specific transition-related skills, to date, the experimental literature has not attempted to measure the impact of these skills on post-school outcomes.”“Therefore, NSTTAC began examining correlational research on in-school predictors of post-school success.”Lesson Plans
59 Have used rigorous research designs Have demonstrated a record of success for improving student outcomesHave undergone systematic review process using quality indicators to evaluate level of evidenceEvidence-Based PracticesResearch-Based PracticesDeveloped based on researchHave demonstrated limited success for effectivenessHave used a ‘weak’ research designPromising PracticesAre not based on researchNo data have been collectedBased on Anecdotal evidence and professional judgment existsUnestablished PracticesEvidence-Based PracticesHave used rigorous research designsHave demonstrated a record of success for improving student outcomesHave undergone systematic review process using quality indicators to evaluate level of evidenceResearch-Based PracticesPromising PracticesDeveloped based on researchHave demonstrated limited success for effectivenessHave used a ‘weak’ research designUnestablished PracticesAre not based on researchNo data have been collectedBased on Anecdotal evidence and professional judgment exists(Helsel, Hitchcock, Miller, Malinow, & Murray, 2006; Lembke & Stormont, 2005; Twyman & Sota, 2008)
60 Let’s examine a few of these practices Using Self-Advocacy Strategy to teach Student involvement in the IEPSelf-Advocacy Strategy (SAS) consists of 5 steps taught over a series of seven acquisition and generalization stages:I – InventoryP – ProvideL – Listen and RespondA – Ask QuestionsN – Name your goals
61 Let’s examine a few of these practices Using Whose Future is it Anyway? to increase self-determination skills and transition planning knowledgeWhose Future is it Anyway? is a student directed transition planning curriculum comprised of six sections:Having self-awareness and disability awarenessDecision making about transition-related outcomesIdentifying and securing community resources to support transition servicesWriting and evaluating goals and objectivesCommunicating effectively in small groupsDeveloping skills to become and effective team member, leader, or self-advocateAvailable: partnerships/zarrow/self-determination-education-materials/whos-future-is-it- anyway.html
62 How does EBP Apply to Transition Planning and Instruction? OAVSNP, 2010 KeynoteHow does EBP Apply to Transition Planning and Instruction?Provides teachers information about secondary transition evidence-based practices for teaching students with disabilitiesCan be used to:Support IEP goals and objectivesSupport skill developmentDeanne Unruh, Ph.D.,
63 Resources Jane Slade Rhode Island Department of Education National Post-School Outcomes CenterNational Secondary Transition Technical Assistance CenterNational Dropout Prevention Center for Students with DisabilitiesFacilitator's Notes:These are three federally funded technical assistance and dissemination centers charged with helping States and LEAs address post-school outcomes (NPSO), effective transition planning (NSTTAC), and graduation and dropout prevention (NDPC-SD).Contact any of the centers for additional information and resources.