1. General Supervision: Highlights on Monitoring and the 09-02 Memo Western Regional Resource Center APR Clinic 2010 November 1-3, 2010 San Francisco, California

2. What is General Supervision? A requirement: IDEA 34 CFR §300.600 The State must monitor the implementation of this part, enforce this part… and annually report on performance under this part. A way to organize what you do An important component of OSEP verification visits and determinations A method of tracking and encouraging continuous improvement

3. Components of General Supervision Helpful Guidance From TA Providers (NCSEAM) State Performance Plan Policies, Procedures & Effective Implementation Data on Processes and Results Targeted Technical Assistance & Professional Development Effective Dispute Resolution Integrated Monitoring Activities Improvement, Correction, Incentives & Sanctions Fiscal Management

4. According to OSEP Five Critical Elements of General Supervision: 1. A system to identify noncompliance in a timely manner using its different components 2. A system to ensure correction of identified noncompliance in a timely manner 3. Procedures and practices to implement the dispute resolution requirements of IDEA Western Regional Resource Center APR Clinic 2010

5. According to OSEP Five Critical Elements of General Supervision: 4.Procedures and practices to improve educational results and functional outcomes for all children with disabilities 5.Procedures and practices to implement other requirements, i.e., fiscal requirements, private schools, NIMAS, assessments, etc.

6. Building Effective General Supervision General Supervision Foundation

7. Step 1 – Identifying an Issue What components are used to identify noncompliance? Which issues to look for: How few are too few? How many is too many? Determining and prioritizing what to look for.

8. Components (examples): On-site Activities IEP/Record Reviews Interviews (Families/Providers/Teachers) Others ? Off-site Activities Database (includes SPP/APR data collections and analysis) Self Assessment Desk Audit Surveys (Families/Providers/Teachers) Contracts Dispute Resolution (formal and informal) Local reporting Others?

9. 9 9 Which Issues to Look For SPP/APR Indicators Related Requirements Your Indicators

10. 10 Compliance and performance Self-identified issues (discussion of progress and slippage, improvement activities) OSEP-identified issues (response table)

11. 11 Coordination of APR data collection and monitoring data Database data vs. census data vs. monitoring data Monitoring Data and the SPP/APR

12. 12 If the SEA/LA receives data through its database that show noncompliance, the SEA/LA must: Make a finding, AND Require correction as soon as possible, and in no case later than one year after the SEA/LAs notification

13. 13 SEA/LA may identify one or more points in time during SPP/APR reporting period when it will review compliance data from database and identify noncompliance In making compliance decisions, SEA/LA should then review all data received since the last time SEA/LA examined data from database and made compliance decisions. SEA/LA may either examine: All data in database, OR Statewide representative sample From OSEP on Databases

14. 14 Findings must be made based on data collected through any method that demonstrates noncompliance (e.g., when the level of compliance is less than 100%) Substantial compliance (e.g., 95% compliance) or other thresholds (e.g., 3 of 4 children had a timely transition conference) do not apply to identification or correction of noncompliance

15. 15 Option 1 Make a finding of noncompliance. Option 2 Verify whether data demonstrate noncompliance, and then issue finding if data do demonstrate noncompliance. Option 3 Verify LEA has corrected noncompliance before State issues written findings of noncompliance, in which case State not required to issue written finding of noncompliance. Slide prepared by OSEP

16. Scenario - Identification

17. 17 On October 9, the Lead Agency monitored Oleander Infant/Toddler Daycare Program and found that 2 of 18 files did not have justification for not being in natural environments. On October 23rd the program faxed the IFSP pages to the LA, with justification statements based on the Medical Team at UNC pediatric services, that two children with active foot/mouth disease were in pediatric medical rehab class for 3 months, after which they would be moved to the regular toddler class. The Lead Agency issued the monitoring report on November 1st.

18. 18 Should the Lead agency issue a finding of noncompliance for the two files that did not have adequate natural environment justifications? What options does the Lead Agency have prior to issuing the monitoring report?

19. 19 Prioritizing Issues – Monitoring with a Focus How can you make this process manageable and still get the data you need?

20. Putting it Together

21. 21 Jan Aug Dec Nov Mar Feb Oct Apr July Sept June May Monthly Activities Resource Specialist Reports/Meetings IEP Count Data Report Evaluations Data Report Exit Data Report Personnel Data Report On-site school visits (Sept - March School Visits (5 in February/March) Examine B13, B11, B12 data to determine noncompliance Issue findings Enter CAPs in tracking log, review CAP progress, verify correction, and determine TA needs Select schools for onsite visits in next school year Report APR performance to the public Set agenda for APR indicator drafts meeting in Fall Provide APR data to indicator teams Schedule training and share resources/tools Hold initial meeting of APR indicator teams Review SPP/APR progress on improvement activities School Visits (5 in October/November) Examine B13, B11, B12 data to determine noncompliance Issue findings Submit 618 Child Count, LRE, and Assessment Data Submit 618 Personnel, Exiting, Discipline, Disp.Res. Submit SPP/APR to OSEP Enter CAPs in tracking log, review CAP progress, verify correction, and determine TA needs Organizing the Identification Process - The Wheel

22. Steps 2 & 3 – Investigating Issues Step 2 – Determine the Extent/Level of the Issue Step 3 – Determine the Cause of the Issue 22

23. 23 Factors to Consider What is the level of compliance/performance? Look at percentages: All (100%), mostly compliant (95%+), moderately (85-94%), somewhat (76-84%), limited compliance (75% or less ) Look at number of instances of compliance in proportion to the N (total): 4 out of 5 -vs.- 40 out of 50

24. 24 Where/with whom is the problem happening? One or more providers/teachers One or more local programs/schools Statewide Factors to Consider

25. 25 Step 2 – Determine the Level/Extent of the Issue

26. 26 Factors to Consider Historical/Trend Data Open CAP on the same issue Previous completed CAPS on the same issue Repeat offense – not really fixing the problem although findings are corrected or not issued Trend data – do data show clear progress or slippage in this area? More applicable when looking at smaller time periods (quarters)

27. 27 Factors to Consider Other Considerations Demonstrated Ability to Correct (previously identified noncompliance corrected within timelines) Exceptional Circumstances Number of findings of noncompliance (multiple noncompliance issues vs. one)

28. 28 Root Cause Analysis Need for Improvement Need for Correction Policies Procedures Practices Keep in mind what the resolution might be - based on the cause

29. 29 Root Cause Analysis

30. 30 Use decisions on the level/extent and the root cause of issues, including whether there is noncompliance, in order to determine: At what level resolution needs to happen Who needs to be responsible What actions should be required What data will be used to verify correction

31. Example

32. 32 The Lead Agency monitored the Mothers & Babies Program during an onsite visit. The findings report identified noncompliance based on the following: 20 out of 50 children did not receive timely services 2 out of 40 children did not have an IFSP meeting in 45 days 5 out of 45 children did not receive written prior notice at the appropriate junctures

33. 33 What actions might the Lead Agency require of 2 the Mothers & Babies Program to ensure correction the noncompliance related to each of the following: timely services (20 of 50)? IFSP meeting within 45 days (2 of 40)? written prior notice (5 of 45)?

34. 34 Step 4 - Assign Accountability for the Issue and its Resolution The resolution should be based on the issue and the analyses conducted (extent/level and root cause) For improvement issues (not a compliance issue) Improvement plan For noncompliance Finding must be issued Corrective action

35. 35 For Improvement Issues Use decisions made about the level/extent and the root cause of the issue to: Determine areas in need of improvement Explore relationship to SPP indicators Determine resources needed (staff, TA providers, best practice) Develop improvement strategies

36. 36 Generally, OSEP expects written notification to be issued less than three months from discovery and should include: SEAs/LAs conclusion that LEA/program is not in compliance Citation of relevant regulatory or statutory requirement Description of quantitative and/or qualitative data supporting SEA/LAs conclusion, AND Statement requiring correction as soon as possible, but in no case later than one year from notification

37. 37 SEA/LAs may choose how they will count and report their monitoring findings: Group individual instances in a program involving the same legal requirement together as one finding, AND/OR Report each of the individual instances of noncompliance as a separate finding Exception: each finding identified through a State complaint or a due process hearing must be counted as a separate finding

38. 38 Corrective action What actions need to be taken to correct the noncompliance (based on analyses)? Submit data to demonstrate correction Corrective Action Plan How will we know they worked? Identify which data will be used to verify correction of the noncompliance

39. 39 Steps 5 & 6 - Verify Correction and Follow-up Step 5 – Verify Resolution of the Issue OSEP Definition of Verifying Correction Step 6 – Follow up on Resolution Continuous Improvement

40. 40 Improvement Issues (not compliance issues) Several tools are available to assist states in: Analyzing indicator data and other performance variables at SEA/LA and school levels Designing effective improvement strategies Evaluating improvement activities www.rrfcnetwork.org

41. 41 Step 5 – Ensure and Verify Resolution of the Issue For improvement issues (not compliance issues), this may be over a period of several years For compliance issues, this is clearly defined by OSEP

42. 42 Compliance Issues (defined by OSEP) Verifying resolution of compliance issues is clearly defined by OSEP. Two main documents explain and clarify the process states are to use to correct and verify correction of noncompliance: OSEP Memo 09-02, October 17, 2008 F.A.Q. on Identification and Correction of Noncompliance, September 3, 2008

43. 43 From OSEP on Timeline for Verifying Correction The timeline for when correction must be verified (as soon as possible but in no case later than one year) begins on the date on which the SEA/LA notifies the LEA/program, in writing, of its finding of noncompliance

44. 44 From OSEP on Timely Correction For an SEA/LA to report that noncompliance has been corrected it must first: Account for ALL noncompliance identified by SEA/LA Determine: in which LEAs/programs noncompliance occurred the level of noncompliance in each, AND the root cause(s) of noncompliance If needed, require change in the LEAs/programs Policies Procedures, AND/OR Practices

45. 45 In order to demonstrate that previously identified noncompliance has been corrected, an SEA/LA must: Prong 1 - Account for the correction of all child-specific instances of noncompliance AND Prong 2 - Determine whether each LEA or Program with identified noncompliance is correctly implementing the specific regulatory requirements (achieved 100% compliance) From OSEP on Timely Correction

46. 46 Both prongs apply to correction of all findings of noncompliance, and noncompliance reported in APRs, whether there is a high level of compliance (but below 100%) or a low level of compliance States cannot use a threshold of less than 100% to conclude that the LEA/program has corrected noncompliance

47. For child-specific noncompliance that is not a timeline requirement, SEA/LA must ensure that LEA/program corrected noncompliance in each individual case, unless: The requirement no longer applies OR The child is no longer within the jurisdiction of LEA/program 47 From OSEP on Correction of Child-Specific Noncompliance

48. 48 For child-specific noncompliance with a timeline requirement, the SEA/LA must ensure that the service/evaluation/etc. was provided, although late, unless: The requirement no longer applies OR The child is no longer within the jurisdiction of LEA/program From OSEP on Correction of Child-Specific Noncompliance

49. 49 To ensure correction of child-specific noncompliance regardless of whether or not it is a timeline requirement… The SEA/LA could review or require a local agency to review all or a sample of the records of affected children to verify correction

50. 50 For an SEA/LA to report LEAs/programs are implementing the specific regulatory requirements, in addition to the correction explained for child specific noncompliance, the SEA/LA must: Based on its review of updated data, and within one year of notifying the program of noncompliance, determine if the LEA/program is in compliance

51. 51 For an SEA/LA to report LEAs/programs are implementing the specific regulatory requirements: Must be based on NEW (updated) data Correction = 100% Hints: May happen very quickly Period of time (at 100%) for verification should depend on the level of noncompliance and the cause of the noncompliance How correction will be verified should be determined before finding is made Verification of Correction of Noncompliance

52. 52 FFY 2009 Monitoring Indicator C8/B12 August 15 th 2009: 1000 children: 50 100 750 80 20 12 children (94%) 2 children (71%) Monitoring Results FFY 2009: By November 12 - C8/B12 - 93%, Three Findings 7 9 200 7 3 Noncompliance Corrected & Verified By November 11, 2010 1. Child Specific 5 210 0 New Children Transitioning from C to B By February 15th, 2010 2 children (33%) 2. Program 1. Child Specific 2. Program 1. Child Specific 2. Program 22616 Quiz A B C D E 0 children (100%) 0 children (100%)

53. 53 From OSEP on Verifying Correction If 100% is not obtained when reviewing updated data to verify correction: a new finding is NOT issued the original finding remains open The child-specific noncompliance identified from this review of updated data must be corrected AND the state must review further updated data until the LEA/program achieves 100%

54. 54 Notification of Verifying Correction After correction has been verified: Notify (in writing) the accountable party (LEA/program/etc.) that correction has been verified and the finding of noncompliance is closed out. Notification may include: Corrective actions taken to correct noncompliance Data used to verify correction Correction of each instance Updated data demonstrating 100% compliance Whether the noncompliance was corrected within 12 months of issuing the finding

55. 55 Documentation of Correction and Verification of Correction What documentation could you use as evidence of how you verify correction? Notification of findings Corrective action plans Notification of verification of correction Procedures for verification of correction How data for verification is collected and verified If samples are used, how they are representative

56. 56 Step 6 - Follow Up on Resolution of the Issue Incentives for correction/improvement Sanctions/enforcement actions for uncorrected noncompliance Easier to use if set up in advance

58. Western Regional Resource Center APR Clinic 2010 November 1-3, 2010 San Francisco, California