2. Using Data to Improve Adult Ed Programs Administrators’ Workshop

3. Workshop Objectives Participants will be able to understand the data- driven decision making process. Participants will be able to identify what types of data can be collected. Participants will be able to identify what data reports are most useful to teachers and administrators. Participants will be able to identify what major barriers hinder the data collection process. Participants will be able to analyze data report examples.

4. What is Data-Driven Decision Making? Data-driven decision making is the process of making choices based on appropriate analysis of relevant information.

5. Why use data to make decisions? More access to better information enables educational professionals to test their assumptions, identify needs, and measure outcomes. Administrators and teachers are using data-driven decision making to: provide more individualized instruction to students track professional development resources identify successful instructional strategies better allocate scarce resources communicate better with parents and the community.

6. What data should we collect and use to make decisions? There is an abundance of information stored. Examples are: student records student assessment human resources student progress special education curriculum management.

7. What data reports are most useful to instructors?  TABE or CASAS test results for entire class  Student attendance information  Teacher developed test results for entire class  Progress reporting information for class  Student retention information for class  Student performance gains (LCP/OCP) for class

8. What data reports are most useful to administrators?  TABE or CASAS test results for program area  Student attendance information for all programs  Progress reporting information for all programs  Student retention information for programs/school site  Student performance gains (LCP/OCP) for programs and for school  Cost effectiveness of program (Income versus cost of salaries, fringe, supplies, equipment)

9. Administrators use all types of data: attendance enrollment student performance (LCP’s, OCP’s) student/teacher/parent satisfaction surveys. Test results are used to assess progress, allocate resources, and create school improvement plans. Information is organized numerically rather than alphabetically. The information includes objective descriptions of data, visual displays of information, and query tools. What common data report formats are most useful to administrators?

10. What are the major barriers to using data based decision-making? Lack of training Interoperability—systems that are unable to share or exchange data Lack of understanding of what to do with the data Absence of clear priorities on what data should be collected Failure to collect data in a uniform manner Outdated technology systems Low quality data – inaccurate or incomplete Timing of data collection

11. What are the major misconceptions about effective use of data in decision making? Build it and they will use it. Teachers need to know how to analyze data and use query systems. Test scores determine the quality of a school and the student’s education.

12. What is necessary for the systematic use of data for decision making? Collection, integration and dissemination of data Analysis and reporting of data Process and procedures for acting on the data Review Analysis Planning

13. Schools need both organizational and individual capacity for improvement: Leadership Professional development. Administrators need training with the opportunity to apply skills learned using their own institutional data. Dialogue with peers keeps the process going. School-based training for faculty and staff is necessary. Instructors need training in different instructional strategies to apply when the data shows that traditional methods are not working. What types of skills are needed to implement systemic data processes?

14. Who are the key decision makers at the school site who should be involved in the data-driven decision making process?  Administrators are the change agents at the school site.  Administrators model data use and encourage it by sharing the benefits and successes.  Site-based specialists or support teams assist administrators and teachers with data mining and analysis.

15. Who are the key decision-makers at the classroom level who should be involved? In addition to using data for determining instruction, teachers can engage students in the decision making process by helping them: view appropriate reports; set learning goals; make decisions about how to meet their goals.

16. Where do we begin? The process: Develop a leadership team Collect various types of data Analyze data patterns Generate hypotheses Develop goal-setting guidelines Design specific strategies Define evaluation criteria Make the commitment

17. What information does our institution need to make decisions that will improve student achievement? What learning strengths and weaknesses are evident in the data? Which groups or subgroups of students are having difficulty learning? What instructional changes might improve student learning? What professional development is need to improve student learning? What materials and equipment are needed to support changes in instruction?

18. Example 1 – Making decisions based on data A school examines its student performance results and finds: As a whole, the school is doing better in reading than in math Students are doing better in basic computation than in problem solving As a subgroup, Hispanic male students perform the lowest on grades and tests at most grade levels

19. Example 1- continued Using this information, the school improvement team decides to find the following: An instructional intervention that specifically addresses basic math computation Interventions that have been especially effective in improving the performance of Hispanic male students

20. Example 1 - continued Professional development is planned to provide instruction on the new intervention: Staff development days are planned for teachers to learn the new intervention before it is implemented. Regular short meetings are planned to give teachers time to discuss their efforts and troubleshoot problems while implementation occurs in the classroom.

21. Example 2: FCAT Classes NameCourseDaysLCP Earned Credit Earned Pre- Class Score Post- Class Score GradeESELEP MariaIntensive Reading T/R1.5248300BNo MaribelIntensive Reading T/R1.5223296BNoYes KasieIntensive Reading T/R1.5245286AYesNo ElysiaIntensive Reading T/R1.5260301BNo EricIntensive Reading T/R1.5246185BNoYes ThordisIntensive Reading T/R1.5297306ANoYes Totals/ Average 632532793.33

22. Example 3 – Budget Data Across Sites SiteFee Support Enrolled FS Percent Enrolled 9-12 Enrolled 9-12 Percent Enrolled Grades 30/31 Enrolled Grades 30/31 Percent Enrolled Total Enrollment BudgetCost per Student ACE 1 13912%29125%71562%1,145 $619,537 $541 ACE 2 889%32233%56758% 977 $682,826 $698 ACE 3 13416%36044$32740% 821 $481,238 $586 ACE 4 122%16627%43871% 616 $460,099 $746 ACE 5 19225%18124%38751% 760 $619,698 $815 Totals/ AVG 56513%1,32031%2,43456%4,319 $2,863,398 $677

23. Example 4 – Co-Enrolled Completion/Retention Semester II Course TitleEnrollmentLCP’s EarnedStudents with no LCP’s Percent Completing Percent Not Retained to Completion English I 128467%33% English II 85363%37% English III 92722%78% English IV 1721512%88% Totals/ Averages 46172933%67%

24. Data Review Other types of information teachers may want to review: Student records with demographic data (LEP, ESE, grade level completed, courses taken, GPA) FCAT scores TABE scores Other test records

25. More Data Review Administrators may collect and analyze other reports: Class attendance School surveys – satisfaction or quality surveys Individual program/class surveys of satisfaction or delivery of services Student exit surveys W-26 reports (high school students withdrawing to attend postsecondary programs)

26. Let’s Review the DATA! Using the handouts provided, break into small groups and discuss the data presented on each handout following directions provided. What assumptions can be made about the data presented? Did the data serve to confirm ideas you had already pondered? What other types of data could you collect about your program(s)?

27. References/Links Data-Based Decision Making – Resources for Educators. http://www.ael.org/dbdm/Tutorial.cfm?&ider= Deve4060 D3M: Helping schools distill data. http://eric.uoregon.edu/search_find/data_anal ysis.html The Toolbelt: A collection of data-driven decision-making tools for educators. http://www.ncrel.org/toolbelt/