1. Slice and Dice Using existing data to answer novel questions about student outcomes February 7, 2012 Jill Kroll Office of Career and Technical Education Michigan Department Of Education Carol Clark CTE TRAC Director & Program Coordinator GASC Technology Center

2. Recognize challenges in using existing data to answer questions Gain knowledge of resources and tools to meet challenges of information needs 1 Learning Objectives

3. Two examples - Two solutions –Example 1: Using existing data –Example 2: Overcoming limitations of existing data Resources Discussion and Questions 2 Overview

4. Example 1: Using Existing Data Do secondary CTE students in articulated programs have higher placement rates? 3

5. Example 1: Using Existing Data What data are available to answer this question? In light of the available data, how might you refine the question? 4

6. Example 1: Using Existing Data Refine your question - “Placement” –Total placement - “In Employment or Continuing Education or Military” –Placement in Continuing Education –Placement in Community College –“Related” Placement Can you think of other available data? 5

7. Example 1: Using Existing Data Step 1: Download Follow Up Survey Data Log in to CTEIS.com Follow Up tab  Report Export Building Survey Data 6

8. 7

9. Example 1: Using Existing Data Save or Open in MSExcel 8

10. Example 1: Using Existing Data Step 2: CTEIS Ad Hoc Query Tool Log In to CTEIS Choose Reports tab, Funding Reports, Ad hoc Query Choose agency, building 4301 (select year) Selection criteria 9

11. Example 1: Using Existing Data Student-Level Data From CTEIS 10 4301-2009-2010

12. Example 1: Using Existing Data Export Student Data to MSExcel 11 4301-2009-2010

13. Example 1: Using Existing Data Analyzing the Data Data analysis software –Relational database software –Spreadsheets –Statistical software –Others 12

14. Example 1: Using Existing Data Tech Prep Program? 2011 Total Placement (Employment or Continuing Education or Military) Not PlacedYes, PlacedTotal Not Tech Prep562 (5.9%)9,029 (94.1%)9,591 Yes, Tech Prep290 (5.3%)5,226 (94.7%)5,516 Total85214,25515,107 13

15. Example 1: Using Existing Data 14 Tech Prep Program? 2011 Attending School Not Attending School Attending SchoolTotal Not Tech Prep2,244 (23.4%)7,347 (76.6%)9,591 Yes, Tech Prep1,212 (22.0%)4,304 (78.0%)5,516 Total3,45611,65115,107

16. Example 1: Using Existing Data 15 Tech Prep Program? 2011 Attending A Community College Attending Another School Type* Attending a Community College Total Not Tech Prep3,898 (53.6%)3,381 (46.4%)7,279 Yes, Tech Prep2,274 (53.1%)2,010 (46.9%)4,284 Total6,1725,39111,563 *Attending business or trade school, college or university, military training or other training

17. Example 1: Using Existing Data 16 Tech Prep Program? 2011 Related Placement in Continuing Education Unrelated PlacementRelated PlacementTotal Not Tech Prep1,249 (24.4%)3,866 (75.6%)5,115 Yes, Tech Prep653 (21.7%)2,359 (78.3%)3,012 Total1,9026,2258,127

18. Example 2: Overcoming Limitations of Existing Data What is the impact of integrated instruction on academic achievement? 17

19. Example 2: Overcoming Limitations of Existing Data Local data: –Number of students signed up for each course –Number of students who took test for academic credit –-Number of students who passed test –Number of students who signed up for RAC 18

20. REQUIRED ACADEMIC CREDIT (RAC) ELA-1214 MATH220 SCIENCE76 Visual/Performing Arts (VPAA) 65 TOTAL375 19 GASC 2010-11 Signed up for RAC ELA-1212 MATH213 SCIENCE65 Visual/Performing Arts (VPAA) 62 TOTAL352 GASC 2010-11 Total Students Achieving RAC

21. Example 2: Overcoming Limitations of Existing Data Count % Increase in 2011-12 (compared to 2010-11) ELA-12157% MATH41287% SCIENCE9424% VPAA11475% TOTAL635 20 GASC 2011-12 Signed up for RAC

22. Example 2: Overcoming Limitations of Existing Data CTE SubjectAcademic Subject Type of Academic Integration Method of Qualifying for Academic Credit Law & Public SafetyEnglish Language Arts Instruction by Academic Teacher Papers graded by English Teacher CTE-specific content, ELA tasks Multiple Programs (2 nd Year Students) MathCollaborative Teaching Model 1 Pre-Test/Post-Test (Standard Math Test Questions) HealthScienceCollaborative Teaching Model (different for each Health program) Integrated post-test: Contextual academic test questions specific to the program Multiple ProgramsVisual/ Performing Arts Collaborative Teaching Model 2 Teachers choose a minimum of one project per semester and grade with a rubric 21

23. Example 2: Overcoming Limitations of Existing Data 22

24. Example 2: Overcoming Limitations of Existing Data Solutions to limitations in existing data: –Modify/Simplify question 23

25. Example 2: Overcoming Limitations of Existing Data Collect supplemental data 24 CTE SubjectAcademic Subject Type of Academic Integration Method of Qualifying for Academic Credit Law & Public SafetyEnglish Language Arts Instruction by Academic Teacher Papers graded by English Teacher CTE-specific content, ELA tasks (blind) Law & Public SafetyNone Papers graded by English Teacher CTE-specific content, ELA tasks (blind)

26. Example 2: Overcoming Limitations of Existing Data Solutions to limitations in existing data –Control extraneous factors Choose one subject area Use three methods of integrating instruction Use one method of qualifying for academic credit (assessing achievement) 25

27. Example 2: Overcoming Limitations of Existing Data 26 CTE Subject Academic Subject Type of Academic Integration Method of Qualifying for Academic Credit Validation Therapeutic Services MathAcademic Instructor Provides Instruction Pre-Test/Post-Test using Standard Math Test Questions Integrated post-test: Contextual Math test questions specific to the program College Course Grade from STARR Data Accuplacer test results Same Collaborative Teaching Model (CTE instructor Provides Academic Instruction) Same Pull-Out Academic Instruction (Tutoring) Same

28. Resources Look for more existing data –National National Center for Education Statistics (NCES) website Midwest Regional Education Laboratory (REL) –Existing local and state data –Peers 27

29. 28 NCES: Data Collection Tools http://nces.ed.gov/

30. NCES: Example Content Areas Early Childhood Parent health, child growth and development High school student attitudes and beliefs Postsecondary students, institutions Student math, science knowledge Teacher salaries, preparation, assignments, employment status Parenting practices Public and Private School characteristics Financial data Libraries Crime and Safety

31. Regional Education Laboratories Funded by: Institute of Education Sciences (ies) Purpose: Provide access to high-quality, scientifically valid education research Publications: http://ies.ed.gov/ncee/edlabs/projects/ http://ies.ed.gov/ncee/edlabs/projects/ Ask A REL: A collaborative reference desk service Midwest REL: American Institutes for Research http://www.learningpt.org/ 30

32. Midwest REL: Resources/Services 31

33. Contact: Jill Kroll Supervisor Office of Career and Technical Education Michigan Department Of Education (517) 241-4354 KrollJ1@Michigan.gov Carol Clark CTE TRAC Director & Health and Human Services Platform Facilitator Genesee Area Skill Center Technology Center (810) 760-1444 ext. 176 clarkcar@gasc.flint.K12.mi.us 32