Slice and Dice Using existing data to answer novel questions about student outcomes February 7, 2012 Jill Kroll Office of Career and Technical Education Michigan Department Of Education Carol Clark CTE TRAC Director & Program Coordinator GASC Technology Center

Recognize challenges in using existing data to answer questions Gain knowledge of resources and tools to meet challenges of information needs 1 Learning Objectives

Two examples - Two solutions –Example 1: Using existing data –Example 2: Overcoming limitations of existing data Resources Discussion and Questions 2 Overview

Example 1: Using Existing Data Do secondary CTE students in articulated programs have higher placement rates? 3

Example 1: Using Existing Data What data are available to answer this question? In light of the available data, how might you refine the question? 4

Example 1: Using Existing Data Refine your question - “Placement” –Total placement - “In Employment or Continuing Education or Military” –Placement in Continuing Education –Placement in Community College –“Related” Placement Can you think of other available data? 5

Example 1: Using Existing Data Step 1: Download Follow Up Survey Data Log in to CTEIS.com Follow Up tab  Report Export Building Survey Data 6

7

Example 1: Using Existing Data Save or Open in MSExcel 8

Example 1: Using Existing Data Step 2: CTEIS Ad Hoc Query Tool Log In to CTEIS Choose Reports tab, Funding Reports, Ad hoc Query Choose agency, building 4301 (select year) Selection criteria 9

Example 1: Using Existing Data Student-Level Data From CTEIS

Example 1: Using Existing Data Export Student Data to MSExcel

Example 1: Using Existing Data Analyzing the Data Data analysis software –Relational database software –Spreadsheets –Statistical software –Others 12

Example 1: Using Existing Data Tech Prep Program? 2011 Total Placement (Employment or Continuing Education or Military) Not PlacedYes, PlacedTotal Not Tech Prep562 (5.9%)9,029 (94.1%)9,591 Yes, Tech Prep290 (5.3%)5,226 (94.7%)5,516 Total85214,25515,107 13

Example 1: Using Existing Data 14 Tech Prep Program? 2011 Attending School Not Attending School Attending SchoolTotal Not Tech Prep2,244 (23.4%)7,347 (76.6%)9,591 Yes, Tech Prep1,212 (22.0%)4,304 (78.0%)5,516 Total3,45611,65115,107

Example 1: Using Existing Data 15 Tech Prep Program? 2011 Attending A Community College Attending Another School Type* Attending a Community College Total Not Tech Prep3,898 (53.6%)3,381 (46.4%)7,279 Yes, Tech Prep2,274 (53.1%)2,010 (46.9%)4,284 Total6,1725,39111,563 *Attending business or trade school, college or university, military training or other training

Example 1: Using Existing Data 16 Tech Prep Program? 2011 Related Placement in Continuing Education Unrelated PlacementRelated PlacementTotal Not Tech Prep1,249 (24.4%)3,866 (75.6%)5,115 Yes, Tech Prep653 (21.7%)2,359 (78.3%)3,012 Total1,9026,2258,127

Example 2: Overcoming Limitations of Existing Data What is the impact of integrated instruction on academic achievement? 17

Example 2: Overcoming Limitations of Existing Data Local data: –Number of students signed up for each course –Number of students who took test for academic credit –-Number of students who passed test –Number of students who signed up for RAC 18

REQUIRED ACADEMIC CREDIT (RAC) ELA-1214 MATH220 SCIENCE76 Visual/Performing Arts (VPAA) 65 TOTAL GASC Signed up for RAC ELA-1212 MATH213 SCIENCE65 Visual/Performing Arts (VPAA) 62 TOTAL352 GASC Total Students Achieving RAC

Example 2: Overcoming Limitations of Existing Data Count % Increase in (compared to ) ELA-12157% MATH41287% SCIENCE9424% VPAA11475% TOTAL GASC Signed up for RAC

Example 2: Overcoming Limitations of Existing Data CTE SubjectAcademic Subject Type of Academic Integration Method of Qualifying for Academic Credit Law & Public SafetyEnglish Language Arts Instruction by Academic Teacher Papers graded by English Teacher CTE-specific content, ELA tasks Multiple Programs (2 nd Year Students) MathCollaborative Teaching Model 1 Pre-Test/Post-Test (Standard Math Test Questions) HealthScienceCollaborative Teaching Model (different for each Health program) Integrated post-test: Contextual academic test questions specific to the program Multiple ProgramsVisual/ Performing Arts Collaborative Teaching Model 2 Teachers choose a minimum of one project per semester and grade with a rubric 21

Example 2: Overcoming Limitations of Existing Data 22

Example 2: Overcoming Limitations of Existing Data Solutions to limitations in existing data: –Modify/Simplify question 23

Example 2: Overcoming Limitations of Existing Data Collect supplemental data 24 CTE SubjectAcademic Subject Type of Academic Integration Method of Qualifying for Academic Credit Law & Public SafetyEnglish Language Arts Instruction by Academic Teacher Papers graded by English Teacher CTE-specific content, ELA tasks (blind) Law & Public SafetyNone Papers graded by English Teacher CTE-specific content, ELA tasks (blind)

Example 2: Overcoming Limitations of Existing Data Solutions to limitations in existing data –Control extraneous factors Choose one subject area Use three methods of integrating instruction Use one method of qualifying for academic credit (assessing achievement) 25

Example 2: Overcoming Limitations of Existing Data 26 CTE Subject Academic Subject Type of Academic Integration Method of Qualifying for Academic Credit Validation Therapeutic Services MathAcademic Instructor Provides Instruction Pre-Test/Post-Test using Standard Math Test Questions Integrated post-test: Contextual Math test questions specific to the program College Course Grade from STARR Data Accuplacer test results Same Collaborative Teaching Model (CTE instructor Provides Academic Instruction) Same Pull-Out Academic Instruction (Tutoring) Same

Resources Look for more existing data –National National Center for Education Statistics (NCES) website Midwest Regional Education Laboratory (REL) –Existing local and state data –Peers 27

28 NCES: Data Collection Tools

NCES: Example Content Areas Early Childhood Parent health, child growth and development High school student attitudes and beliefs Postsecondary students, institutions Student math, science knowledge Teacher salaries, preparation, assignments, employment status Parenting practices Public and Private School characteristics Financial data Libraries Crime and Safety

Regional Education Laboratories Funded by: Institute of Education Sciences (ies) Purpose: Provide access to high-quality, scientifically valid education research Publications: Ask A REL: A collaborative reference desk service Midwest REL: American Institutes for Research 30

Midwest REL: Resources/Services 31

Contact: Jill Kroll Supervisor Office of Career and Technical Education Michigan Department Of Education (517) Carol Clark CTE TRAC Director & Health and Human Services Platform Facilitator Genesee Area Skill Center Technology Center (810) ext