Michael Flory CNA Education Roland O’Daniel & Jo Ann Mosier Collaborative for Teaching & Learning April 2011 The Effects of a Hybrid Program for Algebra I on Grade 9 Students’ Mathematics Achievement Innovation Conference December 2011

2 Agenda Introductions, REL Appalachia at CNA Hybrid Instruction Research Questions The KYVS hybrid program for Algebra I –Materials –Processes –Video Research Design Perceptions from CTL’s Instructional Specialists

3 hybridalgebra.wikispaces.com

4 REL Appalachia at CNA One of 10 regional labs, funded by the Institute for Education Sciences, USDE. Major activities –Long-term evaluations –Shorter-term analyses –Field scientist program –Quick turn-around--  Ask-a-REL, Bridging Research to Practice

Research Team NameAffiliationRole Dr. Linda CavalluzzoCNA EducationPI Dr. Deborah Lowther Dr. Christine Mokher Education Innovations & University of Memphis CNA Education Co-PI Researcher Dr. Xitao FanUVAMethodologist

Implementation Team NameAffiliationRole Dr. Linda Hargan/ Jo Ann Mosier Roland O’Daniel Collaborative for Teaching and Learning (CTL) Site recruiter/ Instructional Specialists, PD Algebra 1 instruction Kari WelchKentucky Virtual High School (KVHS)/KDE Courseware selection and access; Training for Master Teachers; Access to student data

7 Online vs. hybrid instruction Online learning –Teacher of record is at a distance –Student can be anywhere that has internet access –Can bring a course to,  A school that otherwise could not offer it  A student needing to repeat it  A student that wants to take an advanced course Hybrid learning –Online resources complement instructional experience  Courseware or supplemental materials in the classroom  Before or after school access to supplemental materials  Use of online communications to address student questions

8 The KYVS Hybrid Program for Algebra I Algebra I teachers instruct students using online resources in a face- to-face classroom setting –Teachers have a computer with internet and screen for instruction in their classroom –Students have individual access 2 days/week Teachers receive sustained Professional Development (PD) –Learn to use technology to facilitate effective instructional practices

9 Research questions What is the impact of the KYVS Hybrid Program for Algebra I on grade 9 students’: –Math achievement in pre-algebra/algebra in the fall of grade 10 on the PLAN assessment? –Subsequent math course enrollment in the post-intervention year? Does the impact of the intervention vary between: –Male and female students? –Rural and non-rural schools?

10 Study Design and Sample Randomized control trial 41 schools across KY offering Algebra I to 9th graders (6,000 students) 2 Cohorts –25 schools in Cohort I, SY 07/08 –16 new schools in Cohort II, SY 08/09 –Each school remains in their group for 2 years

11 ALGEBRA I HYBRID RESEARCH SCHOOLS

12 Key Internet-based resources For teachers: Spotlight on Algebra I For students: NROC Algebra I LMS: Blackboard

13 The KYVS Hybrid Algebra I Program 3-day F2F orientation and start up 5 Synchronous online summer sessions 1-day end-of-summer F2F capstone During school year –Monthly online sessions –2 site visits from instructional specialists

14 Notable Features of the Professional Development Sustained Work with a learning community Active learning Models the practices teachers will use in their classrooms –Effective practices in a hybrid learning environment

15 kvhs.blackboard.com Username: roland.odaniel PW: password “My Courses” tab

16 kvhs.blackboard.com “My Courses” tab Username: roland.odaniel PW: password

17

18

19

20

21

22

23

24

25 In the classroom The hybrid program facilitates use of effective instructional practices –Teachers set learning goals for each class –“Bell-ringer” a warm-up activity to access prior knowledge –Graphic organizers help students monitor progress through lessons –“Exit slip” a closing activity to assess and respond to student learning –Mathematical Literacy Strategies

26 In the lab Students work alone or in pairs Use a note-taking device to write down what they understand, and what they don’t Teachers become facilitators of learning –Providing individual assistance as needed, or teach a mini-lesson

27

28 Hybrid Algebra 1 Classroom Model Structures and processes –Transparency of learning goals –Transitioning from classroom to virtual classroom –Tracking tool or note taking device –Learning cycle components –Intentional whole group reflection (guided discourse)

29 Hybrid Algebra 1 Classroom Model Typical lessons –Starter activity –Learning expectations (learning product, transitioning from classroom to the laboratory) –KYVS lesson (warm-up, new learning with explorations & use of applets) –Whole group reflections with guided discourse –Assignment –Next steps

30

31

32

33

34

35 Program evaluation Randomized Controlled Trial (RCT) Gold Standard for measurement of impact –High internal validity –Causal by design –If properly randomized, differences between T and C groups are random, except for exposure to treatment Measures the Impact of the Intent to Treat, and not the impact of the treatment itself

36 Theory of Action

37 Study Design: Instruments and Data Collection PRE-INTERVENTION: –2 pretests in grade 8 –Student Information  Demographics  Enrollment records DURING INTERVENTION –Classroom observations –Teacher survey POST-INTERVENTION: –10th grade statewide test –10th grade course –Algebra I end-of-course test  Administered by researchers  Could not use in final report due to low response rates and differential attrition by treatment status

38 Power Analysis—before and after recruiting Underlying assumptions to determine MDE –Number of observations at each level, intraclass correlations, cluster- level covariates, level of significance, potential attrition

39 Attrition – PLAN Outcome Consolidated Standards on Reporting Trials (CONSORT) PLAN Assessment of Pre-Algebra/ Algebra Knowledge and Skills Response rate: 84% Treatment 86% Control

40

41 Perceptions of Instructional Specialists--Strengths Establishment of a Community of Learners Engagement of teachers as learners while in the distance environment:  small group interaction with whole group report out.  reading articles prior to the session – sharing ideas and intentional connections to the hybrid environment.  designing session topics that deepen understanding of the hybrid environment and are responsive to learning needs.

42 Perceptions of Instructional Specialists--Strengths Creation of classroom structures and processes supported through modeling and materials. Extension of content knowledge and best practices in teaching Algebra I within the hybrid model. Formation of new teaching habits needed within the hybrid approach.

43 Perceptions of Instructional Specialists--Challenges Ensuring fidelity of implementation. Maintaining a sense of urgency and need for another approach to learning. – Transitioning the learner from face-to-face instruction to the virtual independent learning environment. – Planning instruction that bridges learning from classroom to the virtual environment.

44 Perceptions of Instructional Specialists--Challenges Accessing the technology that does not always cooperate (e.g., proxy issues). – Affects classroom and PD sessions Logging in and engaging in the distance learning session.

45 Status Report Final Report –Draft report submitted for review, June 2010 –In final phase of review –Release of final report expected, Spring 2012 –RELappalachia.orgRELappalachia.org

46 For Additional Information… Linda Cavalluzzo,CNA Jo Ann Mosier or Roland O’Daniel, CTL or Grace Yeh Kentucky Virtual School, KDE