Effects of Curricular Change in a Freshmen College Applied Algebra Course Dr. Robert Mayes Director of the Institute for Mathematics Learning

Crisis in Mathematics Education Business Higher Education Forum (2005) Business Higher Education Forum (2005) American Diploma Project (2005) American Diploma Project (2005) Percentage of ninth grade students in the U.S. who graduate from high school is 68% Percentage of ninth grade students in the U.S. who graduate from high school is 68% 40% of these same ninth graders start college 40% of these same ninth graders start college 27% of them persist through the second year 27% of them persist through the second year 18% earn a degree 18% earn a degree 22% of college freshmen are not ready for the entry level mathematics course and require remediation 22% of college freshmen are not ready for the entry level mathematics course and require remediation

Crisis in Mathematics Education MAA Task Force on the First College Level Mathematics Course (Kime et al, 2000) MAA Task Force on the First College Level Mathematics Course (Kime et al, 2000) College enrollments are increasing, calculus enrollments are stagnant College enrollments are increasing, calculus enrollments are stagnant 9% of students matriculate into calculus 9% of students matriculate into calculus Majority of first year students’ first college mathematics course is either remedial, liberal arts, or college algebra Majority of first year students’ first college mathematics course is either remedial, liberal arts, or college algebra Failure and withdrawal rates in before calculus courses are often dismal, with numbers between 40% and 60% common Failure and withdrawal rates in before calculus courses are often dismal, with numbers between 40% and 60% common

Institute for Mathematics Learning Reform in before calculus courses over the past 5 years Reform in before calculus courses over the past 5 years Liberal Arts Mathematics, Applied College Algebra, College Algebra, College Trigonometry, Precalculus, and Applied Calculus Liberal Arts Mathematics, Applied College Algebra, College Algebra, College Trigonometry, Precalculus, and Applied Calculus Subsequent Course Success Subsequent Course Success Fall 2004-Spring 2005 students successful in a subsequent course 80% if A or B in IML course, 50% if C in IML course Fall 2004-Spring 2005 students successful in a subsequent course 80% if A or B in IML course, 50% if C in IML course

Success Rates Fall 2004-Spring 2005 Success Rates Fall 2004-Spring 2005 CourseSuccess Rates (A, B, or C grade) Liberal Arts Mathematics65.9% Applied Algebra63.3% College Algebra58.6% College Trigonometry63% Precalculus64% Applied Calculus68%

Applied College Algebra Study Theoretical Framework Theoretical Framework Curricular Revision Curricular Revision Method Method Pilot Study – Fall 04 Pilot Study – Fall 04 Full Study – Spring 05 Full Study – Spring 05 Discussion Discussion

Theoretical Framework Constructivist theory of learning Constructivist theory of learning Curriculum and Evaluation Standards (2001) Curriculum and Evaluation Standards (2001) Achieving Quantitative Literacy (2001) Achieving Quantitative Literacy (2001) CUPM Curriculum Guide for Undergraduate Courses in Mathematical Sciences (2003) CUPM Curriculum Guide for Undergraduate Courses in Mathematical Sciences (2003) A Collective Vision: Voice of Partner Disciplines (2004) A Collective Vision: Voice of Partner Disciplines (2004)

Theoretical Framework Tapping America’s Potential: The Education for Innovation Initiative (Business Round Table, 2005) Tapping America’s Potential: The Education for Innovation Initiative (Business Round Table, 2005) A Commitment to America’s Future: Responding to the Crisis in Mathematics and Science Education (Business Higher Education Forum, 2005) A Commitment to America’s Future: Responding to the Crisis in Mathematics and Science Education (Business Higher Education Forum, 2005) American Diploma Project (Achieve, Inc., 2005) American Diploma Project (Achieve, Inc., 2005)

Theoretical Framework Liberal Studies Program goals Liberal Studies Program goals Introduce the great ideas and controversies in human thought and experience, in this case the function concept. Introduce the great ideas and controversies in human thought and experience, in this case the function concept. Develop the ability to reason clearly, communicate effectively, and understand major influences of society. Develop the ability to reason clearly, communicate effectively, and understand major influences of society. Develop critical thinking by requiring logical inquiry to evaluate decisions, question posing, problem formulation, and interpretation of results. Develop critical thinking by requiring logical inquiry to evaluate decisions, question posing, problem formulation, and interpretation of results. Incorporate a writing component Incorporate a writing component

Theoretical Framework Cognitive science Cognitive science Lyn English (1997) views reasoning as embodied and imaginative Lyn English (1997) views reasoning as embodied and imaginative Learner Centered Instruction movement Learner Centered Instruction movement Treisman’s Model (1992) Treisman’s Model (1992) Collaborate on challenging problems in an environment of high expectations Collaborate on challenging problems in an environment of high expectations Weekly collaborative learning sessions in small groups with student mentors Weekly collaborative learning sessions in small groups with student mentors Rather than remediate - engage in challenging mathematics that is engaging and meaningful Rather than remediate - engage in challenging mathematics that is engaging and meaningful Faculty sponsorship in development and management of the courses Faculty sponsorship in development and management of the courses

Curricular Revision Computer enhanced Computer enhanced Vista WebCT course management software, web sites, and interactive Java applets to provide access to course materials, implement assessment, communicate with students, engage students in exploring and discovering mathematics, and manage the course grades Vista WebCT course management software, web sites, and interactive Java applets to provide access to course materials, implement assessment, communicate with students, engage students in exploring and discovering mathematics, and manage the course grades Grapher Grapher

Curricular Revision 10 online quizzes, four on-line chapter reviews, four on-line exams, and an on- line gateway pre-assessment to determine student mathematical deficiencies 10 online quizzes, four on-line chapter reviews, four on-line exams, and an on- line gateway pre-assessment to determine student mathematical deficiencies Computer laboratory component Computer laboratory component Students are peer mentored while they engage in explorations of mathematical concepts and apply mathematics to solve real world problems Students are peer mentored while they engage in explorations of mathematical concepts and apply mathematics to solve real world problems

Curricular Revision Active student learning and student accountability, implementing teaching strategies that engage students and provide informal formative feedback on their progress Active student learning and student accountability, implementing teaching strategies that engage students and provide informal formative feedback on their progress Personal Response System (PRS) Personal Response System (PRS) Implement 22 classroom participation activities that allow students to respond and receive immediate feedback Implement 22 classroom participation activities that allow students to respond and receive immediate feedback Power point slides guide the course discussion, serve as student lecture outlines, and provide instructors with real world data problems as well as a guide to key concepts Power point slides guide the course discussion, serve as student lecture outlines, and provide instructors with real world data problems as well as a guide to key concepts

Curricular Revision Supplemental Instruction (SI) Supplemental Instruction (SI) Fall paper worksheets focusing on selected skills or applications Fall paper worksheets focusing on selected skills or applications Spring 2005 grounded in programmed instruction (McHale, Christenson, and Roberts, 1986) and implemented with PRS Spring 2005 grounded in programmed instruction (McHale, Christenson, and Roberts, 1986) and implemented with PRS Two versions of SI Two versions of SI Algorithm SI focus on basic skills without context Algorithm SI focus on basic skills without context Application SI used real world applications to motivate the need for learning basic skills Application SI used real world applications to motivate the need for learning basic skills

Curricular Revision Students assigned to SI based upon performance on Pre-assessment. Students assigned to SI based upon performance on Pre-assessment. Taken after a week of reviewing college algebra prerequisites Taken after a week of reviewing college algebra prerequisites Students who scored below 80% on the first attempt were required to attend an SI review session and then retake the Pre- assessment within a week. Students who scored below 80% on the first attempt were required to attend an SI review session and then retake the Pre- assessment within a week. If the student failed to attain an 80% or higher mark on the Pre- assessment on either attempt, then they were required to attend SI for the remainder of the semester (designated Required) Students who scored above 80% were encouraged to attend SI, but their attendance was not required (designated Optional) If the student failed to attain an 80% or higher mark on the Pre- assessment on either attempt, then they were required to attend SI for the remainder of the semester (designated Required) Students who scored above 80% were encouraged to attend SI, but their attendance was not required (designated Optional) Optional students whose performance on one of three subsequent tests was less than 70% were then required to attend SI until they earned at least 70% on a later test Optional students whose performance on one of three subsequent tests was less than 70% were then required to attend SI until they earned at least 70% on a later test

Method Class randomly assigned to receive either Algorithm SI or Application SI Class randomly assigned to receive either Algorithm SI or Application SI Retired version of the standardized ACT Mathematics exam as Pre- Post-test Retired version of the standardized ACT Mathematics exam as Pre- Post-test Mathematics Attitude Inventory (MAI), developed by Mayes (2004) Mathematics Attitude Inventory (MAI), developed by Mayes (2004) Analysis of impact of supplemental on student outcomes within course components, including exams, laboratories, on-line homework quizzes, and overall grade. Analysis of impact of supplemental on student outcomes within course components, including exams, laboratories, on-line homework quizzes, and overall grade.

Method Nonequivalent Control Group Quasi-experimental research design Nonequivalent Control Group Quasi-experimental research design O1 O2 O3 X1 O4 O5 O O1 O2 O3 X2 O4 O5 O6 O1 - O4 : Pre- and Post-Math Attitude Inventory (MAI) O1 - O4 : Pre- and Post-Math Attitude Inventory (MAI) O2 – O5 : Pre- and Post-ACT Exam O2 – O5 : Pre- and Post-ACT Exam O3 : Pre-assessment of basic skills O3 : Pre-assessment of basic skills O6 : Final Exam O6 : Final Exam X1 : Algorithm SI X1 : Algorithm SI X2 : Application SI X2 : Application SI

Method Analysis was conducted using quantitative methods, including univariate and repeated-measures analyses of variance (ANOVA) and correlations. Significant main effects were further analyzed with pairwise comparisons using a Bonferroni adjustment. Analysis was conducted using quantitative methods, including univariate and repeated-measures analyses of variance (ANOVA) and correlations. Significant main effects were further analyzed with pairwise comparisons using a Bonferroni adjustment.

Research Questions What are the plausible effects of a Supplemental Instruction program targeted at students at risk of failure? What are the plausible effects of a Supplemental Instruction program targeted at students at risk of failure? Does Application SI or Algorithm SI have the greatest impact on student cognition and affect? Does Application SI or Algorithm SI have the greatest impact on student cognition and affect? What is the impact of the reformed Applied College Algebra course on student cognition and affect? What is the impact of the reformed Applied College Algebra course on student cognition and affect?

Pilot Study Fall 04 ACT Pre- Post Analysis ACT Pre- Post Analysis 2 x 2 repeated-measures ANOVA with a between-subject factor of SI Requirement (Required or Optional) and a within-subject factor of Test (ACT pretest and ACT posttest) 2 x 2 repeated-measures ANOVA with a between-subject factor of SI Requirement (Required or Optional) and a within-subject factor of Test (ACT pretest and ACT posttest) No significant difference between the cohorts on overall ACT score No significant difference between the cohorts on overall ACT score Significant main effect of ACT, F (1, 283) = , p <.001, with the posttest scaled scores (M = 19.86, SD = 3.02) exceeding the pretest scaled scores (M = 15.71, SD = 3.11) Significant main effect of ACT, F (1, 283) = , p <.001, with the posttest scaled scores (M = 19.86, SD = 3.02) exceeding the pretest scaled scores (M = 15.71, SD = 3.11)

Pilot Study – ACT Analysis CohortNPre-test Mean Pre-test Std. Dev. Post-test Mean Post-test Std. Dev. Optional Earned Optional Earned Optional Earned Required Earned Required Earned Required Earned Required Earned

Pilot Study – ACT Subscales Subscales of Pre-Algebra and Elementary Algebra (PAEA) and Intermediate Algebra and Coordinate Geometry (IACG) Subscales of Pre-Algebra and Elementary Algebra (PAEA) and Intermediate Algebra and Coordinate Geometry (IACG) All Optional cohorts made significant gains on both subscales All Optional cohorts made significant gains on both subscales Required cohorts made significant gains in both subscales when they participated in SI over 50% of the time Required cohorts made significant gains in both subscales when they participated in SI over 50% of the time

Pilot Study – ACT Subscales CohortMean Gain in PAEA Std. Dev. In PAEA Mean Gain in IACG Std. Dev in IACG Optional Earned * **2.179 Optional Earned *** ***2.675 Optional Earned *** ***2.659 Required Earned Required Earned * Required Earned *** ***2.456 Required Earned *** ***2.374

Pilot Study - Final Course Average Univariate ANOVA indicated a significant effect of Cohort on final course average, F (6, 398) = 6.970, p <.001. Univariate ANOVA indicated a significant effect of Cohort on final course average, F (6, 398) = 6.970, p <.001. Pairwise comparisons using a Bonferroni adjustment indicated that both the Required and Optional 6-9 cohorts outperformed the Required 0 cohort, p <.05. Pairwise comparisons using a Bonferroni adjustment indicated that both the Required and Optional 6-9 cohorts outperformed the Required 0 cohort, p <.05. Required and Optional cohorts outperformed the Required 0 cohort, p <.001. Required and Optional cohorts outperformed the Required 0 cohort, p <.001. No significant differences were found between Required and Optional students who earned the same number of points for SI. No significant differences were found between Required and Optional students who earned the same number of points for SI. There is also overwhelming evidence that the more you attend SI, the better you do in the course. There is also overwhelming evidence that the more you attend SI, the better you do in the course. A Required student attending SI 6 to 9 times has a course mean equivalent to an Optional-Earned 6-9 student A Required student attending SI 6 to 9 times has a course mean equivalent to an Optional-Earned 6-9 student A Required student attending 10 to 11 times actually had a higher mean than an Optional-Earned student. A Required student attending 10 to 11 times actually had a higher mean than an Optional-Earned student.

Pilot Study - Affect Statistically significant drop in overall students’ attitudes from the beginning to end of the semester Statistically significant drop in overall students’ attitudes from the beginning to end of the semester Paired-samples t-test (t (459) = 11.92, p<.001). Paired-samples t-test (t (459) = 11.92, p<.001). Required SI students had a significantly poorer attitude at the end of the semester then their Optional SI counterparts Required SI students had a significantly poorer attitude at the end of the semester then their Optional SI counterparts

Pilot Study - Affect OptionalRequired MeanStd. Deviation MeanStd. Deviation Attitude Survey Attitude Survey 2*

Full Study – Spring 2005 PRS in Supplemental Instruction and offering Algorithm SI and Application SI at distinct times in large lecture classrooms, allowed more control over tracking students. The question of which type of SI was most effective could now be addressed. PRS in Supplemental Instruction and offering Algorithm SI and Application SI at distinct times in large lecture classrooms, allowed more control over tracking students. The question of which type of SI was most effective could now be addressed.

Full Study – Spring 2005 A 2 x 2 x 2 ANOVA with between-subjects factors of Section (Application or Algorithm) and SI Requirement, and a within-subjects factor of Test was used to analyze the data. A 2 x 2 x 2 ANOVA with between-subjects factors of Section (Application or Algorithm) and SI Requirement, and a within-subjects factor of Test was used to analyze the data. There were no significant differences between the Application SI and Algorithm SI sections on the ACT pretest or posttest There were no significant differences between the Application SI and Algorithm SI sections on the ACT pretest or posttest

Full Study – Spring 2005 Significant main effect of Test, F (1, 205) = , p <.001, such that performance on the posttest (M = 20.15, SD = 2.47) was better than on the pretest (M = 18.42, SD = 2.64) Significant main effect of Test, F (1, 205) = , p <.001, such that performance on the posttest (M = 20.15, SD = 2.47) was better than on the pretest (M = 18.42, SD = 2.64) Significant main effect for SI Requirement, F (1, 205) = , p =.001, with the Optional group (M = ) outperforming the Required group (M = 18.89). Significant main effect for SI Requirement, F (1, 205) = , p =.001, with the Optional group (M = ) outperforming the Required group (M = 18.89). There were no significant interactions. There were no significant interactions.

Mean scaled ACT scores and standard deviations by section and Supplemental Instruction requirement SectionSupplemental Instruction Requirement ACT Pre Mean ACT Pre SD ACT Post Mean ACT Post SD AlgorithmOptional Required ApplicationOptional Required

Three Subscales of ACT overall effect Three Subscales of ACT overall effect PAEA subscale: post-test (M = 16.82, SD=3.21) exceeding pre-test performance (M = 14.66, SD = 3.67), F(1, 205) = , p <.001 PAEA subscale: post-test (M = 16.82, SD=3.21) exceeding pre-test performance (M = 14.66, SD = 3.67), F(1, 205) = , p <.001 IACG subscale: post-test (M = 8.44, SD = 2.69) exceeding pre-test performance (M = 6.63, SD = 2.64), F(1, 205) = , p <.001 IACG subscale: post-test (M = 8.44, SD = 2.69) exceeding pre-test performance (M = 6.63, SD = 2.64), F(1, 205) = , p <.001 PGTRG subscale: post-test (M =7.54, SD=2.83) exceeding pre-test performance (M = 6.29, SD = 2.64), F(1, 205) = 43.40, p <.001 PGTRG subscale: post-test (M =7.54, SD=2.83) exceeding pre-test performance (M = 6.29, SD = 2.64), F(1, 205) = 43.40, p <.001

Three Subscales of ACT by SI Type Three Subscales of ACT by SI Type PAEA subscale: Optional SI (M = ) outperformed Required SI (M = 15.12), F(1, 205) = , p <.001 PAEA subscale: Optional SI (M = ) outperformed Required SI (M = 15.12), F(1, 205) = , p <.001 IACG subscale: Optional SI (M = 8.69) outperformed Required SI (M = 7.155), F(1, 205) = , p <.001. IACG subscale: Optional SI (M = 8.69) outperformed Required SI (M = 7.155), F(1, 205) = , p <.001. PGTRG subscale: Optional SI (M = 8.02) outperformed Required SI (M = 6.545), F(1, 205) = , p <.001 PGTRG subscale: Optional SI (M = 8.02) outperformed Required SI (M = 6.545), F(1, 205) = , p <.001

ACT Subscale by SI Type SubscaleSectionSI requirement ACT Pre Mean ACT Pre SD ACT Post Mean ACT Post SD PAEAAlgorithmOptional (24)Required ApplicationOptional Required IACGAlgorithmOptional (18)Required ApplicationOptional Required PGTRGAlgorithmOptional (18)Required ApplicationOptional Required

ACT by Cohort Optional 0 students outperformed the Optional 1-5, Optional 11-13, and all Required cohorts on the ACT. Optional 0 students outperformed the Optional 1-5, Optional 11-13, and all Required cohorts on the ACT. No other cohorts were significantly different from one another. No other cohorts were significantly different from one another. This supports the previous findings that while the overall course influenced ACT posttest scores, SI had little impact This supports the previous findings that while the overall course influenced ACT posttest scores, SI had little impact

ACT by SI Attendance CohortNMean ACTSD Optional Optional Optional Optional Required Required Required Required

Final Course Average by Cohort All Optional cohorts outperformed both the Required 0 and Required 1-5 cohorts All Optional cohorts outperformed both the Required 0 and Required 1-5 cohorts Required 0 cohort was outperformed by all other Required cohorts Required 0 cohort was outperformed by all other Required cohorts No other cohorts were significantly different from one another No other cohorts were significantly different from one another This indicates that SI had an overall course impact for Required students who attended at least 50% of the time This indicates that SI had an overall course impact for Required students who attended at least 50% of the time

Final Course Average by Cohort CohortNMean Course AvgSD Optional Optional Optional Optional Required Required Required Required

Exam Average by Cohort Optional 0 cohort outperformed all of the Required cohorts Optional 0 cohort outperformed all of the Required cohorts Optional 1-5 cohort outperformed the Required 0 and Required 1-5 cohorts Optional 1-5 cohort outperformed the Required 0 and Required 1-5 cohorts Optional 6-10 and Optional cohorts outperformed the Required 0, Required 1- 5, and Required cohorts Optional 6-10 and Optional cohorts outperformed the Required 0, Required 1- 5, and Required cohorts Required 6-10 cohort significantly outperformed the Required 0 cohort. No other comparisons were significant. Required 6-10 cohort significantly outperformed the Required 0 cohort. No other comparisons were significant.

Exam Average by Cohort CohortNMean Exam AvgSD Optional Optional Optional Optional Required Required Required Required

Lab Average by Cohort All Optional cohorts outperformed the Required 0 cohort All Optional cohorts outperformed the Required 0 cohort Required 6-10 and Required cohorts outperformed the Required 0 cohort Required 6-10 and Required cohorts outperformed the Required 0 cohort No other comparisons were significant No other comparisons were significant

Lab Average by Cohort CohortNMean Lab AvgSD Optional Optional Optional Optional Required Required Required Required

Exam by Exam In these analyses the cohorts are determined by the most previous test performance In these analyses the cohorts are determined by the most previous test performance Exam 1 (2 SI sessions) Optional 0 cohort outperformed all Required cohorts Exam 1 (2 SI sessions) Optional 0 cohort outperformed all Required cohorts Exam 2 (4 SI sessions) all Optional cohorts outperformed the Required 1 and 2 cohorts. The Optional 0, 1, and 3 outperformed the Required 3 and 4 cohorts. Only the Optional 0 cohort significantly outperformed the Required 0 cohort. Required 0 cohort outperformed the Required 2 cohort Exam 2 (4 SI sessions) all Optional cohorts outperformed the Required 1 and 2 cohorts. The Optional 0, 1, and 3 outperformed the Required 3 and 4 cohorts. Only the Optional 0 cohort significantly outperformed the Required 0 cohort. Required 0 cohort outperformed the Required 2 cohort

Exam by Exam In these analyses the cohorts are determined by the most previous test performance In these analyses the cohorts are determined by the most previous test performance Exam 3 (4 SI sessions) Optional 0, 1, 2, and 3 cohorts outperformed all Required cohorts on Exam 3. Optional 4 cohort significantly outperformed Required 0 and 1 cohorts Exam 3 (4 SI sessions) Optional 0, 1, 2, and 3 cohorts outperformed all Required cohorts on Exam 3. Optional 4 cohort significantly outperformed Required 0 and 1 cohorts Final Exam (3 SI sessions) Optional 0 and 3 cohorts outperformed all Required cohorts. Optional 1 and 2 cohorts outperformed Required 0, 1, and 3 cohorts. Final Exam (3 SI sessions) Optional 0 and 3 cohorts outperformed all Required cohorts. Optional 1 and 2 cohorts outperformed Required 0, 1, and 3 cohorts.

Exam by Exam Overall the examination of SI exam by exam reveals that Optional SI students continue to outperform their Required SI counterpoints. Surprisingly, Required SI students who attend all of the required sessions did not score significantly better on the following exam, although the trend was that exam scores did increase as attendance increased. Overall the examination of SI exam by exam reveals that Optional SI students continue to outperform their Required SI counterpoints. Surprisingly, Required SI students who attend all of the required sessions did not score significantly better on the following exam, although the trend was that exam scores did increase as attendance increased.

Attitude Subscales A paired-samples t-test was conducted on the average responses for the Attitude Survey A paired-samples t-test was conducted on the average responses for the Attitude Survey No significant difference between the two administrations of the Attitude Survey No significant difference between the two administrations of the Attitude Survey No significant correlation between students’ final grades and their responses on the Attitude Surveys No significant correlation between students’ final grades and their responses on the Attitude Surveys There were some general effects on the Utility, Locus of Control, and Belief subscales, where the pretest scores were more positive than the posttest scores There were some general effects on the Utility, Locus of Control, and Belief subscales, where the pretest scores were more positive than the posttest scores There were no significant differences between the Application SI and Algorithm SI sections There were no significant differences between the Application SI and Algorithm SI sections

Attitude Subscales Attitude Survey 1Attitude Survey 2 SubscaleNumber of Questions MeanSDMeanSD *Utility Concept vs. Skill *Locus of Control *Belief about Math Technology

Discussion