1. ORMATYC 2016 ALEKS PPL Experience at Washington State University
Increased Student Success through Proper Math Placement
Sandy Cooper, WSU
All of us are committed to the success of our students and an important component is placing students into math classes for which they are prepared and have a good chance of succeeding. Too high- they struggle, do not pass, and have a higher chance of leaving the institution and giving up on their dreams. Too low- they are bored, do not work hard, and often do not pass due to lack of effort. Key- find the sweet spot that places them into a course that is challenging, yet manageable.

2. Outline Brief background on math placement at WSU Theory behind ALEKS
ALEKS PPL
Implementation of ALEKS PPL at WSU
Change in success rates attributable to accurate placement
Theory behind ALEKS to give you a sense of why we have so much confidence in this product.

3. Background on Math Placement at WSU
1995- beginning of our math placement journey
Original tools: WA State MPE (advanced and intermediate), SAT, ACT, AP Calc exam
Complicated chart and time consuming to check the various options and enforce placement
Started using math placement in 1995 in an attempt to address high failure rates

4. Partial Placement Chart
WSU Eligibility Chart
WSU Math Course Eligibility(1)
Washington Math Placement Test
SAT
Math
ACT
Math (2)
College Board
A.P. AB Test (3)
Prerequisite Course Completion (minimum grade of C required)
Intermed.
Advanced
105 Exploring Math(4)
205 Basic Stat
212 Stat Intro 15 10
500 18 NA
99/101/103
106
College Algebra 20 12
540 22
107
Precalculus 23 14
580 24
108
Trigonometry
201
Business Precalculus
251
K-6 Teacher’s Math
202 Bus Calc 30
650 28 2
106, 107, or 201
140 Bio Cal
171 Calc I
680
107 or
106 &108
Too complicated, required manual checking of prerequisites, an awkward and alienating appeal process, and not effective.
Had been using ALEKS as a course supplement for several years and started using ALEKS Prep for Calculus as a placement tool for STEM students in 2009.

5. 2010 Internal Study of Effectiveness of Math Placement
Process used- Identified how students
placed into a course and grade earned
Criteria- successful placement tool if at
least 70% passed course with a C or
better
Results- discouraging and disappointing
Process used- Determined which students placed into each course by a particular measure (e.g. Quantitative SAT score). Calculated the proportion of those students who passed the course with a C or better.
Criteria- Tool was considered effective if 70% or greater passed.
Results- with the exception of ALEKS, we might have been better served by using shoe size to place students.

6. Move to ALEKS as the Primary Math Placement Tool
Fall directive from the provost to make all placement exams on-line (English and Math)
Immediate inclination was to use ALEKS, but it was really designed as a learning tool, not a placement tool. Contacted ALEKS about using it as a placement tool and learned that they were in the process of developing a commercial placement tool based on ALEKS. Accepted as a beta test site and moved to ALEKS as our primary math placement tool for the Fall 2011 incoming class.

7. WSU Pullman Math Placement Chart
WSU Math Course Eligibility
ALEKS
College Board
A.P. AB Calculus Test
Prerequisite Course Completion (minimum grade of C required)
105 Exploring Math
35% NA
99/101/103
106 College Algebra
40%
108 Trigonometry
106
140 Calculus for Biologists
60% 2
106 &108
171 Calculus I
70%
201 Business Precalculus
202 Business Calculus
50%
106 or 201
205 Statistical Thinking
212 Intro to Statistical Methods
251 Fundamentals of Elementary
Mathematics I

8. Theoretical Foundation for ALEKS- Knowledge Space Theory
Research in mathematical
cognitive science by Jean-Claude
Falmagne (NYU and UC-Irvine) and
Jean-Paul Doignon (University of
Brussels).
1992- Falmagne obtained an NSF grant to
develop ALEKS software based on
knowledge space theory
1996-ALEKS Corporation founded
Wonderful paper by the creators of ALEKS available on the ALEKS website- The Assessment of Knowledge, in Theory and Practice by Jean-Claude Falmagne, Jean-Paul Doignon, Eric Cosyn, and Nicolas Thiéry.

9. Knowledge Structures Precedence relations and diagrams
Knowledge States
Learning Paths
Outer and Inner Fringes

10. Example of a precedence diagram
Precedence relations- built on the assumption that some pieces of knowledge normally precede other pieces of knowledge
Example of a precedence diagram a
Precedence diagram of six types of algebra problems illustrated in table on next slide. c b
Precedence relations may be due to the fact that some prerequisites are required before new knowledge can be mastered. Could also be due to historical reasons or teaching conventions. d e f

11. Six types of problems in Elementary Algebra Problem Type
Six types of problems in Elementary Algebra Problem Type Sample problem
Word problem on proportions
Plotting a point in the coordinate plane
Multiplication of monomials
Find the greatest common factor
of two monomials
Graphing the line through a given point with a given slope
Writing the equation of a line through a given point and perpendicular to a given line
A car travels on the freeway at an average speed of 52 miles per hour. How many miles does it travel n 5 hours and 30 minutes?
Using the pencil, mark the point at the coordinates (1,3).
Perform the following multiplication:
and simplify your answer as much as possible.
Find the greatest common factor of the expressions 14t6y and 4tu5y8.
Graph the line with slope -7 passing through the point (-3, -20).
Write an equation for the line that passes through the point (-5,3) and that is perpendicular to the line
8x+5y=11.

12. Diagram of a precedence relation for Beginning Algebra

13. Knowledge States
Let K represent all the possible knowledge states represented by the precedence diagram given earlier.
K= { Ø, a, b, ab, ac, abc, abcd, abce, abcde,
abcdef}
Note that c is not a knowledge state because c is preceded by a.

14. Learning Paths
Six possible learning paths associated with this precedence diagram. In realistic knowledge structures such as that for precalculus, the number of feasible knowledge states and feasible learning paths is very large.

15. Outer and inner fringes of a knowledge state
Outer fringe- set of all problems p such that adding p to the current knowledge state creates a new knowledge state Inner fringe- set of all problems p such that deleting p from the current knowledge state gives another knowledge state Ex. Knowledge state abc- outer fringe is { d,e}, inner fringe is {b,c}
These two fringes are the main building blocks of the ‘navigational tool’ of the system.

16. More complex example
Once a current knowledge state is identified, you know the problems in both the inner fringe and the outer fringe. Outer fringe- what student is ready to learn; inner fringe- what student has most recently mastered. If a student is having difficulty learning a new concept (represented by a problem in the outer fringe), reviewing concepts from the inner fringe is a profitable place to focus.
Outer and inner fringes are also a good way to characterize the knowledge state.

17. How ALEKS Operates in Learning Mode
Initial assessment- identifies a students current knowledge state
Learning mode- student is provided choices of problems to work on from his or her outer fringe. Once a problem type is mastered, the knowledge state is adjusted.
Periodic progress assessments- verify and adjust, if necessary, current knowledge state
Tutorial, initial assessment, learning mode, progress assessment

18. ALEKS Acronym A - Assessment LE – Learning KS – Knowledge Spaces
Represents the cycle of assessment and learning used when ALEKS is used as a learning supplement.

19. ALEKS Placement, Preparation, and Learning (ALEKS PPL)
Commercially available placement tool introduced in Fall 2013
On-line Placement Package
Initial assessment
Six week learning module (Now 6 months!)
Up to 4 re-assessments
The learning module has sense been extended to 6 months! This package recognizes that students do not always assess at the level they are capable of attaining. The learning module allows the student to spend time refreshing their skills before reassessing. The package also allows institutions to require a certain number of hours of review before the next re-assessment. At WSU we require 5 hours before the first re-assessment and 3 hours before the second.

20. Types of Problems
Over 300 problem types from Basic Math through Precalculus
Problems are algorithmically generated, and vary in numbers, narratives, and other parameters
Questions are open-ended, and avoid multiple-choice
Answer may require students to enter a numerical value, write an algebraic expression, or plot the graph of a function

21. Sample Problem

22. Format of Answers
Some topics do not specify the format of the answer. In that case, ALEKS will accept all equivalent answers. This is such an example.
Since no requirement regarding the format of the answer is stated in the problem, entering the non-simplified ratio of rise over run between the two red points results in a correct answer.

23. How does it work?
Student gets question, based on response to the question, he is given a second question that he should have a 50% of knowing based on the previous response. Continues in this manner and in approximately 30 questions can identify a student’s current knowledge state.
The assessment is adaptive. At all times, ALEKS asks a question for which its expectation (based on the student’s evolving knowledge state) that the particular student will answer correctly is as close as possible to 50 percent. These are the most informative questions. (If ALEKS were to a question for which the probability that the student will answer correctly is substantially more (or less) than 50%, little information would be gained from asking it.)

24. How does it work? (cont.)
At the beginning of an ALEKS placement assessment, ALEKS asks questions that are distributed very roughly in the “middle” of the large range of mathematics covered. As the assessment progresses, questions quickly become focused on the most advanced areas the individual has mastered, whether these areas pertain to Basic Math, or PreCalculus, or anything in between.
For the student, the questions remain challenging throughout the assessment, not overly easy or unduly difficult. Thus, almost every student who takes an ALEKS assessment will have answered very roughly half of the problems correctly. Each student, however, will be correctly evaluated by the assessment in accordance with the particular student’s precise knowledge.

25. Implementation of ALEKS PPL at WSU
Implemented in Fall 13.
Totally uproctored and used cut scores to the middle or lower end of the suggested range.
In second year, instructors were complaining about under preparation of students. Also, concerns about cheating were growing.

26. First Concern- Cut Scores
Data analytics built into ALEKS PPL
Upload grades for a particular course by student ID number
Set certain parameters such as what you consider to be a passing grade
Immediate results
Passing grade- varies based on whether it is a terminal course or prerequisite course in a longer sequence

27. Based on this we recognized that the cut score for Math 103 was way too low.

28. Second Concern- Cheating
Placement data indicated 11% of students may be cheating
Results for ALEKS Placement as of September 30, 2014
Placement
Courses
Minimum Score
Course Level
Initial Assessment # Students
Best Assessment # Students
Best Assessment Percentage
Increase in Level
Frequency
100
726
449
10%
3299
103 30 1
867
642
15%
330
105, 205, 212 40 2
447
437
274
251 45 3
674
650
232
106, 201 55 4
611
727
17%
141
106A, 151, 202 65 5
290
377 9% 85
140 70 6
509
627
14% 27
171 80 7
489
11% 10
Totals
4398
100%
Total
Flag Count
495
Check
11% of the students taking the exam might be cheating- Unreasonable jumps in scores.
Anecdotal evidence pointed to a little cottage industry that had sprung up around placement testing. Consultated with ALEKS Corp. regarding what other institutions were experiencing it and how they were dealing with it, went to proctored re-assessments.

29. Change in Success Rates
Fall 14
% students earning C-, D+, D, W, F
Fall 15
Chi-square test
p-value
Math 103
55.1
35.3
<<.0001
Math 171
32.2
26.1
0.028
Math 201 43
27.9

30. Conclusions ALEKS PPL has a solid research-based foundation
Opportunity for review/remediation built in to the package
Accurately assesses a student’s current knowledge state in approximately 30 questions
Data analytics built in and easy to use