1 Student Perceptions of Assessment Placement: Results and Implications Gregory Anderson ESL Dept (faculty) De Anza College 14 April 2011

2 Session Structure Context for Study Rationale for Mixed Methods Quantitative Study Qualitative Study Lessons learned: cooperation, coordination, and change

3 Context for Study Budget and staffing Faculty perceptions Student complaints Developmental education task force

4 Quantitative Study: Sampling Transfer and developmental level All four basic skills Multiple sections at each level Time of day Instructor status

5 Quantitative Study: Responses N= % had no recollection of taking a placement test 3/4 of the students claimed they had not prepared at all 3/4 felt that time was not an issue (common reason for this revealed in qualitative study)

6 Quantitative Study: Responses Over 80% never retook an exam Of those, less than half saw an improvement 90% “understand the need to take placement tests” 1 in 4 students heard about placement tests from a HS teacher or staff member Those students had a significantly higher degree of confidence in the results of their placement

7 Quantitative Study: Responses Well under half performed as expected, though this was much higher depending on how early they heard about the tests Over 70% said they would take a 30-minute training to help get ready for the placement test Of the students who appealed, less than 2/3 found it smooth and easy to understand

8 Quantitative Study: Results

9 Qualititative Study: Structure N=80 (four focus groups) ethnic, gender, age, level, language, diversity 80 minute sessions typical qualitative anaylis protocol (transcription, repeated reading, theme identification, narrowing, reporting)

10 Qualititative Study: Themes a. Process: taking tests, receiving results b. Students' preparation for placement c. Role and function of placement tests and assessment d. Importance of placement tests--before and after e. Reactions to and actions toward to placement level f. Content of the test g. Students’ suggested improvement

11 Qualititative Study: Theme A a. Process: taking tests, receiving results I was furious When I heard about it, it was just annoying The counselors, they really don’t care, the clerks, they just don’t care I never heard of anyone taking a placement test and getting placed higher It's scary

12 Qualititative Study: Theme B b. Students' preparation for placement test I heard friends talk about it and I think they actually kind of studied for it. I studied for it, so I just went in and took it. I went online because I thought there were some sample tests that you could look at, and that kind of eased my nervousness There’s no guideline. There’s no practice test. There’s nothing that shows you what you need to study or where you need to study I had no idea how things were going, so I just figured you showed some documentation so you can register for that quarter and then you can just enroll in classes online or in person.

13 Qualititative Study: Theme C c. Role and function of placement tests and assessment Waste of time. it’s a bullshit test. It’s not possible to really know anything about us from this kind of thing which is really quick and then it makes no sense when it puts you in a class. That's not assessment or whatever. Wasting people’s money. Waste your time. You can potentially hurt your GPA. You’re also hurting other people. You also lose your motivation. Every placement test has sort of a bias. There is a bias to the test and how it’s done. It’s just the fact that there’s some teachers that any test that you take there’s a person writing that test. The person writing that test writes it to what they believe is important, to where another teacher will believe something else is important

14 Qualititative Study: Theme D d. Importance of placement tests--before and after Because the only test I had to take was math because I had taken English in high school. So I was completely apathetic about the placement tests and getting the results didn’t change my mind because the results just meant all I had to do was take one class and then I would be covered with math for college. I wish that I knew something about these tests before I took them. Now I’ve been here for a long time and still need to take a lot more classes. I think I’ve wasted a lot of time and money. Maybe I didn’t need to. Before, I was not so sure, but after taking the test, I think it's very important

15 Qualititative Study: Theme E e. Reactions to and actions toward to placement level I was pissed because I was placed in the lowest of the lowest of math I feel like I have kind of been let down both times when I got an assessment. Devastating. Because I thought I did really well and I thought I knew all the material or most of it, but I guess I didn’t think the scores proved that. I started doubting myself I was kind of surprised. I felt like definitely like I’d been placed at the wrong level. I was frustrated because I knew that stuff. You got the problem, but then one little mistake, you got the answer wrong. I find myself in a class where nobody was speaking English... They put me in three classes that I wasn’t supposed to be in. Angry and stupid that I couldn't pass math

16 Qualititative Study: Theme F f. Content of the test The English ones are always tricky I just think that the questions they put there is really not relevant, like it makes no sense to be there. Way too long, you get tired. Especially the math section It’s very long and you don’t know when it’s going to end. It’s like it’s never ending. You keep going and it depends if you get the answer right or wrong, depending on the difficulty, so the test just kept getting longer. They only give you one topic to write about, it’s only a half an hour

17 Qualititative Study: Theme G g. Students’ suggested improvement The counselors, admissions, your teachers, even the cashier should tell you about them The clerks there ought to be more knowledgeable about the actual tests and be more sensitive and there needs to be some kind of progress when you’re taking the test so you know They probably need to give you a little more information as to what kind of a test you’re going to be taking Personalizing would be a good step, transparency as to what exactly you’re going to be tested over and the reasons as to why you’re being tested. Having an orientation before you take a placement test could go a long way as to preparing students for what they’re going to expect, and keep the students who don’t want to follow that path from wasting their time

18 Other lessons learned Coordination Cooperation Change

19 Measures of Student Success Questions?