1. Effects of Targeted Troubleshooting Activities on Student Confidence in a Statistics Computer Lab
Meredith A. Henry, M.S.
Department of Psychology
University of Alabama at Birmingham
Please note: this is a suggested template, please feel free to edit as needed, but please keep this to three slides.

2. THE PROBLEM
Successful psychology students are competent conducting and reporting statistical analyses.
But, students often struggle with/hold negative attitudes towards statistics and research design. (Mills, 2004)
One reason may be lack of confidence/intimidation when confronted with analytical packages such as SAS (Statistical Analysis System).
Previous experience teaching a graduate statistics lab also suggests:
A) Students struggle with troubleshooting SAS code/analyses
B) Students express preference for hands-on class activities
The current study tested the effectiveness of a series of exercises, designed to give students experience troubleshooting the SAS program, in increasing student confidence across different domains of statistical skills
Questions from the Audience

3. THE STUDENTS
PY 716-L is the lab section of the first of a three-course graduate statistics sequence for students enrolled in UAB’s three psychology graduate programs.
Student characteristics:
Questions from the Audience

4. THE INNOVATION: Make it Work Exercises
Selections of “bad code” given to students
Designed to include “common” SAS errors
Students’ task:
1) Use knowledge from lectures and error messages in the SAS log window to fix coding errors
2) Interpret results of analyses once run
3) Report results in appropriate written form
Exercises assigned at the beginning of the class. Students have 10 minutes to complete them, then the whole class reviews and discusses.
Rated confidence for a variety of statistical skills pre- and post-course on 5-point Likert scale
Traditional course structure:
Lecture gives students appropriate code to run a variety of analyses in SAS 9.3.
Assignments ask students to modify code to successfully run, interpret, and report analyses
A group project involves conceiving a study design, creating data, choosing analyses, running analyses, interpreting results, and reporting conclusions.
Questions from the Audience

5. DID IT WORK? Increasing Overall Confidence
Repeated measures t-tests; all significant at p<.001
Questions from the Audience

6. DID IT WORK? Group A vs. Group B
Topics covered in the course were divided into 2 groups:
Group A topics had a MIW exercise assigned in both 2014 and 2015.
Group B topics only had a MIW exercise assigned in 2015.
In Fall 2014, students reported significantly more confidence gain for Group A topics than for Group B topics for both analytic [t(11) = 4.12, p < .005] and troubleshooting [t(12) = 2.50, p < .05] confidence.
This suggested that the MIW exercises caused greater confidence gains.
If so, we would expect no significant difference between Group A and Group B in Fall 2015.
However, we saw the same pattern of results. Students still gained more confidence in both domains for Group A vs. Group B topics [t(11) = 4.19, p < .005; t(11) = 2.73, p <.05]
In an attempt to explain this surprising finding, we looked back at the properties of the MIW exercises themselves.
Questions from the Audience

7. DID IT WORK? Group A vs. Group B
Topics covered in the course were divided into 2 groups:
Group A topics had a MIW exercise assigned in both 2014 and 2015.
Group B topics only had a MIW exercise assigned in 2015.
In Fall 2014, students reported significantly more confidence gain for Group A topics than for Group B topics for both analytic [t(11) = 4.12, p < .005] and troubleshooting [t(12) = 2.50, p < .05] confidence.
This suggested that the MIW exercises caused greater confidence gains.
If so, we would expect no significant difference between Group A and Group B in Fall 2015.
However, we saw the same pattern of results. Students still gained more confidence in both domains for Group A vs. Group B topics [t(11) = 4.19, p < .005; t(11) = 2.73, p <.05]
In an attempt to explain this surprising finding, we looked back at the properties of the MIW exercises themselves.
Questions from the Audience

8. WHAT’S GOING ON? Exploring the Influence of Practice
Statistics is a discipline that often “builds” upon itself
Some SAS skills (e.g., entering data, running tests of normality, etc.), while their own topic of instruction, are also necessary parts of more advanced analyses.
As such, some skills will appear in multiple lectures, assignments, and MIW exercises.
We proposed that students would gain more confidence for these “practiced” skills vs. “unpracticed” skills.
2015 students did report significantly more confidence for “practiced “skills, t(11) = 3.68, p < .005.
However, 2014 students reported no difference in confidence based on amount of “practice”, t(11) = 2.109, p = n.s.

9. By chance, many Group A topics were also “practiced” skills.
Fall 2014 students were exposed to “practiced” skills multiple times in lectures, but didn’t have as many MIW opportunities for hands-on engagement.
Fall 2015 students had MIW activities for all topics, but still ended up “practicing” some of them more.
I propose that neither the MIW exercises nor repeated exposure to SAS skills alone is sufficient to optimize gains in student confidence. Rather, targeted, hands-on activities like the MIW exercises, administered multiple times will lead to the greatest levels of student confidence in statistical skill.
It is possible that the consistently higher gains in confidence for Group A skills, and the failure of Fall 2014 students to benefit from “practice” both reflect the same mechanism
Questions from the Audience

10. DID THE STUDENTS LIKE THEM?
Across both semesters, 60% of students ranked MIW as one of the top three most useful aspects of the course.
Qualitative Feedback:
“I found them helpful because they featured common mistakes.”
“Helpful—hands on experience with the code is always good.”
“They were helpful; a few more would have been great.” [2014, emphasis added]
“I think the more practice you get at picking up on those small errors can make a big difference.”
“I like them. I thought they were very helpful in understanding the syntax via troubleshooting. I just think doing more of them would be better. Maybe start the class with that?” [2014, emphasis added]
I liked the opportunity to work independently; I was able to process and retain the info better this way compared to during lecture.

11. BOTTOM LINE
1) The MIW exercises are effective in increasing student confidence, although future studies should include more subjects, with greater diversity, and try to systematically investigate the possible MIW*practice interaction.
2) Greater confidence was associated with higher grades (r = 0.82, p = .001), and 3) student enjoy the MIW exercises
Thus, there is sufficient evidence supporting continued use & refinement of MIW exercises to improve the statistics lab.
Questions from the Audience