1. Asking Students What They Think: Student User Experience (UX) Research Studies to Inform Online Course Design
AACE E-Learn Conference
Washington, D.C.
November 14-16, 2016

2. https://sites.psu.edu/canvasux/
Who We Are
Amy Garbrick—Director, Learning Design College of Information Sciences and Technology
Ronda Reid—Instructional Designer College of Information Sciences and Technology
Dr. Vicki Williams—Assessment Manager Teaching & Learning with Technology
Dr. Andrea Gregg—Online Learning Research World Campus

3. https://sites.psu.edu/canvasux/
Overview
Gathered Student Feedback through Two Research Studies
Process for Those Studies
Why We Found it Beneficial

4. What is User Experience (UX)?
User Experience (UX) focuses on having an understanding of users, what they value and need, and promoting the quality of the users’ interaction with a specified service or product —Usability.gov
Common in the online realm, but not as common in online education

5. https://sites.psu.edu/canvasux/
Our Environment
Transitioning to a new Learning Management System (LMS)—Canvas
Good time to redesign/re-evaluate what we were doing with courses
After many, many, many meetings, we developed a new course template for our courses and adopted new design decisions

6. https://sites.psu.edu/canvasux/
Our Environment
We felt pretty good about it
Image: The Verge

7. What Do Students Think about What We’re Doing?
Simple question
Apply to any LMS or online course
Image: quickmeme.com

8. https://sites.psu.edu/canvasux/
Two Main Efforts
User Experience (Think-Aloud)
User Preference (Survey)

9. User Experience (Think-Aloud)

10. https://sites.psu.edu/canvasux/
User Experience: Goal
By better understanding how students interact with the course, we can address potential barriers to student learning that may be present in the course design.
Image: Dilbert by Scott Adams

11. User Experience: Process
Used a complete course that had course template applied
Used simple, but pointed, tasks that represented what students may typically do in a course
Find when an assignment is due
Find your grade for a particular assignment
Contact your group
Used student interns to verify the process and test
Residential undergraduate population
AKA, our beloved guinea pigs
Image: Megan van der Elst

12. User Experience: Process, Continued
Asked online faculty to distribute a Recruitment survey to our online students
Yielded 43 responses over a 10-day window
Online students visited our main campus
5 total participants
Represented online learner demographics
User experience best practices: 5 is enough
From Jakob Nielsen, User Experience Guru

13. User Experience: Process, Continued
Recorded mouse movements while interacting with course design
Made audio and video recordings of each student
Asked students to record ranking/thoughts on paper form
Used a scale of 1 (easy) to 5 (difficult) to rank tasks
Gave participants a $50 gift certificate to Amazon.com

14. User Experience: Why We Found it Beneficial
Main benefit: watching students go through design and seeing potential design pitfalls
Assumptions we make as Learning Designers are not the same assumptions students make
Study was replicated by four other colleges/units with their designs

15. User Experience: Why We Found it Beneficial, Continued
Students want to be heard: one student traveled 8 hours just to be a part of the testing
“I wish I could have some input and given that opportunity I felt like I should take advantage of it. Being able to contribute towards that might be able to help somebody else out too.“
— Subject 2

16. User Preference (Survey)

17. https://sites.psu.edu/canvasux/
User Survey: Goal
A better understanding of student design preference allows us to improve course design and make it more intuitive to students.
Image: BestMotivationalPosters.com

18. https://sites.psu.edu/canvasux/
User Survey: Process
Distributed survey to almost 5,000 online students
Response rate: slightly more than 10%
Took students about 20 minutes to complete survey
Survey open for seven-week window
Presented them multiple screenshots and asked preferences and questions related to:
Course landing page
Navigation
Syllabus

19. User Survey: Process, Continued
Gave students opportunity to use a scale to rank preferences
Students also wrote in comments
Students who completed survey were entered into a drawing for one of five $25 gift certificates to Amazon.com

20. User Survey: Why We Found it Beneficial
Won’t cover specifics, but…
Findings point to potential for broader reaching consistency implications across the university that transcends individual course design
Students have desire for an affective connection with their University Community
Findings point to task-based nature of adult students
Wanted to be able to find pertinent information quickly and efficiently
Get an overview of work required

21. User Survey: Why We Found it Beneficial, Continued
Received many comments indicating students appreciated being asked
“Thank you for the opportunity to contribute to the learning experience at PSU.”
Image: Funcatpictures.com

22. https://sites.psu.edu/canvasux/
Bottom Line
Challenge Design Assumptions
Involve Students in the Process
Image: amorebeautifulquestion.com

23. https://sites.psu.edu/canvasux/
For More Info

24. https://sites.psu.edu/canvasux/
Thank You!
Questions?