1. Use of Visual Analog Scales in Occupational Therapy Scholarship of Teaching and Learning
Margaret L. Morris OTD OTR/L BCP
Tufts University – Boston School of Occupational Therapy
Background
Content Concept Map (abbrev)
Assignment Concept Map
“Big Rocks”
One common measure of student satisfaction with a university course and instructor is the anonymous student course evaluation at the end of each semester.
Students have the opportunity to anonymously evaluate a course and the instructor through the Tufts University system each semester, providing general ordinal and qualitative data.
This data is delivered to the instructor; however, provides no infomration to the instructor that is directly related to the learning objectives for the course.
Given the broader data supplied to the instructor post-course in the anonymous, not-designed-by-instructor survey (data that is not tied to learning objectives), it is difficult to plan appropriate refinements/changes for future iterations of the course.
Instructor-designed learning activities to meet the learning objectives in the course give the instructor a sense of the student’s level of learning, but do not capture the student’s perception of their own level of comfort with knowledge gained.
Learners will value the input of family members as experts on the team.
Learners will practice how to integrate clinical reasoning and available evidence to determine the effectiveness and efficiency of assessment tools used during the evaluation process with pediatric clients and their families.
Learners will practice the evaluation process, demonstrate an emerging understanding for how to interpret evaluation results, and communicate evaluation findings (orally and written), intervention options, and recommendations.
Learners will practice how to integrate clinical reasoning, available evidence, and evaluation findings to judiciously plan and safely implement interventions for pediatric clients and their families within appropriate service delivery options.
Learners will practice how to evaluate the effectiveness of interventions (outcomes).
Purpose
Visual Analog Scale Items
Quantitative Results
This project measures student perception of knowledge comfort gained over a semester of Occupational Therapy Practice in Pediatrics, a required entry-level graduate-level course with 72 contact (class and lab) hours over 13 weeks, plus a level 1 fieldwork experience, using Visual Analog Scales (VAS).
Objectives:
Can a visual analog scale with items tied to course learning objectives provide the instructor with information useful for course content change/refinements?
Can a visual analog scale with items tied to course learning objectives be a sensitive measure for change in participant perception of comfort with level of knowledge gained in the identified areas?
Visual Analog Scales Data Analysis
Pre/Post Semester (N = 116)
Item #
Item Name
Pre-semester
Mean (SD)
Post-semester
Two-tailed
p value*
Size effect
Cohen’s d
1.1
Parent/Collab
8.41 (2.64)
12.00 (1.44)
.000
2.11
1.2
FamCtrCare
6.24 (3.00)
11.20 (1.73)
2.03
2.1
AxToolCrit
4.15 (5.47)
10.44 (2.00)
1.53
2.2
AxTool
3.70 (3.00)
10.46 (1.90)
2.69
2.3
AxToolPeds
3.53 (2.56
9.40 (2.00)
2.56
2.4
AxToolEnv
6.35 (2.55)
11.31 (1.70)
2.29
3.1
AxInterp
3.58 (2.60)
10.30 (2.40)
3.2
TxFoR
3.48 (2.38)
10.68 (1.87)
3.36
4.1
TxGoals
5.32 (2.56)
9.93 (2.13)
1.96
4.2
TxRecServDel
3.55 (2.35)
9.62 (2.10)
2.72
4.3
TxEBP/CR
4.70 (2.34)
10.78 (1.77)
2.93
5.1
TxDataOutcome
4.50 (2.56)
10.23 (2.00)
2.49
Effect Size Key
0.2
small
0.5
medium
>0.8
large
Methods
* p value < .05 is significant
Design:
Conclusions
References
This study is a quasi-experimental pre/post design; 13 weeks between data collection periods. Participants are a convenience sample of graduate occupational therapy students; six academic semesters of data were collected (Sp15, Fa15, Sp16, Fa16, Sp17, Fa17); change scores pre/post were analyzed.
N = 116 students; 1392 individual items over semesters
Findings:
In general, students’ perceptions of their comfort level with knowledge related to course learning objectives improved from pre-semester to post-semester; course objectives include knowledge, analysis, interpretation, and application of information presented.
Improvement was significant for each item/learning objective, at a p value of .000.
Effect size for all items was large, with a range of 1.53 – 3.36.
Interpretation/Application:
With visual inspection of the raw data each semester, the instructor is able to edit and refine course content in those areas which demonstrate inconsistent or limited change over the course of that semester.
With this statistical analysis of aggregated semesters, the instructor can be confident that students’ perceive they have improved their comfort level with knowledge and application of content related to the course learning objectives.
A visual analog scale can be a useful outcome measure for a graduate-level occupational therapy course, as it appears sensitive to change in students’ perception of their knowledge comfort as related to meeting the course learning objectives.
Couper, M, Tourangeau, R., Conrad, F., & Singer, E. (2006). Evaluating the effectiveness of visual analog scales: A web experiment. Social Science Computer Review, 24(2),
Gupta, J. & Bilics, A. (2014). Brief Report-Scholarship and research in occupational therapy education. American Journal of Occupational Therapy, 68, S
 Hayes, M.H., & Patterson, D.G. (1921). Experimental development of the graphic rating method. Psychological Bulletin, 18,
Maxwell, C. (1978). Sensitivity and accuracy of the visual analogue scale: A psycho-physical classroom experiment. British Journal of Clinical Pharmacology, 6,
McCarthy, M., Chang, C-H, Pickard, A., Giobbie-Hurder, A., Price, D., Jonasson, O., Gibbs, J., Fitzgibbons, R., & Neumayer, L. (2005). Visual analog scales for assessing surgical pain. Journal of the American College of Surgeons, 201 (2),
Nilson, L. (2007). The graphic syllabus and the outcomes map: Communicating your course. New Jersey: Wiley Jossey-Bass.
Data Analysis:
Pre-test marks were measured in centimeters from the left edge of the line; similar for the post- test marks.
Simple descriptive statistics were used to describe the data; inferential statistics (paired t-tests) were used to determine significance of the difference scores. Effect sizes were calculated with Cohen’s d.