1. OBM Research Using the PSI Classroom as an Analogue Work Environment
Amanda Matthews
Elizabeth Hagen
Charles R. Crowell
Department of Psychology
University of Notre Dame
Paul D. Worland
Press Ganey Associates, Inc.
, ext. 114

2. OBM Research Venues
Field research in organizational settings—traditional yet challenging
Analogue research in more controlled lab-based environments—perhaps limited

3. Analogue Work Environments
Incentives/feedback (Dickinson & colleagues; e.g., Bucklin, McGee, & Dickinson, 2003)
Safety practices/observer effects (Austin & colleagues; e.g., Alvero & Austin, 2004)
Management/leadership (Mahwhinney & colleagues; e.g., Mawhinney, 2005)

4. Worrisome Aspects of Workplace Analogues
Nature of task Complex and engaging or simple?
Duration of observation/interventions Ongoing or short-term?
Available measures Multiple or singular dimensions?
Interactions with management Realistic or artificial?

5. Of Concern to Performance Managers
Some Traditional
Problems
Of Concern to Performance Managers
Employees who
Don’t really know what is expected of them
Don’t really know how they are doing each workday
Get performance evaluations once (maybe twice) a year
Are left alone unless they screw up
Don’t really know what is important in the material
Don’t really know how they are doing each class
Have only a few large tests a semester
Must wait until the end of the class for a final grade
Students who
Xxxxxxxxxxxxxx

6. The PSI Classroom What is PSI? (Keller, 1968)
Emphasis on written material
Self-paced progress
Small units with frequent testing
Unit mastery
Use of student proctors
How might it resemble a workplace?
Workers (students) must complete units
Quality and timeliness criteria may apply
Proctors can serve in the role of managers

7. Parallels between OBM and PSI
Ways to Fix the
Problems
Parallels between OBM and PSI
Don’t really know what is important in the material
Don’t really know how they are doing each class
Have only a few large tests a semester
Must wait until the end of the class for a final grade
Students who
Employees who
Don’t really know what is expected of them
Don’t really know how they are doing each workday
Get performance evaluations once (maybe twice) a year
Are left alone unless they screw up
Learning objectives
Many small units/point system
Frequent tests with retake options
Proctor interactions for recognition and remediation
PSI Solution
Task clarification
Feedback
Coaching
Reward and recognition
OBM Solution
Xxxxxxxx

8. Proctors Have a Critical Role in PSI As Performance Managers
Role of proctors was thought to be critical in Keller’s original view of PSI
Proctors were thought to supply a “personal-social” enhancement of the learning process
Proctors were viewed contingency managers to shape and reinforce student behaviors
In original plan, proctors had key administrative roles with course management
In other variants of PSI, role of proctors may differ 11

9. Proctors Have Altered Role
In Our Version of PSI
We created a computer system to assume routine administrative functions of proctors
Upsides were greater uniformity and a smaller number of required proctors
Downsides were reduced contact and lowered course satisfaction—performance was unaffected
Prompted us to institute a message system
Then set about to systematically explore effects of message content 11

10. A Study of Proctor Message Content On Performance and Satisfaction
We systematically varied praise, encouragement, and group information in 2 X 2 X 2 factorial
Group average info was added (or not) to individual performance feedback present in all messages
Praise (or not) was a statement of recognition if performance measures were improved or maintained at or above a predefined goal
Encouragement (or not) was a statement of the need for improvement if performance was below the goal or had not improved 11

11. Correlation Matrix N=72 Quality Timeliness Satisfaction
Final Exam
Average Unit Score
Initial Unit Score
Retake Unit Score
Bonus Points
Days to Complete Units
Perceived Amount Learned in Course
Proctor Message Satisfaction
Satisfaction with Course Organization
Course Satisfaction Q U A L I T Y
1.00
.62*
.48*
.52*
.31*
-.04
.28*
-.06
.21
.24*
.82*
-.07
.44*
-.08
.12
.33*
.60*
-.18
.34*
.01
.23
.45*
.35*
-.09
.02
.17 M E N S
.08
.14
.20
Days to Complete
.18
.05 F C O
Amount Learned
-.05
.25*
.51*
Proctor Message
-.15
.15
Course Organization
Complicated slide with lots of information.
Want to make just a few points:
1. Consider the various measures: three of the four traditional categories of work measures are represented. Quality, timeliness and satisfaction.
Quantity is not really an issue in our version of PSI as it is held constant.
Look at the inter-correlations within categories.
Look at the correlations across categories. 11

12. Proctor Message Content Study
General Findings
No impact of message content factors on quality measures
Praise enhanced the pace of unit completion
Praise and encouragement interacted in their effects on bonus earned and course satisfaction
Group information slowed down the pace of unit completion but enhanced satisfaction with messages. 11

13. Proctor Message Content Study Days to Complete Blocks

14. Proctor Message Content Study Average Bonus Points Earned

15. Proctor Message Content Study
Course Satisfaction

16. Proctor Message Content Study Days to Complete Blocks

17. Proctor Message Content Study Satisfaction with Messages

18. Preliminary Conclusions
PSI classrooms can be viewed as work-like environments amenable to OBM style interventions over semester-long periods
Multiple worker (i.e., student) response measures are available including performance and satisfaction indicators
Proctors are analogous to managers and their behaviors can influence worker (i.e., student) performance and satisfaction. 11