1. Sharing the Secret Recipe: Deconstructing Key Performance Indicators for Quality Distance Social Work Education
Social Work Distance Education Conference
Indianapolis, Indiana
April 15, 2015
Debora Rice, Ph.D., MSW., LCSW
Pablo Arriaza, Ph.D., MSW., LICSW
Sally Brocksen, Ph.D., MSW., MBA

2. Presentation Objectives
Critically evaluate performance measures used in traditional higher education
Identify factors for determining quality in online education around teaching, learning, and support
Present key performance indicators (KPIs) used in Walden University’s Social Work Program
Discuss how three experienced online social work professors interpret and use KPIs to identify quality and encourage performance

3. Reluctance to Embrace Online Education
Only 28% of higher education leadership say their faculty accept the “value and legitimacy of online education”
66% of faculty consider online courses as inferior or somewhat inferior to a comparable face-to-face course
79% of faculty say they have concerns about the quality of online education at for-profit institutions
Social work faculty specifically appear to have an embedded hesitation to embrace social work education in an online environment
References:
Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium.
Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.
To change the mindset, hard data about quality are needed.
The indicators of quality and performance have to adapt to the new environment.
Walden’s model provides the evidence of performance and quality.
Faculty and accrediting bodies might be surprised to discover there are two levels of additional metrics for quality online education beyond those of traditional bricks & mortar
Sources:
Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium.
Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.

4. Conceptualizing Quality Performance
Traditional
Early Distance Education
Walden’s Online
Learning outcome measures
Student evaluation of faculty
Faculty self-evaluation
N/A
Absent
Learning outcome measures
Student evaluation of faculty
Faculty self-evaluation
Evaluation of technology delivery system
Absent
Learning outcome measures
Student evaluation of faculty
Faculty self-evaluation
Evaluation of technology delivery system
Automated student accountability
Faculty Key Performance Indicators (KPIs)
Compare and contrast the information used to evaluate quality performance in the traditional face-to-face classroom, the early distance education courses, and Walden’s current online program.
Two additional metrics used in Walden’s online environment are:
Automated capture of student attendance, participation, documentation of no plagiarism, etc.
Faculty KPIs

5. Today’s online environment requires:
Evidence-based benchmarks specific to the online environment
Online social work education requires:
A baseline for accreditation and reaffirmation of hybrid and online programs

6. Questions for Today
What factors (beyond the traditional) help determine quality performance in online instruction?
How can Walden University's KPIs and Lead Faculty Model demonstrate successful evaluation and high quality of teaching in the online environment?

7. Factors That Help Determine the Quality of Online Instruction
Responsiveness & availability
Frequency & quality of presence
Facilitation of discussion
Prompt, quality feedback
Developing cooperation among students
Responding to diverse talents & styles
Management of administrative aspects
Setting high expectations
References:
Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education,15(2), March 2012,
Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.
This list assumes quality technology is available, accessible, etc.
Walden’s KPIs evaluate these factors
Sources:
Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education, 15(2), March 2012,
Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.

8. Operationalizing Walden’s Model
KPI data goes to the Program Director weekly
In cases where KPIs are not met, data goes to the individual professor and Lead Faculty for immediate action
Lead Faculty serve as:
Subject matter experts (SME)
First line problem solvers
Evaluators
Coaches/Cheerleaders

9. Walden University’s Key Performance Indicators (KPIs)

10. Walden’s Key Performance Indicators (KPIs)
Log into class 4 out of 7 days
Participate in discussion posts 4 out of 7 days
Respond to at least 2/3 of student posts to each discussion question
Timely grade submissions with specific, substantial feedback
Assignments and grades for discussion questions returned within 7 calendar days from the due date on the course calendar

11. Additional Walden KPIs
Personalize the classroom
Upload picture, CV, and brief narrative
Review explicit class expectations
Provide substantial feedback to promote/enhance students’ critical thinking
Respond to students’ questions on the Class Café and Contact the Instructor tabs within 48 hours
Use Announcements to update students, provide relevant external links/resources, and promote a sense of presence
Create and maintain a respectful and safe environment in the classroom

12. The Trajectory and Experiences of Three Core Faculty Professors
Interpreting, Embracing, and Applying Walden’s Model of Excellence in Distance Education
The Trajectory
and Experiences
of Three Core
Faculty Professors
_ _ _ _ ___ _______ __________

13. Debbie’s Story Pablo’s Story Sally’s Story
Three Stories
Debbie’s Story Pablo’s Story Sally’s Story
Three faculty with significant online teaching experience will discuss how they interpret, translate, and apply the KPIs in their roles as faculty and program director.

14. Questions? Additional indicators relevant to social work?

15. References
Allen, I. E. & Seaman, J. (2012). Changing course: Ten years of tracking online education in the United States. United States: The Sloan Consortium.
Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium.
Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.
Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education, 15(2), March 2012,
Council on Social Work Education (CSWE) (2012/2013). Annual report. Retrieved 24 October, 2014 from
Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.

16. References
Cummings, S.M., Chaffin, K.M., Cockerham, C. (2015). Comparative analysis of an online and a traditional MSW program: Educational outcomes, Journal of Social Work Education, 51:
Flynn, M. Maiden, P. R. , Smith W., Wiley, J. & Wood. G. (2013). Launching the virtual academic center: Issues and challenges in innovation. Journal of Teaching in Social Work, 33, 331–338.
Lackey, K. (2011). Faculty development: An analysis of current and effective training strategies for preparing faculty to teach online, Online Journal of Distance Learning Administration, 14(4). Retrieved from
Shelton, Kaye. (2011). A review of paradigms for evaluating the quality of online education programs. Online Journal of Distance Learning Administration, IV(I), Retrieved from
Thyer, B. A. and Polk, G. (1997). Distance learning in social work education: A preliminary evaluation, Journal of Social Work Education, 33(2),