1. Helen Walmsley-Smith Staffordshire University
Comparing Distance and Blended Learning Designs with retention and attainment data to improve practice
Helen Walmsley-Smith
Staffordshire University

2. Online Distance Learning Retention
Open University research suggests graduation rates may be as low as 22% (Simpson, 2013)

3. What impacts on retention?
What is said to impact on retention?
External factors
Academic and social integration (Tinto 1987) adapted for online by Rovai (2003)
Transactional Distance (Moore 1993)
Course design, level and subject
Internal factors
Motivation (Bekele 2010), e.g. ARCS theory (Keller 2008)
Previous qualification, IT skills etc , (Harrell II and Bower 2011)
Satisfaction, learning community, support and communication etc, (Hart 2012)
What Methodologies have been used?
Student surveys
Case studies
Data analysis
Terminology

4. Impact of Learning Design
Interaction improves satisfaction and retention (Croxton 2014)
Feedback improves learning (Hattie 2012)
No easy way to represent amount/type of interaction or feedback in learning design
Lesson plan?
Little agreement on terminology
Terminology

5. Learning design instrument
Type of Interaction
Student-Tutor
Student-Student
Student-Content
Type of Feedback
Tutor Feedback
Peer
Self
Automatic

6. Online distance learning courses
Four convenience courses used for pilot:
LLM Business Law – International Trade and Commerce (level 7)
Physical Activity and Mental Health (level 7)
Money Advice Practice (DEMSA) (level 4)
The Management of Database systems (level 7)

7. Sample Learning Activities
Sample activities:
Read through the "lecture" slides and make notes
Tutorial – exercise e.g., given this information, answer the following: The table shows five transactions T1-T5 which act on database variables A-E. For each operation show the log entry and the values of the variables on disk.
[In answer for your portfolio] discuss how each transaction recovers from failure. Refer to the model answers
Discuss in the forum the purpose of "intercoms" and how a FAS contract works.
How does a FAS contract differ from a FOB contract?

8. Content Analysis Raters were trained on how to use the instrument
A sample activity completed and reviewed
Raters completed categorisation of 4 courses
Data analysed with Cohen’s Kappa to see level of rater agreement

9. Inter-rater reliability
During training and discussions it became clear that what were assumed to be ‘clear’ concepts were difficult to apply to the actual learning design.
There was little inter-rater agreement
The terminology was changed several times in response to these discussions

10. Learning Design Instrument
Activity:
In a pair/small group, review the sample activities and try to identify any:
Interaction – what type?
Feedback – what type?
Other
Comment on where you disagree!

11. Discussion Online learning designs are ‘messy’
Representing learning designs is challenging
Learning design instrument shows potential to describe and represent learning designs quantitatively
Learning design instrument allows for comparison to retention and attainment data as well as to other designs
Learning design does not necessarily show what students /tutors actually do
Final learning design instrument to be used to review a number of courses and the inter-rater reliability evaluated