Iain Weir, Rhys Gwynllyw & Karen Henderson CETL-MSOR 2014 An innovative use of technology to aid the service teaching and assessment of statistics to a large cohort Iain Weir, Rhys Gwynllyw & Karen Henderson CETL-MSOR 2014

Motivation Level 2 research skills module delivered to 650+ Business School Students Short course on statistics covering a challenging amount of material together with learning to use SPSS Designed to provide a solid foundation for students to undertake Level 3 project work Electronic assessment of bespoke data

Weekly schedule Exploratory data analysis (EDA), one sample T-test & nonparametric equivalent Two sample T-tests & nonparametric equivalents One-way ANOVA & nonparametric equivalent Correlation & simple linear regression Multiple regression Principal Component Analysis (PCA) & Cluster Analysis Two online tests: one-way ANOVA & Multiple regression

Basic statistical analysis tasks Transfer from Excel files to required SPSS format Exploratory data analysis for summary statistics, graphics and assumption testing Identification of appropriate statistical test (parametric or nonparametric equivalent) Interpretation and reporting of test output Typically assessed using a written report

Lectures One hour per week Notes covered statistics and use of SPSS No reference in lectures to use of SPSS to gain output in examples presented Students can concentrate on the statistical concepts and SPSS output interpretation

Computer lab preparation Expectation between of engagement between lecture and following computer lab Tasks requiring students to recreate the SPSS analyses in the notes Transfer of data into required SPSS format covered Self-learning aided by a suite of SPSS “how-to” videos Students can concentrate on the mechanics of the creation of output and familiarisation of concepts

Computer labs One hour per week Students given questions that covered various data outcome scenarios (para/non-para, sig/not sig, post-hocs) First question was designated for the student’s Learning Journal; a pre-written complete analysis template with parts removed for the student to complete Students requiring SPSS “button pushing” help were directed to notes and videos Staff concentrate in labs on giving statistical understanding and interpretation advice

Learning Journal extract

DEWIS test 1: One-way ANOVA A parametric test to evaluate the equality of 3 or more population means using sample data If the test assumptions not met then a non-parametric test to evaluate the equality of the population distributions is carried out If there is evidence of a difference then Post-Hoc tests are employed to detect where the differences lie

Stage 1: Importing the data from Excel to SPSS Multiple submissions allowed Stage 2: EDA and identifying the appropriate test One submission allowed Stage 3: One-way ANOVA or Kruskal-Wallis test One submission allowed NB Students can quit after submitting a stage and marks already accrued will be saved

e-Assessment mechanics e-Assessment system Student DEWIS

e-Assessment system: DEWIS Can communicate with the R statistical package which was employed to generate bespoke student data and generate answers that would match SPSS screen output Run staged assessments Implement continuation marking for a large number of inputs Provide dynamic feedback specific to student inputs

Students raw data Students taught to pay attention to precision of recorded data

Correct SPSS format DEWIS marks using expected precision

Incorrect SPSS format

Stage 1 Download data Unlimited attempts Checks: transfer precision

Submission

Feedback Answers not given Correct Incorrect Close Extra information: PDF Video

Extra Information: PDF Link from within DEWIS to personal web page host 9 pages later:

Extra information: video Captive slide shows with voice overs Link from within DEWIS to personal web page host http://www.cems.uwe.ac.uk/~isweir/MoE/DEWIS1/ReadingDEWIS1Data.htm

Resubmission When correct the student is allowed to proceed to Stage 2

Stage 2 13 questions 67 activities only one submission

Feedback: The Question

Feedback: The Solution

Feedback: The Report

Stage 3 Only correct test choice allowed 6 questions 26 activities only one submission Need to design balanced questions for all competing scenarios

Marking scheme Stage 1 is worth 7 marks for successful data transfer otherwise 0  Stages 2 and 3 Numerical value: correct=1 incorrect=0 Multiple choice: correct=1 incorrect=-1 not answered=0 In total 100 activities each worth a mark Negative marks for Stages 2 or 3 are truncated to zero

Outcomes Results have been excellent Teaching and assessing in this way has made the challenging task of delivering and assessing this material in a short space of time achievable The fact that students can refer to their Learning Journals and access “how-to” SPSS videos will be beneficial to their further studies

Colleague quote “…..we are getting them to carry out quite sophisticated analyses, with a lot more success than I believed possible at the beginning!”

Next? Refine work in light of first delivery experience Extend to other modules Examples of DEWIS with R in public domain Funding