Presentation on theme: "University of Central Lancashire in Preston, April 24th, 2014."— Presentation transcript:
1 University of Central Lancashire in Preston, April 24th, 2014. Developing and implementing collaborative evaluation approaches with MOOCs and SLA Liam Murray, Elaine Riordan, Amanda Whyler, Laura Swilley, Aylar Vazirabad, Mansour Alammar, Norah Banafi, Vera Carvalho, Yanuar DwiPrastyo, Geraldine Exton, Ingrid Deloughery, Lisa Cadogan, Sinead Nash, Qing Miao, Mazin Allyhani and Osama Al-Moughera University of Limerick, IrelandPresentation at“Massive Open Online Courses in the Arts and Humanities: Opportunities, Challenges and Implications Across UK Higher Education”, atUniversity of Central Lancashire in Preston, April 24th, 2014.
2 Overview The Evaluation Team and types of MOOCs followed; Evaluation MethodsCommon and discrete features of the MOOC experiences;Pedagogic Implications of MOOCs;Analysis and Assessment Issues;Results and recommendations on potential for repurposing and designing for SLA;Adding to the current debate.
3 The Evaluation Team and types of MOOCs followed Team 8 PhD students 6 MA students 2 lecturers Evaluation: was part of their own module on ICT and SLA, but participation in MOOC activity was purely voluntary and not assessed.Types of MOOCsGames programming (X2)Academic writing (X3)AnatomyGamification (X2) (Coursera)Corpus Linguistics (X5)Web science: how the web is changing the world (X2)How the brain functions.
4 Our Evaluation Methods Set tasks: possible to repurpose an existing MOOC for SLA? Possible to design a dedicated MOOC for SLA based on their MOOC experiences?Iterative and summativeQualitative and experiential rather than quantitativeCourses ranged from 3-10 weeks.
5 Common and discrete features of the MOOC experiences Common featuresStandard and personalised s at the beginning and end of each weekAn introduction and summary each weekShort lecture-type video recordings (with PPT and transcripts)Resources, links and references on weekly topicsDiscussion forums (moderated but not mandatory)Weekly quizzes (MCQs)Discrete featuresUse of and access to software specific to areas (e.g. Corpus Linguistics – length of access unknown)Anonymous peer feedback on assignments (e.g. Writing)Revision and feedback on writing (e.g. Writing)Essay writing and analytical activities.
6 Pedagogic Implications of MOOCs Autonomous learning (playing ‘catch-up’)Control theory: two control systems—inner controls and outer controls—work against our tendencies to deviate. Social bonds & personal investments.Motivation theory (both intrinsic and extrinsic motivation)ConnectivismLow affective filterAnonymous (pros and cons)LurkingInformation seekers Curiosity Learning.
7 Analysis and Assessment Issues Peer evaluation: students not happy with this (cultural issues? Anxious about the unknown).However: “Coursera’s foray into crowd-sourced peer-assessment seems to be a useful response to the problem of the M in Massive”.Note: Coursera’s students would be docked 20% of our marks if refuse to self-evaluate and locked out until we completed evaluation of 5 others students’ work.Self evaluation: perceived as worthwhileMCQs: could be completed without reading the materialAssessment :Was complexWas dependent on aims: professional development or learn about a topicNeeds to be (1) credible and (2) scalable.
8 Assessing writing in MOOCS (Balfour, 2013) Automated Essay Scoring (AES) Vs Calibrated Peer Review (CPR)AES: good on grammar, syntax, speeling and “irrelevant segments”.AES: bad on complex novel metaphors, humour, slang and offered “vague feedback”.CPR: given a MC rubric; peer review & self-evaluation; receive all feedback from peersCPR: Some peer-resistance but improves particular writing skills; limitations in a MOOC environment.
9 Balfour’s Conclusions For both AES and CPR, the more unique or creative a piece of work is, then less likelihood of producing a good evaluation for the student.Same for original research pieces.Final conclusion: not every UG course that uses writing as a form of assessment will translate to the MOOC format.
10 Results and recommendations on potential for repurposing and designing for SLA ConsTechnical issuesNot a lot of variety in the deliveryDiscussion forums too largeNo chat functionPersonalised support needed depending on the field/areaNot ab initioProsNo feesClear Guidelines givenAbility to work at own paceWell-known academics delivering materialSome courses followed PPP model (implications for SLA)Suitable as tasters
11 Results and recommendations on potential for repurposing and designing for SLA Have more variety in interaction patterns (synchronous and asynchronous)Have more variety in modes of delivery and presenters/lecturersOrganise smaller discussion groupsOffer clear guidelines and instructionsStage input and activitiesOffer credits
12 Adding to the current debate MOOCs to SPOOCs for SLAMaintaining motivationFinancing & monetising the CoursesWeighting (accreditation).