1. 1 The Real Costs of e-Learning What are they? How can we find out? Professor Paul Bacsich UK eUniversities Worldwide Limited Presentation to Venezualan Delegation,16 March 2004

2. 2 Contents  Key issues in a borderless world  UK Transparency Review  Activity Based Costing [CNL2]  Activities for Networked Learning [CNL1]  Application to e-learning

3. 3 Key issues - and a challenge  There is now increasing agreement on the methodologies of costing e-learning  So why is there a dilemma where education see “No Significant Difference” whereas training sees “Return on Investment”?  The challenge is to find a uniform evaluation methodology, including costs, which copes with a world without borders…  Borders are not only geographic

4. 4 UK Transparency Review  HEFCE Initiative - and other Funding Councils  Focus on costs of research and teaching  Compulsory  Reports on 5 main activities: T 2 R 2 O  Uses “Activity Based Costing” terminology  Uses Activity Time Sheets  Phased implementation, mainly led by research-intensive universities

5. 5 Transparency - conclusions  It has been a useful if painful exercise for universities and funding agencies  Pressure is increasing on research funding agencies to raise the overheads they allow as real costs become clear BUT, in reference to the “eternal dilemma”  It does not seem to have led to operational conclusions  Because it is not detailed enough?

6. 6 Scope of CNL2 project  Sarah Heginbotham (SHU then UKeU then Canada)  Funded by JISC (Joint Info Systems Cttee)  From CNL1: Recommendation that activity- based costing be trialled in a university  Chose the School of Computing and Management Sciences at Sheffield Hallam University, UK  6 months project, started summer 2000  Transparency Review was going on also

7. 7 Cooper and Kaplan - ABC  Developed in 1988 as an alternative costing methodology at Harvard Business School  They argued that traditional costing distorted product costs  “peanut butter spread” approach  There is a cost to all activities carried out within an organisation  e.g. transport of peanuts  Activity costs should be distributed to products in relation to their use of resources

8. 8 Advantages of ABC  Gives more than just financial information  “Fairer” system of overhead allocation  Accountability of central services e.g. IT  Highlights cross-subsidisation  Recognises the changing cost behaviour of different activities as they grow and mature  Can provide data for other initiatives  e.g. Transparency Review and Quality

9. 9 Disadvantages of ABC  Data collection can be costly and time consuming  Initially it can be challenging to collect the data you want  Need to determine appropriate and acceptable cost drivers  ABC is a more complex system than traditional management accounting  and relies on that being done well !

10. 10 CNL2 Conclusions  ABC does work in HE and will be increasingly used in institutions, especially “ABC-lite/ABM”  It works on different levels within the institution  Suitable software and professional support is essential, in the early days at least  Academic scepticism can be overcome  Costings data needs to be placed in context

11. 11 Networked Learning - CNL1  Paul Bacsich  and Charlotte Ash (now in public sector “Best Value” programme)  Sheffield Hallam University, UK  6 month study (in 1999-2000)  Funded by JISC  Aim - to produce a planning document and financial schema that together accurately record the Hidden Costs of Networked Learning

12. 12 Examples of Hidden Costs  Increased telephone call and printing bills for students due to Internet usage  Entertainment expenses “necessarily” incurred by academic staff at conferences but not reimbursed by the Institution  Administrator time answering student queries  Support costs of a new Learning Environment  Costs of content - “created in one’s spare time”  Costs of institutional collaboration  Costs of conformance to standards

13. 13 Mega-activities for teaching: Lifecycle model  Cost items recorded from literature  All current educational and training costing models analysed (Rumble, Bates, etc)  Traditional financial accounting analysed  Consultation with academic staff  Various cyclic models proposed, leading to...

14. 14 Course Lifecycle Model Planning and Development Production and Delivery Maintenance and Evaluation Three-phase model of course development

15. 15 Breakdown of three- phase model (activities)

16. 16 Financial Schema - “Stakeholders”

17. 17 Conclusions from CNL1 In order to accurately record the Costs of Networked Learning you must have a sectorally-accepted method of what costs to record and how, in place from the start, which takes into account all stakeholders and uses Activity Based Costing within a framework of phases of the course development life cycle

18. 18 What are the True Costs?  Q: What is the rate per study hour?  Different kinds of study hours:  Active engagement with specifically created “teaching” material (expensive)  Thinking, conversation  Doing assessment  Studying “resource” e-content (cheaper)  Surfing the open web (cheapest, but worst?

19. 19 Rough guide - rule of thirds  1: Study of “engaging” multimedia  2: Study of existing or slightly modified learning resources  3: Working on assignments (maybe in collaboration)  Costs (US, UK and Germany)  average $12000 per m/m hour, but v. large   10% of this for simple material  could one aim for $1000 per hour?

20. 20 Content development 1  Determine the level of demand  Produce an outline plan of the courseware  Conceptualise the subject knowledge  Structure and design the learning experience to be provided by the courseware materials  Decide interactivity with the learner, the level and ways of building tutorial support into the material itself and the links with other materials.

21. 21 Content development 2  Decide the formative assessment exercises and the ways of providing feedback to the learner (and to the tutor if appropriate)  Decide the summative assessment associated with each module  Establish a range for the possible levels of on-line tutor support for each module.  Consider the interface between the learning programme and its sub-parts and the administrative support software tools

22. 22 Consortia - Critical Success Factors  High “binding energy”  may come from top-down or funding-driven  but more likely from friendship or shared vision  does not depend much on legal aspects  Organisational homogeneity or managed diversity  Linguistic and cultural similarity  Stratification  “Safely far away” [Source: TL-NCE, UNESCO report]

23. 23 Other topical questions  Will research advances reduce costs?  Will Open Source reduce costs?

24. 24 Research  This may be too much of a personal view  I have been conference organiser, evaluator, reviewer,...  Look at impact from EU research work  Look at impact of work elsewhere  UK  TL-NCE (Canada)  US  Australia, New Zealand, Singapore Hong Kong….

25. 25 Research conclusions  European research: FP3 set the scene; FP4 added little, FP5 more promising  Canadian work credible and integrated, but lacks evidence of scalable approaches  Too much gap between computing theorists and industrial-strength pedagogic practice  few theorists in universities are seriously active in e-learning services  US is very synchronous and transmissive, paradigms that do not fit a time-poor world

26. 26 What research might reduce costs?  Focus on co-operative learning  Start with basic asynchronous “BBS” model  Allow new models to be supported, especially those with business potential  Develop scalable approaches  more focus on automated assessment?  Support multiple media and devices using same content  Develop more effective learning esp time-effective

27. 27 Can e-learning save time?  travel time  reduce “time on task”  by “better” training (25-40%)  by individualised training...  by using “time of the 3rd kind”  Time 1 : on duty - used to be called “at work”  Time 2 : off duty (not at work)  by consolidating time fragments

28. 28 Open source issues  Exemplars:  Linux, MIT, Canadian, Finnish, IMS, UK interest  Purpose:  Challenge commercial vendors  Facilitate research by providing flexible system  Cheaper?  Yes, to buy  No, to support (scalability, robustness?)  Few successful large e-learning players use “pure” open source, but several use modified

29. 29 Features of a uniform cost-aware evaluation methodology  Costing basis must be clear  To gain political buy-in and breadth (multi- sector, etc), depth will be small  Students treated as people not only learners  Focus on level of knowledge and skill gained  Focus on time issues, not only time on task  time 1, time 2, time 3 - and differential value?  More feasible when there is a more uniform curriculum or exams (as in schools)

30. 30 Details about CNL costings projects - At http://www.shu.ac.uk/cnl/ thanks to support from SHU and JISChttp://www.shu.ac.uk/cnl/ Professor Paul Bacsich pbacsich@ukeu.com