1. Open Education Network http :// www. open - ed. net National Capacity Building of ODL Professionals on OER Quality -------------------------------------------------------------------------- open educational resources C E M C AB R A O U workshop organised by the Commonwealth Educational Media Centre for Asia in collaboration with Dr B R Ambedkar Open University

2. Open Education Network http :// www. open - ed. net The TIPS Quality Assurance Framework for creating OER : Validation -------------------------------- ------ ------------------------------------ open educational resources Commonwealth Educational Media Centre for Asia

3. Open Education Network http :// www. open - ed. net The TIPS Quality Assurance Framework for creating OER : Validation -------------------------------- ------ ------------------------------------ Paul Kawachi FRSA kawachi @ open - ed. net http : // www. open - ed. net / oer-quality / validation. ppt

4. Open Education Network http :// www. open - ed. net overview TIPS Framework http://cemca.org.in/ckfinder/userfiles/files/ OERQ_TIPS_978-81-88770-07-6.pdf Version 1. 0

5. Open Education Network http :// www. open - ed. net overview TIPS Framework four layers of ideas TIPSTIPS

6. Open Education Network http :// www. open - ed. net overview TIPS Framework

7. Open Education Network http :// www. open - ed. net overview TIPS Framework from among 205 QA criteria the most comprehensive compilation to date 65 items indicated by 60+ global experts http://www.open-ed.net/oer-quality/criteria.pdf

8. Open Education Network http :// www. open - ed. net overview for teachers in class

9. Open Education Network http :// www. open - ed. net overview TIPS Framework 65 items are here subjected to delphi-style referral for content validation

10. Open Education Network http :// www. open - ed. net definitions : an open educational resource (OER) is defined as a digital self-contained unit of self-assessable teaching with an explicit measurable learning objective, having an open-licence clearly attached to allow adapting, and generally being free-of-cost to reuse UNESCO - CoL 2011 ; Kawachi 2013 introduction

11. Open Education Network http :// www. open - ed. net wave analysis to confirm coverage - Leslie 1972 content validity ratio analysis - Lawshe 1975 multiple surveys and online discussions methods

12. Open Education Network http :// www. open - ed. net wave analysis three waves of responses from individual OER experts

13. Open Education Network http :// www. open - ed. net the three waves each constructed using the survey responses from ten individual OER experts are visually similar increasing confidence that a sufficient number of data are collected, and CVR analysis can proceed wave analysis

14. Open Education Network http :// www. open - ed. net wave analysis similar to fourth wave in gold from anon OER groups

15. Open Education Network http :// www. open - ed. net wave analysis fairly similar to waves 5 & 6 from individual teachers

16. Open Education Network http :// www. open - ed. net age age distribution for each set of responses

17. Open Education Network http :// www. open - ed. net age distribution : there was no clear normal distribution there are tendencies for the - Set-1 OER experts to be older - Set-2 OER group participants to be younger and - Set-3 teachers also to be relatively young age

18. Open Education Network http :// www. open - ed. net geographic distribution of all responses geographic CC BY SA blank world map image by anon from WikiMedia http://commons.wikimedia.org/wiki/File:World_map_blank_black_lines_4500px_monochrome.png

19. Open Education Network http :// www. open - ed. net survey population characteristics : gender Set-1 : individual OER experts Set-2 : group OER experts Set-3 : individual teachers N 35 14 21 male 21 7 10 38 female 13 7 11 31

20. Open Education Network http :// www. open - ed. net content validity ratio : survey asks content experts to rank each criterion as validation Essential Not-essential but useful Not necessary

21. Open Education Network http :// www. open - ed. net content validity ratio : min av CVR for a criterion to be retained at p < 0.05 validation

22. Open Education Network http :// www. open - ed. net content validity ratio : av CVR E for all items = CV Index for instrument ~ 0.80 for validity at p < 0.05 validation Set-1 Set-2 Set-3 retaining 24 criteria gives overall CVI = 0.80 0.79 0.79

23. Open Education Network http :// www. open - ed. net content validity ratio : validation Set-1 Set-2 Set-3 ( 9 th May 2014 ) N=32 N=13 N=19 retaining 21 criteria : overall CVI = 0.80 0.79 0.79 ( 5 th June 2014 ) N=35 N=14 N=21 retaining 38 criteria : overall CVI = 0.84 0.81 0.79 ( 27 th May 2014 ) N=35 N=13 N=19 retaining 24 criteria : overall CVI = 0.84 0.79 0.79

24. Open Education Network http :// www. open - ed. net content validity ratio : 5 th June 2014 these 38 criteria retained are all those described in the shorter version TIPS Framework validated at CVI > 0.80 at p < 0.05 validation http://www.open-ed.net/oer-quality/tips.pdf

25. Open Education Network http :// www. open - ed. net content validity ratio : moreover the original 65 criteria of version 1.0 TIPS Framework has CVR E+U = 0.94 by the 35 OER experts indicating original was valid validation

26. Open Education Network http :// www. open - ed. net suggestions : six criteria indicated at highest CVR E by OER experts validation Don't use difficult or complex language, and do check the readability to ensure it is appropriate to age / level Make sure that the knowledge and skills you want the student to learn are up-to-date, accurate and reliable. Consider asking a subject-matter expert for advice Be sure the open licence is clearly visible Ensure your OER is easy to access and engage Present your material in a clear, concise, and coherent way, taking care with sound quality Use open formats for delivery of OER to enable maximum reuse and re-mix.

27. Open Education Network http :// www. open - ed. net Quality Assurance Guidelines for Open Educational Resources version 1.0 TIPS Framework created from 205 criteria collated from global OER experts results version 1.0 65 criteria

28. Open Education Network http :// www. open - ed. net Quality Assurance Guidelines for Open Educational Resources version 2.0 TIPS Framework revised criteria with some highlighted as more important and rubric added for Quality Improvement results version 2.0 38 criteria

29. Open Education Network http :// www. open - ed. net results

30. Open Education Network http :// www. open - ed. net literature references : CoL (2011). UNESCO-CoL Guidelines for Open Educational Resources (OER) in Higher Education : OER Workshop, 25 May, Dar-es-Salaam. Retrieved April 25, 2014, from http://oerworkshop.weebly.com/uploads/4/1/3/4/4134458/2011.04.22.oer_guidelines_for_higher_education.v2.pdf Kawachi, P. (2013a). Quality assurance guidelines for open educational resources : TIPS framework, version 1.0. New Delhi, India : Commonwealth Educational Media Centre for Asia (CEMCA). Retrieved March 7, 2014, from http://cemca.org.in/ckfinder/userfiles/files/OERQ_TIPS_978-81-88770-07-6.pdf Kawachi, P. (2013b). Quality not static. OER Quality, blog posting 30th January. Retrieved March 7, 2014, from http://oerquality.wordpress.com/2013/01/30/quality-not-static/ Kawachi, P. (2014). Quality Assurance Guidelines for Open Educational Resources : TIPS Framework, version 2.0. New Delhi, India : Commonwealth Educational Media Centre for Asia (CEMCA). Retrieved March 16, 2015, from http://cemca.org.in/ckfinder/userfiles/files/TIPS%20Framework_Version%202_0_Low.pdf Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel Psychology, 28 (4), 563-575. Retrieved February 21, 2014, from http://www.bwgriffin.com/gsu/courses/edur9131/content/Lawshe_content_valdity.pdf Leslie, L.L. (1972). Are high response rates essential to valid surveys ? Social Science Research, 1, 323-334. references

31. Open Education Network http :// www. open - ed. net acknowledgements : the author thanks Sanjaya Mishra and the Commonwealth Educational Media Centre for Asia (CEMCA), New Delhi, and the Commonwealth of Learning (CoL), Vancouver, for their support the author also thanks Ramesh Sharma for technical assistance in reaching the survey population thank you

32. Open Education Network http :// www. open - ed. net You can download these slides freely from the website http : // www. open - ed. net / oer-quality / validation. ppt ______________________________________________ or by email to me at kawachi @ open - ed. net