CBLT Design 4 Level 1 learners, EEM and H&SatWork Metrics, standards and evaluation. Ms Gaya Wijayawardena Dr John Woollard February 2008

2 Who are Level 1 learners? The Level 1 learner is frequently a mature adult with a new or renewed desire to start learning or gain qualifications. The learners will probably have a failed education thus possessing few or no qualifications. The desire to learn and /or gain qualifications may arise directly from work and the requirements to remain employed or a personal desire for self-betterment. Level 1 learners can have poor learning skills and poor meta-learning awareness. Frequently, they will have a low self-esteem, particularly in relation to academic pursuit. The learning styles of Level 1 learners are as varied as any other group of learners.

3 Who are your Level 1 learners?

4 Literacy skills Reading - at this level, adults are expected to: Follow a short narrative on a familiar topic. Read and understand some words, signs and symbols. Use reading strategies. describes the requirements (expectations) of level 1 learners

5 Numeracy skills describes the requirements (expectations) of level 1 learners Visual literacy skills Computer literacy skills describes the requirements (expectations) of level 1 learners

6 Principles of EEM Supportive Collaborative Specific Succeeds better if you are a “double expert” with knowledge of usability combined with expertise in the kind of interface under evaluation.

7 Advantages/Disadvantages of EEM Cost effective. Quick. Can identify a majority of usability problems. Useful for identifying and eliminating major problems before conducting user research. tends to enhance the credibility of an instructional product. less accurate and thorough than usability testing regardless of the evaluators’ skill and experience, they remain expert evaluators who emulate users and not typical users.

8 EEM references CHRIS CHINIEN, P. D. & FRANCE BOUTIN, P. D. (2005) Framework for strengthening research in ICT-mediated learning. Information Technology Based Higher Education and Training. UNEVOC, UNESCO, Canada, IEEE. FLOW-INTERACTIVE (2007) Expert evaluation. Flow Interactive. KANTNER, L. & ROSENBAUM, S. (1997) Usability Studies of WWW Sites: Heuristic Evaluation vs. Laboratory Testing. SIGDOC. Salt Lake City, UT, Association for Computing Machinery. LEPISTÖ, A. & OVASKA, S. (2004) Usability evaluation involving participants with cognitive disabilities. ACM International Conference Proceeding Tampere, Finland ACM. ROSENBAUM, S. (1989) Usability Evaluations Versus Usability Testing: When and Why? Professional Communication, IEEE Transactions Volume 32, TEITAC (2007) Theme:Testability.

9 EEM of H&S in the Workplace considering the instructions, information, activities and assessments: is the navigation intuitive? are they visually well presented? are they aurally well presented? are they verbally well presented? is the application motivating/encouraging/supportive?

10 Underpinning principles Constructivism Social constructivism Cognitivism Behaviourism Constructionism?

11 A cognitive metric? Bloom’s Taxonomy applied to assessment of work, evaluation of session opportunities, evaluation of a web resource, evaluation of a software application Evaluation The ability to judge the value of material using explicit and coherent criteria. Synthesis The ability to reassemble these parts into a new and meaningful relationship. Analysis The ability to break material down into its constituent parts and see the relationship between them. Application The ability to apply knowledge and comprehension in new and concrete situations. Comprehension An understanding of the meaning of this knowledge. Knowledge Simple knowledge of facts, terms etc.

12 Teaching through feedback

13 Evaluating good e-teaching secure knowledge and understanding of the subject effective planning setting clear objectives planning for differentiation meeting individual needs motivating learners having high expectations asking relevant and challenging questions supporting the development of critical thinking supporting the development of imagination supporting the development of creativity focussing on attainment progress effective management of learner etc.

14 Evaluating good e-learning (response) extent to which learners show interest · extent to which learners show intellectual curiosity · do the learners enjoy learning · are learners intent on learning · are learners proud of their work · is there sustained concentration · do they work collaboratively · ability to work independently · ability to work co-operatively · having a good relationship with teachers/trainers · learners taking responsibility · learners showing initiative · learners showing a self awareness, courtesy, trustworthiness · learners showing a respect for others views and beliefs · confidence to reflect upon their own attainment and values · respect for rules and convention

15 Evaluating progress If the teaching is good and the response of the learners is appropriate then progress should be made. How do we measure progress?

16 Describing a learning object What is the difference between aims and objectives? Define learning outcomes, learning objectives, learning activities. What is the difference between assessment and attainment? What is the difference between continuity and progression? What is chronology? Define and exemplify session starts, mini-plenary and plenary. What is the difference between long-term, medium-term and short-term planning?

17 Questions?