Presentation is loading. Please wait.

Presentation is loading. Please wait.

2nd UK Workshop on Constructive Alignment

Similar presentations


Presentation on theme: "2nd UK Workshop on Constructive Alignment"— Presentation transcript:

1 2nd UK Workshop on Constructive Alignment
28/03/2017 Measuring Constructive Alignment: An alignment metric to guide good practice Dr Jon Tepper Learning and Teaching Coordinator School of Science and Technology Dr Jon Tepper, School of Science and Technology

2 Activity to get you in the mood…
2nd UK Workshop on Constructive Alignment 28/03/2017 Activity to get you in the mood… In groups of two or three: For each learning outcome, consider how well aligned the associated learning objectives, teaching and learning activities and the assessment tasks are; Identify precisely why the students and External Examiner were unhappy; Discuss and note down how you think the design could be improved. School of Science and Technology Dr Jon Tepper, School of Science and Technology

3 The Era of Learning Outcomes
2nd UK Workshop on Constructive Alignment 28/03/2017 The Era of Learning Outcomes During period of 1989 – 1994: More unis = more products = more competition = more student choice Therefore a need for standards to allow students to make comparisons and informed choice Criteria and process must be transparent and fair HEQC performed a review of HE and in 1997 they recommended the following to Dearing: ‘formulation of general educational expectations which guide the creation of programme educational…learning outcomes and more specific learning outcomes in the taught elements of the programme’ (HEQC, 1997) Before discussing the big bang in 1997, lets just be clear on what learning outcomes are….. School of Science and Technology Dr Jon Tepper, School of Science and Technology

4 2nd UK Workshop on Constructive Alignment
28/03/2017 UK Big Bang Subject Benchmarking QAA Catalyst National Qualifications Framework Subject Review The Dearing Report (NCIHE, 1997) accepts HEQC’s (1997) analysis Dearing recommends that in Programme Specifications, learning intentions are to be expressed as learning outcomes ILT/HEA So… Dearing accepted HEQC’s recommendations Provided rationale for an outcome-based perspective QAA+ILT NationalQualframework+AcademicReview+SubjectBenchmarking are all policy vehicles for promoting system-wide adoption of outcome-based learning Its also used as a common framework for use of credit accumulation and transfer in HE For Subject Benchmarking see: National Qualifications Framework see: TQi see: HE Academy see: APL see accreditation of prior learning: intended learning outcomes of previous study may be used as key reference points against which decisions about the accreditation of prior learning can be made. TQI Outcome-based Learning APL School of Science and Technology Dr Jon Tepper, School of Science and Technology

5 International Big Bang
2nd UK Workshop on Constructive Alignment 28/03/2017 International Big Bang Declaration signed in June 1999 Calls for the harmonisation of HE qualification systems in Europe Creation of the European Higher Education Area by 2010 Learning outcomes to play a key role The Bologna Process derives its name from the so-called Bologna Declaration, which was signed on 19 June 1999 by ministers in charge of higher education from 29 European countries. It is an intergovernmental European reform process aimed at establishing the European Higher Education Area (EHEA) by 2010. Learning outcomes will support the: i) Adoption of easily readable and comparable degrees ii) Adoption of a system essentially based on two (now three) main cycles iii) Establishment of a system of credits iv) Promotion of mobility v) Promotion of cooperation in quality assurance With respect to iii)….will establish a system of credits, such that learning outcomes will extend European Credit Transfer System (ECTS) and map as follows onto awards: 60 – 120 ECTS Certificate and Diploma, respectively. 180 ECTS Bachelor degrees of various types (general degree) or 240 ECTS Advanced Bachelor degrees (honours degree) 60 ECTS postgraduate certificate/diploma 60 – 120 ECTS Master degrees of various types (not credit-rated) Doctorate/Ph.D. The implication is that one full year of study gives 60 ECTS and that some diplomas and postgraduate degrees take two years, hence The Bachelor system is based on a three or four year programme, hence 180 or 240, with the first 3 years as general and the final year as what we might call honours. This system exists in some Scottish universities, but not in England with its 3-year honours degree. The foundation degree in the UK would probably conform to this system. This implies the development of the European Credit Transfer System (ECTS=credits for student workload to complete modules) from a simple credit transfer tool into a more sophisticated and powerful credit accumulation and transfer system. Problem with using ECTS as currency for HE transfer is the lack of levels in ECTS and the imprecise nature of ECTS credits, which in practice at the institutional level are not defined in terms of learning outcomes. There are current moves to remedy these deficiencies ( see section 3.6). Credits expressed in terms of learning outcomes are a powerful way to recognise and quantify learning achievement from different contexts; they also provide an effective structure for relating qualifications. The addition of the learning outcomes dimension has the potential to improve dramatically the effectiveness of ECTS as a true pan-European system. British Bologna Website School of Science and Technology Dr Jon Tepper, School of Science and Technology

6 Drivers for Outcomes-based Approach…
2nd UK Workshop on Constructive Alignment 28/03/2017 Drivers for Outcomes-based Approach… QAA policy for Programme Specification encourages outcome-based learning (OBL) (QAA, 2000) Government’s Widening Participation agenda (NCIHE 1997) gathering pace Student fees are here to stay (DfES, 2003) Emergence of TQI (HEFCE 2003) WP 44% of young people now experiencing HE Government aims for 50% by 2010 Student fees Expected increase to £3k from 2006 (to 2010) (see DfES, 2003) University Market already here! (non-EU students) TQI Programme specs are to become public domain Emphasis on learning outcomes for customers to base selection School of Science and Technology Dr Jon Tepper, School of Science and Technology

7 2nd UK Workshop on Constructive Alignment
28/03/2017 Implications Students (parents?) are clients and quality of programme designs will be a major selection criterion Widening participation = increased academic variability in student ability Cannot take for granted that all learners are independent and self-motivated Learning and teaching strategies used should consistently ensure that all students accepted on course are given opportunity to meet the intended learning outcomes TQI If promotion and reality are different then possible litigation Not all practitioners are aware of outcome-based approaches to educational design Many academics are experts in their field and naturally conscious of the way they learn….however it is questionable as to whether most are aware of how to make the important transition from learning outcomes to designing appropriate student-focused learning and assessment activities. School of Science and Technology Dr Jon Tepper, School of Science and Technology

8 Outcome-based Educational Design
2nd UK Workshop on Constructive Alignment 28/03/2017 Outcome-based Educational Design So learning outcomes are important and could be considered currency for academics and students! It is therefore very important that we use the most appropriate practices to ensure that students who are admitted onto our degree programmes are given every opportunity to acquire the stated learning outcomes and to demonstrate them through appropriate means. In this outcome-driven world, constructive alignment can help us meet the many challenges we face in ensuring we offer courses that are balanced, fair and appropriate School of Science and Technology Dr Jon Tepper, School of Science and Technology

9 Basic Components of a Module Design
2nd UK Workshop on Constructive Alignment 28/03/2017 Basic Components of a Module Design A Simple View Assessment Tasks Formative Summative Intended Learning outcomes Teaching and Learning Activities (TLAs) Learning or Curriculum Objectives Learning outcomes what the students ‘should’ have learnt having completed the teaching and learning activities Learning objectives Central pillar to teaching a unit or module. Teacher-orientated and/or student-orientated statements that specify what activities the students need to perform to achieve the learning outcomes. They govern the TLAs used (component activities). Refers to content topics and criterion for the level of learning required and that assessment can address i.e. the verb. (not teaching to test but how well they know something). Such objectives do not preclude unintended outcomes. Higher-level cognitive activities are open-ended and welcome unforeseen outcomes (generalise, solve unseen problems, develop theory to explain why) When defining learning objective its essential to consider the existing knowledge and experience of prototypical student entering the module. I take a generative view: Given a learning outcome, practitioner generates a sequence of learning objectives that lead the student from their current level of knowledge to the desired outcome. The learning objectives determine the types of TLA to use. Teaching and learning activities teaching methods and techniques that are chosen to get the students to do what the objectives nominate Assessment tasks (AT - summative) those student activities defined by the teacher to make official judgments in relation to student academic performance and on which awards are based Input Process Output School of Science and Technology Dr Jon Tepper, School of Science and Technology

10 Constructive Alignment (Biggs, 1996)
2nd UK Workshop on Constructive Alignment 28/03/2017 Constructive Alignment (Biggs, 1996) Outcome-based methodology for designing, promoting and assessing deep student learning Fundamental assumptions are constructivist rather than instructivist Students constructs his or her own learning through relevant learning activities Teachers create a learning environment that supports learning activities appropriate to achieving the desired outcomes School of Science and Technology Dr Jon Tepper, School of Science and Technology

11 2nd UK Workshop on Constructive Alignment
28/03/2017 Motivating Principle ‘If students are to learn desired outcomes in a reasonably effective manner, then the teacher’s fundamental task is to get students to engage in learning activities that are likely to result in their achieving those outcomes’ (Shuell, 1986: 429; emphasis added) Common sense? Well its amazing how common sense can evade common practice Biggs’s (1996) constructive alignment facilitates us to do just that! School of Science and Technology Dr Jon Tepper, School of Science and Technology

12 Biggs’s Constructive Alignment Model
2nd UK Workshop on Constructive Alignment 28/03/2017 Biggs’s Constructive Alignment Model I. The Teaching System Curriculum objectives expressed as verbs that students have to enact A The very best understanding that could be reasonably expected: might contain verbs such as hypothesize, apply to ‘far’ domains B Highly satisfactory understanding: might contain verbs such as explain, solve, analyze, compare C Quite satisfactory learning: might contain verbs such as elaborate, classify, cover topics a to n D Understanding at a level that would warrant a pass: low level verbs, also inadequate but salveagable higher level attempts TLAs designed to generate elicit desired verbs May be: Teacher-controlled; Peer-controlled; Self-controlled as best suits context Assessment tasks evaluate how well the target verbs are deployed in context The highest level verb to be clearly manifested becomes the final grade (A,B,C etc) Two systems: teaching system, which is what the teacher constructs, and the learning system which is how the student reacts Teaching system Curriculum/learning objectives lie in the centre – get them right and decisions has to how they are taught and assessed follow. Learning objectives refer to constructive activities that are most likely to achieve the desired outcomes for the topic or unit. Activities are verbs so we specify appropriate verbs. Rather than just specifying content we are specifying levels of understanding. The level of understanding may be determined by a taxonomy such as SOLO taxonomy or Blooms. When we develop the objectives, we design teaching and learning activities (TLAs) that are likely to encourage students to engage the optimal verbs. Finally we then select the assessment tasks that will tell us whether and how well each student can meet the criteria expressed in the objectives Learning outcomes Learning activities II. The Learning System Dr Jon Tepper, School of Science and Technology

13 Questions, questions, questions
2nd UK Workshop on Constructive Alignment 28/03/2017 Questions, questions, questions “What do we base alignment on?” “How do we make alignment visible and evident to all?” “What is the student’s role?” “How can we measure alignment? If we can measure it, we can manage it can’t we?” “aligning to predefined objectives restricts development doesn’t it?” “It’s what we do already isn’t it?” Popular project management mantra! How can we change something if we cant quantify the level of disorder or misalignment? In my experience, I don’t think alignment is visible or clearly evident in practice or audit (perhaps I’m naïve) but reasons for this could be: Traditional transmission theories of teaching ignore alignment. Norm-ref – just want general idea of how students compare with each other. No relation between what is taught and what is tested. Some administrative factors, such as resource limitations, which dictate large classes with mass lecturing and multiple-choice testing, make alignment different. People hadnt thought of it before – focused more on content Others might like to use the principle but they don’t know how to “Wouldn’t it be nice if there was a system that could guide practitioners to useful consistent answers” Dr Jon Tepper, School of Science and Technology

14 2nd UK Workshop on Constructive Alignment
28/03/2017 Alignment is Based on The VERB! TLAs Learning Outcomes Bloom’s Taxonomy (Bloom et al 1956) 6. Evaluation: judge, appraise, assess 5. Synthesis: design, organise, propose 4. Analysis: analyse, inspect, compute 3. Application: apply, use, demonstrate 2. Comprehension: describe, explain, discuss 1. Knowledge: define, list, name Learning Objectives ATs SEE ALIGNMENT TABLES HANDOUT ALIGNMENT EXAMPLE HANDOUT Alignment is based on what students are expected to do Verb-matching schemes as defined and referenced in Biggs (1999) are used to cluster or categorise outcomes, objectives, TLAs and ATs according to the level of cognitive ability they elicit or assess from the student. BUT not an exact science! Dr Jon Tepper, School of Science and Technology

15 A Metric for Constructive Alignment
2nd UK Workshop on Constructive Alignment 28/03/2017 A Metric for Constructive Alignment Purpose Provide a quantitative measure of module alignment Facilitate teachers to adapt their practice to improve the alignment of their modules Premise Adopt a systemic and structural view of learning and teaching Categorise system components according to the level of cognitive ability they elicit from the student Use linear algebra to represent and compute alignment We can measure the degree to which a module/programme is aligned School of Science and Technology Dr Jon Tepper, School of Science and Technology

16 2nd UK Workshop on Constructive Alignment
28/03/2017 Generating Trees <empty> Lbn (a) Lo tree, c1=3 (b) Lb tree, c2=2 (c) AT tree, c3=4 Lo0 …….. Lo1 Lo2 Lom Lb1 Lb2 Lb3 Lb4 Lb5 Lb0 TLA1 TLA2 TLA3 AT0 AT1 AT2 ATp Lb10 Nodes have dominance and trees have fixed valence Generate only three tree types School of Science and Technology Dr Jon Tepper, School of Science and Technology

17 Structural Perspective
2nd UK Workshop on Constructive Alignment 28/03/2017 Structural Perspective AT0 AT1 AT2 Lo1 Lo0 Lb3 Lb1 Lb2 TLA1 TLA2 TLA3 <empty> TLA4 Lo2 Teaching & Learning Assessment School of Science and Technology Dr Jon Tepper, School of Science and Technology

18 2nd UK Workshop on Constructive Alignment
28/03/2017 Computing Alignment Index each textual definition of an outcome, objective, TLA and AT to a level in Bloom’s taxonomy Individually, calculate the alignment between Learning outcomes (W) and learning objectives (X) Learning objectives (X) and TLAs (Z) Learning objectives (X) and ATs (Y) Alignment between W and Z is deduced from a) and b) Alignment between W and Y is deduced from a) and c) School of Science and Technology Dr Jon Tepper, School of Science and Technology

19 From Theory to Practice
2nd UK Workshop on Constructive Alignment 28/03/2017 From Theory to Practice Learning outcome #1: Construct a context data flow diagram (DFD) given a specification (5) Describe the purpose of DFDs (2) Construct a context DFD for a simple system (5) Learning outcome #2: Describe the process of DFD balancing (2) Explain the process of DFD levelling and balancing (2) <blank> Learning outcome #3: Construct a level 1 DFD given a context diagram and high-level system description (5) Define the components of a context diagram (1) List the stages required to create a level 1 DFD (1) A fully worked alignment example is provided in the proceedings. For this presentation we will just calculate alignment between outcomes and objectives School of Science and Technology Dr Jon Tepper, School of Science and Technology

20 2nd UK Workshop on Constructive Alignment
A Worked Example. . . 28/03/2017 Calculate alignment between learning outcomes (W) and learning objectives (X) So there are m outcomes (i.e. 3) and n objectives (i.e. 2) for each outcome. Vector x2 from matrix X refers to the teacher defined objectives for outcome 2 i.e. x2=[2 2]T Where vector d1i from D is the vector of n ‘desired’ objectives for outcome i i.e. d12=[2 2]T 1. Calculate the desired alignment values Naïve view of ‘desired’: assume that each objective elicits same level as associated outcome thus t1i=n*wi2 for all i = 1 .. m From text to vectors and matrices and thus computation School of Science and Technology Dr Jon Tepper, School of Science and Technology

21 2nd UK Workshop on Constructive Alignment
28/03/2017 A Worked Example. . . Calculate alignment between learning outcomes (W) and learning objectives (X) 2. Calculate the actual alignment values for all i = 1 .. m 3. Calculate the misalignment (error) values Explain –ve misalignment We also have a notion of +ve misalignment -ve misalignment perfect alignment Dr Jon Tepper, School of Science and Technology

22 2nd UK Workshop on Constructive Alignment
28/03/2017 A Worked Example. . . Calculate alignment between learning outcomes (W) and learning objectives (X) 3. Calculate the misalignment (error) values cont’d… For each learning outcome i Do If |e1i|<=τ Then If one or more x’ji = wi’ (for each j) Then the learning objectives are aligned with learning outcome i Else learning objectives are not fully aligned with learning outcome i Else If |e1i|>τ AND e1i >0 Then the learning objectives are positively misaligned with learning outcome i The learning objectives are not fully aligned with learning outcome i The learning objectives are negatively misaligned with learning outcome i We set a threshold for an acceptable error level… School of Science and Technology Dr Jon Tepper, School of Science and Technology

23 2nd UK Workshop on Constructive Alignment
28/03/2017 A Worked Example. . . Calculate alignment between learning outcomes (W) and learning objectives (X) 4. Calculate the overall alignment value for W and X It is unlikely that an equilibrium of 0 (perfect alignment) will be achieved, however an acceptable threshold τ should be empirically established. We use the common root mean squared error to give us an average measure of alignment across the outcomes (in the case shown). Here, I’ve shown the specific alignment value for that between outcomes and objectives Assuming V2 (X and Z) and V3 (X and Y) equilibriums have been calculated, the alignment value for the entire module is simply: Dr Jon Tepper, School of Science and Technology

24 2nd UK Workshop on Constructive Alignment
28/03/2017 Feedback Good So Far…. After using the model to constructively align a Level 1 undergraduate computing module, the following student feedback was received: Of the 47 respondents: 97% agreed module was well-organised (40% strongly), an increase of 35% over the previous year 94% agreed seminars were useful (30% strongly), an increase of 4% 98% felt that my teaching methods and style were effective (30% strongly) a significant increase of 22% Constructive alignment appears to enhance the student learning experience Selection of objectives, TLAs and ATs guided by model. School of Science and Technology Dr Jon Tepper, School of Science and Technology

25 2nd UK Workshop on Constructive Alignment
28/03/2017 But…. Computation of alignment metric is dependent on three important factors: the ability to accurately cluster outcomes, objectives, ATs and TLAs according to the level of cognitive ability they elicit or assess; a priori definitions of acceptable prototypes of perfect or ‘desired’ alignment values from which to ‘benchmark’ against; defining realistic alignment threshold values. School of Science and Technology Dr Jon Tepper, School of Science and Technology

26 2nd UK Workshop on Constructive Alignment
28/03/2017 Summary Constructive alignment integrates instructional design with constructivist principles to produce a framework that systematically operationalises the important characteristics of a good teaching practitioner We have shown that the level of constructive alignment can be measured using vectorial representations and computations to provide numerical measures of alignment for both holistic and individual aspects of an educational design Alignment metric is hopefully a contribution to a wider collection of metrics to measure educational phenomena Educametrics? Biggs states that any discussions about good teaching should include that of alignment models School of Science and Technology Dr Jon Tepper, School of Science and Technology

27 2nd UK Workshop on Constructive Alignment
28/03/2017 Summary Currently putting collaborative ESRC/EPSRC bid together with colleagues from Warwick, Birmingham, Aston and Bedfordshire universities to develop an intelligent software system that facilitates staff to create constructively aligned curricula Development of a metric engine Intelligent natural language processor to associate verbs with TLAs, ATs Graphical representations of programme and module alignment (jargon free!) Integration with popular VLEs Potential cross-over with systems development (automatic generation of aligned system given requirement statements) Biggs states that any discussions about good teaching should include that of alignment models School of Science and Technology Dr Jon Tepper, School of Science and Technology

28 2nd UK Workshop on Constructive Alignment
28/03/2017 References Biggs, J. (1996) Enhancing teaching through constructive alignment. Higher Education, 32: 1-18. Biggs, J. (1999) Teaching for Quality Learning at University, The Society for Research into Higher Education, Buckingham: Open University Press. Bloom, B. (1956) Taxonomy of Educational Objectives Handbook I: Cognitive Domain, New York: Longman. Cohen, S. A. (1987) Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8): Department for Education and Skills (DfES) (2003) The Future of Higher Education: DfES White Paper, London: HMSO. School of Science and Technology Dr Jon Tepper, School of Science and Technology

29 2nd UK Workshop on Constructive Alignment
28/03/2017 References HEFCE (2003) Information on quality and standards in higher education: Final guidance [WWW] (6 January 2005), p.10. HEQC (1997) Graduate Standards Programme: Final Report, London: Higher Education Quality Council. National Committee of Inquiry into Higher Education (NCIHE), (1997) The Dearing Report: Higher Education in the Learning Society, London: HMSO. QAA (2000) Guidelines for Programme Specification. Gloucester: Quality Assurance Agency for Higher Education Shuell, T. J. (1986) Cognitive conceptions of learning. Review of Educational Research, 56: School of Science and Technology Dr Jon Tepper, School of Science and Technology


Download ppt "2nd UK Workshop on Constructive Alignment"

Similar presentations


Ads by Google