Promoting constructive alignment through programme specification and subject benchmarks Warren Houghton School of Engineering and Computer Science, University of Exeter

Plan. 1.Programme Specification a.How we went about it in Engineering at Exeter b.Discussion of process 2.Threshold standards 3.Defining differentiated assessment criteria at module level –Example and discussion 4.Helping students to manage their own learning 5.Levels of thinking about learning processes 6.A reflective framework for thinking about learning & teaching

Themes Responsibility Alignment Reflection

Context of examples University of Exeter – mid ranking old university – research lead Department of Engineering –in School of Engineering and Computer Science –small general engineering department –26 full time academic staff –approx 400 U/G students Heavily constrained by PEI accreditation

Context of examples 3-yr general BSc 3-yr BEng accredited ElectronicMechanicalCivil Eng. & Management 4-yr MEng accredited ElectronicMechanicalCivil Eng. & Man Common first year

How we wrote Programme Specifications 1.Put Subject Benchmark Statement to one side ! 2.Wrote aims and ILOs for existing programmes –many iterations - emergent outcomes 3.Wrote aims and ILOs for existing modules –many iterations –drawing out what staff were already doing 4.Then, checked against Benchmark Statements etc. 5.Did not try to achieve one-to-one mapping

Why not use Benchmark Statements as blueprints? What authority should we give the Benchmark Statements ? –What does the QAA say ? Is there a correct answer ? How can we obtain a set of required ILOs ? –From industry? Do we have to take responsibility, with our own ideas?

William Perrys positions: 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

What position do we take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

What position are we encouraged to take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

NOT as some definitive right answer We have to take responsibility for creating/recreating the curriculum –drawing on –our own experience –others experience - set out in benchmarks etc. an iterative, reflective, process How should we use benchmarks?

Curriculum development Emergent outcomes Rewrite Programme Specification Implement changes Read available literature. Benchmark Statements etc. Contribute to national discussion Compare curriculum with Benchmark Statements etc. Articulate current aims, ILOs etc. Curriculum experienced by teachers and students

A limitation of benchmarks? They are not explicitly multidimensional

A two-dimensional table for each assessment criteria heading Assessment criteria MEng BEng BSc 3 rd st Breadth and depth of programme Degree classification (performance)

MEng and BEng programmes : volumes in a space that is at least three-dimensional.

Dimensions of learning outcomes Order of thinking (e.g. Blooms hierarchy) Range of concepts difficulty of concepts

Simplistic levels view of a degree Order of thinking e.g. Blooms hierarchy Range of concepts

Allowing for other dimensions Order of thinking e.g. Blooms hierarchy Range or difficulty of concepts

and allowing for life before university ! Order of thinking e.g. Blooms hierarchy Range or difficulty of concepts Prior learning Degree programme

Plan. Programme Specification –How we went about it in Engineering at Exeter –Discussion of process Threshold standardsThreshold standards Defining differentiated assessment criteria at module level –Example and discussion Helping students to manage their own learning Levels of thinking about learning processes A reflective framework for thinking about learning & teaching

Threshold standards benchmarking implies... all graduates will meet all threshold standards we need to show how we may have to change our assessment QAA(2000) Engineering Benchmark Statement.

Threshold standards Effective in (e.g.) the Royal Navy Different in HE – why? –Education and training are different –Is certification realistic / useable ? Too much / different for employers to read PDP offers a solution to both problems

setting and assessing threshold standards in core academic modules Example: setting and assessing threshold standards in core academic modules in Engineering at Exeter

Traditional examination 3 hr paper Choose 5 out of 8 questions Pass mark 40% Pass provides evidence of 25% of ILOs tested But which 25% ? What can we build further learning on ?

define detailed ILOs/assessment criteria... For all 1 st and 2 nd year engineering modules

Module specification – A/B structure DETAILED LEARNING OUTCOMES / ASSESSMENT CRITERIA List A comprises core outcomes that will be covered fully in lectures and must be achieved by all students to meet the minimum requirement for progression. List B comprises outcomes that are EITHER more difficult to achieve OR are to be achieved by private study (or both). A: THRESHOLD LEVELB: GOOD TO EXCELLENT. Apply nodal analysis, with step by step prompting, to 2 loop circuits.. Apply nodal analysis, without guidance or prompting, to 2 and 3 loop circuits..

Assessment - examinations Paper A: –Covering list A ILOs only –Typically, short straightforward questions –No choice –Expected mark >80% –Criteria referenced Paper B –All ILOs, and some choice –Longer, more challenging questions with no easy parts –Expected average < 40%

Doesnt this approach mean that we are blatantly teaching to the examination? Yes !

Development Accepted because of PEI accreditation –evidence that students with different marks had achieved identifiably different learning outcomes Originally developed as part of a scheme to give better guidance to students

Impact on staff Staff find it hard to split ILOs this way BUT asking for differentiated ILOs seems to work better than just asking for single level ILOs

Realism Any test is demanding when the pass mark is 80% If we are genuinely going to test all ILOs they must be achievable. We have to be honest.

Deep vs. surface learning Are we encouraging surface learning? Other factors enable deep learning... Consider structure of the learning (noun) –Hierarchy of concepts What happens if students try to understand complex concepts when they havent grasped the components? A/B approach makes deep learning possible

a problem of success ?

Supporting students A/B split originally introduced for student guidance students asked to identify progress against ILOs on weekly basis now whole of 1 st year ILOs are a prerequisite to PDP

Why is PDP important ? It is about students becoming autonomous, independent, thinking for themselves –the real purpose of HE It enables students to articulate what they can do

PDP: is not a bolt on extra it is an integral part of learning in HE it must be addressed by all academic teaching staff

Academic staff view PDP in qualitatively different ways and many have difficulty with valuing PDP Why?

Levels of thinking about teaching Biggs (1 to 3): Focus on: 1. what the student is 2. what the teacher does 3. what the student does plus? 4. how the student can manage what the student does (PDP) 4. how the student can manage what the student does (PDP)

Ways of thinking about Generic Graduate Attributes Barrie (1 to 4): 1. Necessary basic PRECURSOR skills but irrelevant as they are a prerequisite for university entry 2. Useful skills that COMPLEMENT or round out disciplinary learning 3. These are the abilities that let students TRANSLATE, make use of or apply disciplinary knowledge in the world 4. They are the abilities that infuse and ENABLE university learning and knowledge

Biggs - teaching (plus) 1. What student is 2. What teacher does 3. What student does 4. How student manages learning 1. PRECURSOR, irrelevant 2. Useful COMPLEMENT 3. TRANSLATE learning 4. ENABLE university learning Barrie - attributes (skills+)

Paradigm shifts Plan Experience Reflect Theorise Kolb

Paradigm shifts - double loop learning New understanding Paradigm shift

Biggs - teaching (plus) 1. What student is 2. What teacher does 3. What student does 4. How student manages learning 1. PRECURSOR, irrelevant 2. Useful COMPLEMENT 3. TRANSLATE learning 4. ENABLE university learning Barrie - attributes (skills+)

Learning to learn Teachers must : have theories of learning not just bags of skills (see Ramsden) STUDENTS need study skills AND learning about learning –theories of learning –tools for metacognition / reflection / self management

A Reflective Framework Teacher as creator of subject learning environment Student working within created learning environment Teacher as provider of tools for reflection on learning Student acquiring tools for reflection on learning Teacher as learning tutor Student discussing own learning Teacher as subject tutor Teacher as subject tutor Student learning through subject related dialogue Teacher as reflective practitioner Student engaged in PDP

Teacher as creator of subject learning environment Student working within created learning environment Student engaged in PDP The created learning environment must be designed so that students can manage their own learning within it. It must offer: real choices for students to make resources to support different choices information required to make choices (ILOs etc.) assessment outcomes clearly linked to choices (i.e. aligned assessment)

P.B.L. ? Teacher as creator of subject learning environment Student working within created learning environment Teacher as reflective practitioner Teacher as provider of tools for reflection on learning Student learning about learning Teacher as subject tutor Teacher as subject tutor Student learning through subject related dialogue Student engaged in PDP Teacher as learning tutor Student discussing own learning

A dialogue about learning is essential : Teacher as creator of subject learning environment Student working within created learning environment Teacher as reflective practitioner Teacher as provider of tools for reflection on learning Student learning about learning Teacher as subject tutor Teacher as subject tutor Student learning through subject related dialogue Student engaged in PDP Teacher as learning tutor Student discussing own learning PDP as a bolt on extra

PDP: should be an integral part of the academic experience requires teachers who are reflective practitioners should have a profound impact on learning can be used as a tool for curriculum development

Plan. Programme Specification –How we did it in Engineering at Exeter –Discussion of process Threshold standards Defining differentiated assessment criteria at module level –Example and discussion Helping students to manage their own learning Levels of thinking about learning processes A reflective framework for thinking about learning & teaching

Summary Responsibility Reflection Alignment

Teacher as creator of subject learning environment Student working within created learning environment Teacher as reflective practitioner Teacher as provider of tools for reflection on learning Student learning about learning Teacher as subject tutor Teacher as subject tutor Student learning through subject related dialogue Teacher as learning tutor Student discussing own learning Studenttakingresponsibility

How do we achieve alignment of: with How academic staff think about research in their disciplines How academic staff approach curriculum development with How we want academic staff to teach How academic staff are managed ?