Presentation on theme: "ESCalate Research Project ‘Using reflective dialogue to assess professional learning’ website Ruth."— Presentation transcript:
ESCalate Research Project ‘Using reflective dialogue to assess professional learning’ website http://escalate.ac.uk/6333http://escalate.ac.uk/6333 Ruth Pilkington RMHPilkington@uclan.ac.uk
Formal Learning Non-formal Learning Induction or Toolkit WeekUK PSF Courses I lead, e.g. University HE Teaching Toolkit Standard Descriptor One PG Cert LTHE or another PG Certificate Standard Descriptor Two PG Diploma & CPD awards MEdStandard Descriptor Three NEW HE STAFF EXPERIENCED HE STAFF Informal, on-the-job development Professional Dialogue Assessment Process A CPD FRAMEWORK FOR UCLan showing formal and informal learning routes to professional development and accreditation
The Uclan Approach Formal Courses Toolkit PGCert LTHE Certificates/other PGCerts PGDiploma MEd EdD Informal Process Experienced Discipline-specific Reflective Developmental Discursive Meaning-making = Professional Dialogue
Informal Learning Route: The ‘Professional Dialogue’ Normally takes place between an experienced ‘other’ and the ‘student’. It involves a 3-stage process: –an initial set-up dialogue, –an exploratory dialogic process, and –the assessed dialogue. Stage 3 is recorded and involves a second assessor to act as moderator or second marker. In Stage 3, one assessor comes from the subject or ‘field’ and acts as mentor through the preparation phase of dialogue, the other may have a broader, objective, generic teaching and learning perspective
Set up Dialogue: high Mentor input; low Mentee ownership Dialogue 1: high Mentor leadership; increased Mentee ownership Dialogue 2: shared ownership; increased Mentee input and determination Assessed Dialogue: celebration of Mentee role and practice Assessment and Reward: the Professional Dialogue showing Timeline, Input and Process Mentee Participant Mentor Adapted and adopted by Ruth Pilkington from the process outlined by Brockbank and McGill (2007)
Benefits Ethos of process Peer-led, supportive, celebratory, affirmative Status to teaching and learning; space Sharing and exchange Communication medium: reliability Rigour of process Can work with teams – culture change Meaning-making, reflective, subject and individual focus Not ideal for new staff or directed development needs
The Professional Dialogue Model Applies a model by Brockbank & McGill (2007). They outline specific conditions supporting a collaborative approach to process-oriented, active, peer-supported, reflective learning: –Ascertain a reflective dialogue has taken place (ideally with others); –Establish evidence of learner’s participation in dialogue; –Identify evidence of developmental process over time, regardless of the start / end point; –Ascertain evidence that process review has taken place, enabling students to take away understanding of the learning process and replicate it elsewhere. (p194) Boud & Falchikov (2007) use a cognitive model of coaching particularly relevant to peer assessment of professional learning called ‘cognitive apprenticeship ‘. Cognitive apprenticeship focuses on dialogue over an indefinite period using a structured process of ‘modelling, scaffolding, fading and coaching’ (Collins et al 1991:2, cited Boud & Falchikov, 2007:130). Coaching forms a thread that runs throughout ensuring that the collegial relationship is built on trust, confidentiality and empowerment. Meta-cognitive apprenticeship skills are developed in partnership with the experienced ‘other’ who initially models behaviours, scaffolds professional learning using reflective processes, encourages interrogation and engagement with self assessment, and then gradually fades out as ‘expert’.
Areas of Activity 1. Design and planning of learning activities and/or programmes of study 2. Teaching and/or supporting student learning 3. Assessment and giving feedback to learners 4. Developing effective environments and student support and guidance 5. Integration of scholarship, research and professional activities with teaching and supporting learning 6. Evaluation of practice and continuing professional development Core Knowledge 1. The subject material 2. Appropriate methods for teaching and learning in the subject area and at the level of the academic programme 3. How students learn, both generally and in the subject 4. The use of appropriate learning technologies 5. Methods for evaluating the effectiveness of teaching 6. The implications of quality assurance and enhancement for professional practice Values Statements for Professional Standards a) respect for individual learners b) commitment to incorporating the process and outcomes of relevant research, scholarship and/or professional practice c) commitment to development of learning communities d) commitment to encouraging participation in higher education, acknowledging diversity and promoting equality of opportunity e) commitment to continuing professional development and evaluation of practice The UK Professional Standards Framework
Project Process Four UK Institutions Professional Dialogues used 1.To assess academic leadership (Standard Descriptor 3) 2.To assess new lecturers wanting to meet Standard Descriptor 1 3.To assess experienced lecturers seeking Standard Descriptor 2 (core lecturer role) Dialogues recorded and transcribed Analysis of themes emerging (Tag Cloud, thematic identification, reflective cyclical review). attentiveness to issues – a personal response, discussion with colleagues – strengthen and derive codes – Structure of dialogues and management of interaction; facilitation and support; – Evidence of reflection and evidence of meeting Standard Descriptors – Phatics, use of small talk – Rigour of dialogue as assessment tool – Judgement and power; performance and control – Storytelling, narrative of meaning, cosntruction Reflection by team and review of data, experience Interview data collected from participants and assessors on process
Group Activity Explore within groups how this might work within your context Examine examples provided to discuss issues of dialogue within the process Can you use the model for your assessment? – (Share outlines from institutions and documents) What would be the issues for you in terms of assessing professional learning?
AFTERNOON SESSION Issues and Outcomes Power, Judgment Training and Support for Assessors
Review of Dialogues Modelling of dialogue Emergence of issues of power Analysis of different types of dialogue; ways that power emerges Evidence of judgment – a complex thing
Use of Questions Open Qs / Direct Closed Qs / Open Qs – Redirect issues or focus Closed questions move things on; focus attention Open Questions probe and explore Affirmatory and Summative statements
Dialogue – power and parity Power (assessor): scene setting, introductions; interruptions, questioning, probing & direction, judgment Power (assessee): taking ownership by managing exploration, redirecting questions, pursuing own line of thought, own judgments Reflecting back at assessee; affirmation; noise; comments on issues; sharing own thoughts; empathy – support; meta-cognitory unpacking Equality: shared exchanges; shared feelings; asides, exploratory detours
Enhancement components Reflection: description, thinking things through, exploring own experience, reasons behind, recognition of own values Storytelling: evidence, authenticity, entry to reflection, role and identity Relationship building: sharing, humour, use of phatics, empathy Meaning making: exploration, explanation, interpretation, reflecting back, meta-analysis, positioning within outcomes
Dialogue - Types Parity, Equality Assessor control Incr. Input by assessor Incr. Input by assessee Incr. Control from assessor Incr. Control by assessee monologue interrogation interview presentation dialogue supported monologue interview
Dialogue Cycles QUESTION PROBE REVIEW NEW QUESTION SHARE 1 SET CONTEXT TOPIC AFFIRMRESPONSE PROMPT / PROBE NEW TOPIC JUDGMENT EXAMPLE DESCRIBE LINK MADE CONCLUSION REPEAT CYCLES SUMMARISE GAP CHECK FINAL STAGE REFLECT REVIEW FEED BACK INTRODUCTION 2
Institution A Preparatory dialogue and final dialogue 2 nd final dialogue Peer dialogue Video of final dialogue by deaf studies Mentoring process Co-construction Story telling Turn-taking Power and management Open v. closed questions Judgment
Assessment, Reflection and Professional Judgement within Dialogue – an exciting outcome ICEBERG Dialogue An Interesting ‘Sub-text’ Assessment of professional knowledge Judgment Assessment of values Use of reflective models to assess professional learning
Judgment Expertise and experience of assessor Comfort and familiarity with outcomes (UK PSF) and field (L&T), and appreciation of subject (of assessee) Familiarity with setting: evidence, locus, complexity, authenticity – areas of activity Reading body language, examples, stories, language – truth, reliability, values Contextualising: alignment to outcomes; tacit v. explicit; hidden text Gap analysis: Is it enough? What more is needed?
Use of Dialogue Purpose of Dialogue Position of Dialogue Process Training of Assessor, Participant
Brockbank A & McGill I 2007 Facilitating Reflective Learning in HE SRHE/OUP 2 nd Ed. Bowen Clewley L ‘Assessing against competency standards in the workplace’ in 207-227 in Arguelles & Gonczi (2000) Eraut M 1997 Professional Learning and Competence Ghaye T & Lillyman S 2006 Learning Journals and Critical Incidents RP for HE Professionals 2 nd Ed Quay Books London O’Donovan B, Price M and Rust C (2004) ‘Know what I mean? Enhancing Students understanding of assessment standards and criteria’ in Teaching in HE Vol 9, No 3, July 2004 Knight P & Yorke M (2003) Assessment Learning and Employability SRHE/OUP Boud D and Falchikov N Eds Rethinking assessment in HE 2007 Routledge Bryan C & Clegg K 2006 Innovative Assessment in HE Routledge Goodfellow R & Lea MR (2007) Challenging E-learning in the University: a literacies perspective SRHE / OUP Larrivee, Barbara(2008)'Development of a tool to assess teachers' level of reflective practice', Reflective Practice,9:3,341 — 360 Moon J (2004) A Handbook of Reflective and Experiential Learning: Theory & Practice Routledge Falmer Pedler M (1996), Action Learning for Managers, The Learning Company Project Shulman LS ‘Knowledge and Teaching’ pp61-77 in Leach J, Moon B (eds) (1999) Learners and Pedagogy Sage/ Paul Chapman pubs