Valuing evaluation: A Case Study of Professional Development to Support Academic Engagement in Online Evaluation Processes and Outcomes Dr. Diana Quinn Senior Lecturer: Professional Development University of South Australia

2 Academic centred professional development Conceptual shift from ‘I will tell you this and therefore you will learn’ to: ‘I want to help you in ways which are effective for you and match your needs’. Starts planning professional development based on the current conceptions of academics Aims to meet academics’ needs (information and psychological)

University of South Australia3 Case study: Understanding the learners Education Arts and Social Science Division at University of South Australia –2003 academics : 238 FT; 66 fractional (planning); 260 sessional ( distribution list) –From 2005, evaluation of courses and teaching will only be able to be administered online Academics current conceptions –Negative values about evaluation –Evaluation linked to promotion and teaching awards –Online evaluation has low response rate –Students have ‘evaluation overload’

University of South Australia4 ACPD approach Strategies to improve response rates to online evaluation selected as topic Process –Identify those academics performing well –Interview, collate and share local best practice Strategies to improve accessibility to online Strategies to improve students’ understanding of evaluation –Collaborative document produced –Model desired behaviour (model s)

University of South Australia5 Three indicators of change 1.Number of student responses to course evaluation 2.Academics responses to online survey about ACPD 3.Informal comments

University of South Australia6 Number of responses to course evaluation in EASS

Responses to online survey Item Number of respondents (n=15) AgreeNeutralDisagree I thought that collating and sharing successful strategies from EASS academics to improve response rates was a useful way to conduct professional development 1311 I trusted these strategies as I knew they had worked for my peers 860 I appreciated being sent the model s suggesting how I could communicate with students about the CEI 1310 The model s were inappropriate for use with my students 159 I think I can continue to maximise response rates by applying these strategies 960 Compared to last year I have improved my understanding of online evaluation 960 Compared to last year, I feel more confident about using online techniques for evaluation 762

University of South Australia8 Informal comments I can’t thank you enough for this – it really helped the penny drop for me as far as online evaluation was concerned (personal communication, 2003).

University of South Australia9 Discussion Break the cycle of educators (and students) not valuing evaluation processes and outcomes Information needed to connected to academic’s current conceptions Collaboratively produced document was a comprehensive resource – trust the learner Academics psychological needs considered (belonging, power, freedom and fun)

University of South Australia10 Conclusion As long as there is no requirement for some type of educational qualification for academics to teach at university, there will be a need to provide meaningful and successful professional development about evaluation To make PD meaningful and successful, it needs to be academic-centred In ACPD, the information presented to academics needs to –recognise their current conceptions –respond to psychological needs

University of South Australia11

University of South Australia12 Questions What are the psychological needs of academics and how did they map against the professional development literature?

University of South Australia13 Support the need to belong Sharing best practice (Rogers, 1983) Foster peer learning in local environment (Boud 1999) Develop local communities of practice (Ryan 2000) Develop trust between academic and PD staff (Ryan 2000) Improve communication at all levels (Walsh & Metcalf 2003) Share local best practice, foster peer learning, develop trust, use honest and open communication (Glasser 1999)

University of South Australia14 Support the need for power Let academics lead, own and participate in all stages of the process (Walsh 2003) Emphasise how it will improve the institution (Walsh 2003) Provide access to models and support to complete task (Rogers 1983) Make exemplars and scholarly practice visible (Boud 1999, Ryan 2000, Walsh 2003) Integration of local and centralised support systems (Ryan 2000) Involve learner in decision making process, model what needs to be done, demonstrate the significance of the outcomes, provide adequate materials and support (Glasser 1999)

University of South Australia15 Support the need for freedom Offer the capacity to try new processes without commitment (Rogers 1983) Build academic’s self esteem (Boud 1999) Allow exploration without commitment, provide models and examples for them to either develop their interpretation from, or choose to ignore (Glasser 1999)

University of South Australia16 Support the need for fun Fun and humour are different Fun and learning interrelated ‘… when any of us is in any situation where we decide that we no longer want to learn, we stop having fun’ (Glasser 1988). Not a part of this project, but it is a consideration for future ACPD activities

University of South Australia17

University of South Australia18 Questions What is the psychological model that use used to plan ACPD?

University of South Australia19 Glasser’s Choice theory: filters Perceived World Knowledge FilterKnowledge Filter Values FilterValues Filter The Real World

University of South Australia20 Glasser’s Choice theory: perceived world Perceived World Knowledge FilterKnowledge Filter Values FilterValues Filter The Real World

University of South Australia21 Glasser’s Choice theory: Basic needs Belonging Power Freedom Fun Perceived World Knowledge FilterKnowledge Filter Values FilterValues Filter The Real World Needs

University of South Australia22 Glasser’s Choice theory: Quality world Belonging Power Freedom Fun Quality World Perceived World Knowledge FilterKnowledge Filter Values FilterValues Filter The Real World Needs

University of South Australia23 Comparison Behaviour Choices Glasser’s Choice theory: Choose to change Belonging Power Freedom Fun Quality World Perceived World Knowledge FilterKnowledge Filter Values FilterValues Filter The Real World Needs

University of South Australia24

University of South Australia25 Make sure you are heard Use language and context that is relevant to the learner (Learner-centred) (Various, including Moore and Patrick 2002) Ensure the proposed changes are compatible with academics current practice (Rogers 1983) Recognise the current knowledge and values held by learner and present information in a format that they will engage with (Glasser 1999)