Title Slide Making Evaluation Judgements A quick overview He Oranga Poutama Provider Hui, 12-13 October 2009 - Te Manukanuka o Hoturoa Airport Marae Nan.

Slides:



Advertisements
Similar presentations
Performance Assessment
Advertisements

The Enchanted Forest. Project Aim To use a Storyline approach to study the effects on early literacy.
The National PE & Sport Professional Development Programme PD/H: Assessing progress and attainment in PE.
Te Tukunga Iho o Te Pu o Te Wheke National Whānau Ora Hui Holiday Inn, August 2011 Te Pu o Te Wheke.
Creative Movement in the Foundation Phase
E-asTTle Writing All you ever wanted to know……. “Launched in November 2007, the Revised New Zealand Curriculum sets the direction for teaching and learning.
Foundation of Nursing Studies in partnership with the Burdett Trust for Nursing Patients First: Supporting nurse led innovation in practice Workshop 1.
Assessment Assessment should be an integral part of a unit of work and should support student learning. Assessment is the process of identifying, gathering.
Nau mai, haere mai Introductory Webinar Please dial into the audio conference a few minutes before the session is due to begin. On your phone dial: At.
Hillsmeade Primary School Term Teacher Professional Leave These PD and focus group sessions are designed to assist all staff to gain an understanding.
P.A.T.H Planning tool Presentation to the Whānau Ora Hui – Sharing the Learning Auckland, 15 – 16 August 2011 Kataraina Pipi and Mariao Hohaia (PATH Facilitators)
1 Cheryl Rau & Jenny Ritchie University of Waikato Presentation to “The Politics of Early Childhood Education” 4th Annual Professional Development Symposium.
TECHNOLOGY IN THE NEW ZEALAND CURRICULUM 2007 What does it all mean? 1.
Matariki Waihi Matariki During Matariki we celebrate our unique place in the world. We give respect to the whenua on which we live, and admiration.
Culture in Aotearoa New Zealand Level 3 Social practice : Bridgepoint Waitakere :
Implementing Values through Community Action Research Dr Josephine Bleach
Consistency of Assessment
Paula White & Amohia Boulton Aotearoa New Zealand Evaluation Association (anzea) Conference August 2011.
Workshop 1. Patients First Introductions Poster viewing Confidence line Time capsule Hopes, fears and expectations.
Impact Measurement and You Understanding Impact Measurement, Practical Tools, Key Questions.
Parent teacher evening 20/2/12
Curriculum Innovation 20 th November. Programme for the day Purpose and understanding of innovation Engaging stakeholders Leadership and management.
Critical Issues in Māori Education – Building a Research Agenda Jessica Hutchings, Alex Barnes, Trina Taupo, Nicola Bright – Te Wāhanga, NZCER Leonie Pihama,
Theme 2: Expanding Assessment and Evaluation for FNMI Students Goal #1: First Nations, Métis and Inuit student achievement is increased as measured by.
WELCOME to WELCOME to Implementation Training for: The Revised Ontario Curriculum for First Nations, Métis, and Inuit Studies Grades Day 1.
Our three year strategy >Our vision >Children and young people in families and communities where they can be safe, strong and thrive. >Our mission >Embed.
Welcome to Workshop 3 The Foundation of Nursing Studies (FoNS) in Partnership with the Burdett Trust for Nursing Patients First: Supporting Nurse-led.
Interstate New Teacher Assessment and Support Consortium (INTASC)
WHANAU ORA DELIVERING ON THE PROMISE ‘Parallel Journeys and Shared Opportunities’ LYVIA MARSDEN NZRN QSM.
NSW Curriculum and Learning Innovation Centre Draft Senior Secondary Curriculum ENGLISH May, 2012.
1 Adopting and Implementing a Shared Core Practice Framework A Briefing/Discussion Objectives: Provide a brief overview and context for: Practice Models.
 Community Coaching for Planning, Action, and Evaluation A CYFERnet-Community Online Workshop May 18, 2011 Laura Laumatia University of Idaho Susan Jakes.
Developing a Culture for Care Jeanette Cookson – Locality Manager.
APAPDC National Safe Schools Framework Project. Aim of the project To assist schools with no or limited systemic support to align their policies, programs.
Reading and Writing National Standards workshop phase 2 Term
Thomas College Name Major Expected date of graduation address
Values based framework & reflective practice. Outline Values- Definition HPS National Strategic Framework- values driven How we identified the shared.
Gifted and Talented: Self review Presented 28 Sept 4.15 – Webinar 3 of a series of 4 Carolyn English LEARNING MEDIA.
Key Principles for Growing e-Learning Teacher Capability in NZ schools A Pilot Project With a focus on meeting the needs of Māori students
Evaluation. HPS is a “change” process that takes place within a school community A key consideration is that the change needs to be sustainable.
Where to start? Reflection on a new curriculum in Hauora Māori for Early Learners in Medicine Joanne Baxter
A Curriculum for the future The new Secondary Curriculum Phase 2 Implementation (key messages)
Managing Organizational Change A Framework to Implement and Sustain Initiatives in a Public Agency Lisa Molinar M.A.
College of Science and Engineering Learning and Teaching Strategy Planning Meeting Initial Reflections Nick Hulton.
Exploring Evidence.
VELS The Arts. VELS (3 STRANDS) Physical, Personal and Social Learning Discipline-based Learning Interdisciplinary Learning.
Performance and Development Teacher Librarian Network
Mountains and Plains Child Welfare Implementation Center Maria Scannapieco, Ph.D. Professor & Director Center for Child Welfare UTA SSW National Resource.
International Baccalaureate GLOBAL CONTEXTS. What are Global Contexts? You learn best when your learning experiences have context and are connected to.
Mountains and Plains Child Welfare Implementation Center Maria Scannapieco, Ph.D. Professor & Director Center for Child Welfare UTA SSW Steven Preister,
Transforming Patient Experience: The essential guide
Developing a Framework In Support of a Community of Practice in ABI Jason Newberry, Research Director Tanya Darisi, Senior Researcher
The Value of Rubrics in Messy Non-Profit Evaluation Contexts Kate McKegg American Evaluation Association Conference Washington DC, 2013.
How To Analyze a Reading Presented By: Dr. Akassi Content From The Norton’s Field Guide To Writing.
Case Studies and Review Week 4 NJ Kang. 5) Studying Cases Case study is a strategy for doing research which involves an empirical investigation of a particular.
KA HIKITIA Knowing Our Learner: Culturally Responsive Curriculum Kiri Rua School Support Services University of Waikato Trident School, 15/10/2010.
Analysis and Critical Thinking in Assessment 1. What is the problem? Gathering information Using information to inform decisions/ judgment Synthesising.
MY TIME, OUR PLACE Framework for School Age Care In Australia Prepared by: Children’s Services Central April 2012 Team Meeting Package.
Proposal for the National Curriculum Framework (NCF) for Children from Birth to Four And the Support Programme for Practitioners and Parents Hasina Ebrahim.
Presentation Presenter: Denise Forte (UK) CAME Project 6 th October 2011 Project training materials: Workbooks and Trainers’ Notes from the CA-ME project.
Copyright © May 2014, Montessori Centre International.
AES 2010 Kate McKegg, Nan Wehipeihana, Jane Davidson.
Ko te reo te tuariki. Language is my identity. Ko te reo tōku ahurei. Language is my uniqueness. Ko te reo te ora. Language is my life.
A lens to ensure each student successfully completes their educational program in Prince Rupert with a sense of hope, purpose, and control.
AES 2012 Kate McKegg and Syd King (for Nan Wehipeihana)
WE WANT EVERY LEARNER TO HAVE:
Introduction and Overview
Introduction and Overview
A Whole School Approach
TE TIRITI O WAITANGI PARTNERSHIP He Kōrerorero
Presentation transcript:

Title Slide Making Evaluation Judgements A quick overview He Oranga Poutama Provider Hui, October Te Manukanuka o Hoturoa Airport Marae Nan Wehipeihana, Kataraina Pipi and Kate McKegg Grappling with Uncertainty in Innovative and Complex Settings: Weaving Quality in Developmental Evaluation Kataraina Pipi, Nan Wehipeihana and Kate McKegg

Introduction Kataraina, Nan and Kate are the developmental evaluators for the SPARC He Oranga Poutama Programme – A culturally based sport and recreation programme delivered in 12 locations in New Zealand In this presentation we explore quality in a cultural context – on a developmental evaluation journey

The programme aims to increase participation in sport and recreation in culturally specific ways – ‘As Māori’ A programme review resulted in a ‘re-visioning’ A new focus on culturally led and driven participation Key drivers: – Tino rangatiratanga (self determination) – Drawing on 20 years of Māori socio-cultural and political development The story of He Oranga Poutama

Why developmental evaluation? Uncertainty around the new programme concept and direction – Exploration of ‘as Māori’ participation in sport and recreation Traditional evaluation methods not suitable – Emergent understanding of culturally based indicators and outcomes No formal culturally based evaluation evidence, lack of data available to measure key cultural concepts and variables Complex environment – contracting economy, political environment, changing organization

What does quality look like in Developmental Evaluation in indigenous contexts? Kataraina Pipi

Quality evaluation in a cultural context Cultural practice at the centre of our process The right thing to do Context + relationships = quality Cultural norms and values guide our practice as evaluators

Cultural tools – the key to generating quality information Cultural icebreakers – Creating connections Cultural critical thinking tools – Tapping into Māori ways of thinking and being Cultural contemporary evaluative synthesis tool – Cultural analytic lens

Cultural ‘ice breakers’ create connections In pairs – one thing you love about the place you call home In fours – one thing we have in common The point is… – Relationships matters!

Values Exercise ‘As Māori’ critical thinking (adapted from Te Ngaru Learning Systems) What were the messages you received from your grandparents about being Māori? What were the messages you received from your parents about being Māori? What messages would you like your grandchildren to have about being Māori? What messages are you giving / have you given your children about being Māori? Being Māori

‘Surfaces’ implicit values and assumptions Draws out implicit values and assumptions Makes explicit – The things we privilege – The things we prioritize – The source of our ‘valuing’ It is our belief systems which determine the outcomes that we value – and bring to evaluation

‘As Māori’ evaluation synthesis activity ActivityEvaluative CriteriaCredible evidence of excellence Data / tools Provide line dancing classes for kaumatua/k uia (elders) at the marae (traditional meeting place) 1.The number of kaumatua and kuia who turn up to the first class 2.The extent of positive feedback from kaumatua and kuia following the class 1.The numbers of kaumatua and kuia who come back for more classes 1.The level of interest from other marae High turnout of kaumatua and kuia at classes There is lots of positive feedback from kaumatua and kuia about the classes ‘It’s just one of those events you don’t miss’ High numbers are sustained over the course of the classes. Kaumatua and kuia spread the word about the classes to others. Demand from other marae to run the classes Registrations Attendance records Kaumatua and Kuia feedback Other whānau (family) feedback Videos and photos Attendance records Follow up calls Log of enquiries (phone, and verbal) from others

Cultural traditions and values at the heart of quality… Cultural traditions and practices privileged Context + relationships = quality

‘Talking Past Each Other’: The language of the developmental evaluator in indigenous contexts and its link to quality Nan Wehipeihana

It is through language that we come to know, understand and give expression to all aspects of who we are (Takao et al, 2010) It’s through language that we give meaning to the world… It’s in our language that our values are expressed… It’s in our language that identity is embedded The centrality of language

“It’s the damn English…” (Kirimatao Paipa) The importance of language for understanding The importance of language for validity Inaccurate or incomplete understandings of culture introduce systematic error that threatens validity Evaluation quality is about accuracy and trustworthiness of our understanding and the evaluative judgements we make

Examples from the field that worked Use the word ‘Poutama’ – Depicts a stairway or journey – Signal developmental or ongoing progression Mauri Ora – An expression of wellbeing – likened to excellence Rehua – Literally a constellation of stars – draws on traditional knowledge – The rehua (the constellation) is a visual anchor and reference for the evaluation framework

Evaluation language - that didn’t work ‘Rich description’ – For some, funders historically not interested in the cultural detail – For others, the term was not understood ‘Credible evidence’ – Reframed as what would you see, hear, feel – that would convince critics of success? – What is valid in cultural contexts and settings – to elders, to communities, to tribes? ‘Criteria’ – ‘the values that we practice, that we can measure’ (Thomas Tawhiri 2010)

Language is the connector Language can marginalize, disempower and disenfranchise Sometimes meaning is lost in translation – Start with language that connects It’s not about dumbing down… – Use of Māori language, metaphor and cultural symbols as a starting point for understanding  quality in evaluation

Drawing on deep values to ensure evaluation quality in emergent and uncertain contexts Kate McKegg

Applying developmental evaluation in a cultural context Draws on cultural values, needs, strengths and aspirations Cultural concepts, language and values are foundational within evaluation thinking, processes, tools, frameworks, data collection, reporting etc Achieves genuine and valid evaluation experience in the eyes of our communities

Applying developmental evaluation questions in a cultural context DE QuestionApplied Example (HOP) What?What’s being developed?What does / might ‘as Māori’ look like in the sport and recreation sector? What’s emerging? What sense can we make of emerging issues, evidence, data about this development? What are other’s doing in the ‘as Māori space? What are the patterns of ideas and practice emerging from the provider communities about ‘as Māori’ So what?What’s the value / significance / importance of what we’re doing and achieving? Whose values are we applying to our judgments? And what importance do different values place on what’s emerging? Now what?What does this mean for how we should now act? Into the future? How will be take our new understandings of ‘as Māori’ and apply these to the next phase of development?

The cultural framework (Rehua) dimensions By MāoriRefers to the extent of rangatiratanga / control by Māori in the governance, management and delivery of an initiative For MāoriEmphasises the importance of whānau, hapu, iwi, (family and tribal relationships) and other Māori collectives bought together for Māori purposes With te reo me ona tikanga Points to the importance of language and customs as central to the survival and affirmation of what it means to live ‘as Māori’ In/on places of cultural significance to Māori From ancestral domains (e.g., rivers, mountains and gathering places such as meeting houses), to more contemporary places such as kōhanga reo, kura etc. founded on Māori principles ThroughThose activities that are distinguished in uniquely cultural ways e.g., traditional and contemporary sport and recreational activities

Applying the framework Used to inform program design – to ensure that program activities aligned to the ‘as Māori’ outcome goal / vision Informed the development of evaluative criteria Influenced and changed the funders contract process Informed baseline data collection Reporting

How significant is the location? Extremely significant Strong connections to whakapapa, whanaungatanga and history (identity as Māori) It is the ideal place and cultural fit for the activity For example, it’s the right environment because you don’t have to explain it, it belongs there, there is no questioning of the authenticity The location or place enriches and enhances (strongly adds to) the overall experience For example, tikanga tells you it’s the right place because of the mauri and the wairua of the place Very significant There are some connections to whakapapa, whanaungatanga and history The location or place adds to or enhances the overall quality but connections are more general rather than specific It’s a culturally safe environment For example, it’s a place where it still feels OK – no one can say what are we doing here? There is no question of the appropriateness of undertaking the activity in that place

Reporting Cultural framework reflected in data collection and reporting

Culture is foundational to quality evaluation Valuing, affirming and privileging cultural practices, relationships, language and concepts leads to credible, valid evaluation Developmental evaluation is responsive to culture and cultural context