Evaluating Education for Sustainability - A reflection after 10 years AES 2011, Sydney Larraine J Larri

1. What is education for sustainability and how did I get involved in evaluating it? 2. From Environmental Education to Learnscapes 3. From Learnscapes to AuSSI – 10 years on from Pilot to Mature program 4. Reforms in pedagogy and environmental education – understanding the challenges and implications for the evaluator Productive pedagogies and EfS Systemic adoption of EfS Implications for evaluation 5. The contribution of evaluation to EfS 6. The contribution of EfS to evaluation 7. Overview

1. What is education for sustainability and how did I get involved in evaluating it? education for sustainability (EfS). EfS is a big idea that tries to rethink the way we learn, live and work together while respecting the capacity of the Earth’s natural systems to support life. Josephine Lang (2007) Why do people care for the environment? Altruism? Commitment to collective action ? Can we care for the environment and at the same time be materialistic and individualistic? Which societal values are the drivers for environmental sustainability? How can economic development be sustainable and profitable?

© L Larri, Renshaw-Hitchen & Assocs P/L Photo: Simon O'Dwyer 25 Jan 2007 The Age Mighty Murray Learnscapes Critical Success Factors /05 AuSSI Pilots NSW & Vic Plus ‘Comparative Assessment’ 2006 Waste Wise WA & future link with AuSSI WA 2008 AuSSI-Tas Pilot Model School & Systems levels 2008 Global Communities for Sustainability Pilot - Impact on student learning 2007, 2008 AuSSI facilitation National Partnership Workplan - M&E framework 2009/10 CarbonKids Pilot The Science of Climate Change and link to AuSSI Global Communities for Sustainability Phase 2 - Expansion AuSSI-ACT After 4 years AuSSI-Tas After 2.5 years Scenarios for the future of Australia From Environmental Education to Learnscapes

3. From Learnscapes to AuSSI – 10 years on from Pilot to Mature program The Australian Sustainable Schools Initiative (AuSSI) is a partnership of the Australian Government, and the States and Territories that seeks to support schools and their communities to become sustainable. The AuSSI involves participants in a whole-of-school approach, to explore through real-life learning experiences, improvements in a school’s management of resources and facilities including energy, waste, water, biodiversity, landscape design, products and materials. It also addresses associated social and financial issues. The Initiative’s vision is for all Australian schools and their communities to be sustainable a sophisticated and holistic program that sees the school as a model system for the broader community to achieve environmental sustainability

3. From Learnscapes to AuSSI – 10 years on from Pilot to Mature program program logic matrix (PL) was used to focus and plan the evaluations. a participatory process involving a range of stakeholders to develop the outcomes hierarchy, and contribute to fleshing out the other details in the matrix. Capacity building in evaluation

4. Reforms in pedagogy and environmental education – understanding the challenges and implications for the evaluator 4.1 Productive pedagogies and EfS how reforms in pedagogy and the adoption of EfS have been linked Gore and colleagues (2001) developed what they called, the ‘Productive Pedagogy Framework’ to guide teachers in the construction of teaching and learning programs Intellectual quality  Higher-order thinking  Deep knowledge  Deep understanding  Substantive conversation  Knowledge as problematic  Metalanguage Relevance (connectedness)  Connectedness to the world  Problem-based curriculum  Knowledge integration  Background knowledge Social support  Student control  Student support  Engagement  Self-regulation  Explicit criteria Recognition of difference  Cultural knowledge  Inclusivity  Narrative  Group identity  Citizenship Authentic Pedagogy - USA Productive Pedagogies - Queensland Essential Learnings – SA, Vic, Tas Quality Teaching & Learning - NSW

4. Reforms in pedagogy and environmental education – understanding the challenges and implications for the evaluator 4.2 Systemic adoption of EfS Ongoing tension between achieving environmental management targets as well as integrating pedagogical outcomes educationists core business is to achieve student outcomes and environmentalists whose concern is achieving behaviour change towards environmental sustainability. convincing schools and their communities as well as stakeholders that environmental education has a legitimate place in the school curriculum, equally that it should be a school’s objective to become a role-model for sustainable practices in the community. United Nations Decade of Education for Sustainable Development ( , DESD, UNESCO) Adelaide Declaration on National Goals for Schools in the Twenty-First Century - included Goal 17 Melbourne Declaration on Educational Goals for Young Australians (December 2008) April 2009, National Action Plan for Education for Sustainability, Living Sustainably. The plan aims to, equip all Australians with the knowledge and skills required to live sustainably. It is Australia’s contribution to the DESD inclusion of ecological sustainability in the definition of ‘contemporary science’; and ‘sustainability’ as a cross-curriculum perspective in the new national Australian science curriculum K–12

4. Reforms in pedagogy and environmental education – understanding the challenges and implications for the evaluator 4.3 Implications for the evaluator Understanding the context... appreciation of : pedagogical approaches involved in EfS and challenges for teachers in planning how to integrate it into the school’s curriculum. the challenge to roles of school leaders who are primarily educationists and have an expanded role of managing resources and the school environment. adoption of a whole school approach – systems thinking.

5. The contribution of evaluation to EfS Program logic Participatory evaluation 7. The local community manages it environment in ecologically sustainable ways 6. The school environment is effectively managed for teaching and learning in ways that are ecologically sustainable 5. The school and local community implement their Learnscapes Project Management Plan 4. The school Learnscapes Committee devises their Project Management Plan 3. The school establishes a Learnscapes Committee to manage environmental improvements through a curriculum response 1a. A school committee is established to manage the school’s external environment 1b. The school is aware of environmental education 2. The school is aware of Learnscapes as a way of combining environmental improvements with teaching and learning Learnscapes hierarchy of intended outcomes

5. The contribution of evaluation to EfS

participatory evaluation evaluation approaches that fit with the aspirations of EfS programs. Social connectedness is at the heart of sustainability. Embedded in the AuSSI model are structures dependent on whole school participation that values multiple perspectives in decision-making, in particular student involvement and strong links with community partners.

© L Larri, Renshaw-Hitchen & Assocs P/L 1

2008 AuSSI-Tas Pilot Model School & Systems levels

CarbonKids, Global Communities for Sustainability – Some benefits using Social Software in evaluation The nature of EfS programs: Social connectedness is at the heart of sustainability. The technologies being used have the dual ability to gather evidence and enhance networking across Australia and support cross-cultural, intergenerational, collaborative dialogue. Using online technologies is arguably more environmentally responsible than travelling across Australia to conduct case studies and interviews. Data quality: Quantitative and qualitative survey data: qualitative data is provided directly by the participants. It functions both as data and as online content. Engaging school systems in the evaluation: Technology literacy - Information and communication technology (ICT) skills of participants are increased and this is attractive to schools looking for appropriate real life opportunities for students and teachers. Involving the participants in the evaluation process: Participant use of aggregated data of baseline surveys, also ‘on completion’ data fed back to individual schools and for all schools. Space to seek feedback and critical reflection comment about the evaluation. Evaluator able to post notices and comments. Value for money: With limited budgets, the technology provides excellent affordability, reliability and ease of use. Reduces travel costs and has carbon offset benefits

6. The contribution of EfS to evaluation Rich opportunities for topics that stretch evaluation & research the development of students to become environmental citizens, take action and be leaders the nature and dynamics of school leadership in enabling change; determining the degree to which behaviours and attitudes have changed in relation to resource management and usage (i.e. water, waste, energy) and care for the environment (i.e. biodiversity) researching the ways in which student well-being has been affected by involvement in activities related to EfS

6. The contribution of EfS to evaluation 2: The nature and dynamics of school leadership in enabling change Data collection instruments and some findings from the AuSSI-ACT evaluation Question in Teacher Survey (AuSSI-ACT) How important has the School Leadership Team been in contributing to ‘successful’ behavioural change towards and adopting EfS? Items for rating: have been a driving force in motivating staff to adopt of EfS have supported and encouraged the EfS / Environmental Leadership Team have been involved in the development of the school's vision for EfS have promoted the vision of sustainability that the school developed e.g.they talk about it often, they are role models take action to remove barriers, and empower people to change, they are 'enablers' they generate enthusiasm about achievements and successes actively help the EfS leadership to build on achievements and keep going they promote ways to embed EfS into school systems and processes e.g. purchasing, curriculum plan etc What AuSSI-ACT Principals considered as Key Leadership Ingredients Excerpt of from focus group  Lead by example, be a role model, show that the program is valued by leadership and that it is a priority; communicate on many levels why the school is doing the Program, how it is progressing, and its achievements.  Aim for student ownership and efficacy - involve students in ways that increase their agency. AuSSI-ACT Teachers comments from surveys and focus group Principal support is imperative – the inspiration from the top is critical: - supportive in time/finances – total belief in the vision of being sustainable.

6. The contribution of EfS to evaluation Box 3: Items used in teacher survey to gain an impression of shifts in student behaviours Water: From using water indiscriminately and behaving as though it was abundant, to using water sparingly and valuing it as a precious resource. Waste: From creating waste without knowing or caring what happens to it to seeing it as able to be avoided, reduced, or a resource for reuse or recycling. Energy: From using energy indiscriminately to using renewable and non-renewable energy in ways that reduce energy consumption, lessen environmental impact and extend existing energy resources. Biodiversity: From not thinking about what impact we have on the environment to considering our actions and the effects they may have on biodiversity. Global environmental citizenship: From being concerned only about oneself to being concerned for local, national and global communities. Environmental sustainability leadership: From not being able to express concern about sustainability to recognising and taking responsibility for the changes required to develop a more sustainable society.

6. The contribution of EfS to evaluation :Dimensions of well-being used in AuSSI-ACT and AuSSI-Tas evaluations (based on Fraillon, J, 2004, ACER). The scenario of establishing and maintaining a school worm farm was used to assist teacher responses. Whilst this was not necessarily a good fit in all examples, it was hoped that it would act as a pointer towards a general direction for each dimension of well-being. Intrapersonal: a student's sense of self and their capacity to function in the school community. Interpersonal: a students’ appraisal of their social circumstances and their capacity to function in the school community. Autonomy : showing independence, actions and/or values are fully endorsed Emotional regulation: the degree to which the student's emotional responses are appropriate to the events surrounding Resilience: the capacity to manage, recover and move on from critically challenging events Self-efficacy: the degree to which the student believes they are effective Self-esteem: the way students feel about themselves Spirituality: a positive sense of meaning and purpose in life Curiosity: an intrinsic desire to learn more Engagement: engagement with the learning process and the school community Mastery orientation: the desire to complete tasks to the best of one's ability Communicative efficacy: using communicative skills in context to achieve a purpose Empathy: responding to another person with the same emotion and being able to understand another person's perspective Acceptance: having respect, tolerance, trust to be able to understand society and the qualities of others Connectedness: meaningful linkage with a wide range of people

6. The contribution of EfS to evaluation Two questionnaires were devised - one for teachers and the other for students (Year 4 or above). To date, the instruments have been used in the ACT and Tasmania. The Tasmanian data represent a significant sample. The respondent numbers were: ACT: 27 teachers (representing 27 schools i.e. 24% of the participating schools) and 43 students (from Years 4, 5 and 6) Tasmania: 50 teachers (representing 40 schools i.e. 53% of the participating schools) and 240 students (from Years 4 to 10 in 48 schools) teachers were able to use the instrument to identify significant positive impacts of EfS for their student’s well- being across most dimensions – they also gave examples of observed behaviours that supported their assessments. Students agreed that, ‘doing things for the environment made them feel better’ – from the ACT, 34 (or 87%); and from Tasmania, 143 (79%) in Tasmania. They also ranked a series of statements that linked with the dimensions used in the teacher survey. I feel like a plant with water. I feel like I'm doing better things for Australia. I feel happier and more safe. I feel really proud of myself because I know that I’m doing the right thing. It makes me feel happy inside that I can change the world in one small step. I feel like I’m eating a cake full of smiley faces and rainbows!!!!!

7. Overview each State/Territory has been able to contextualise the AuSSI model for their policy and schooling systems in a way that is achieving the structural shifts towards the AuSSI vision For all Australian schools and their communities to be sustainable. After 4 or so years... a critical mass of schools that understand the model and how its component elements can be structured to effectively achieve systemic behavioural change towards sustainability. The challenge... ensuring that this strong foundation becomes self-sustaining within each of the States/Territories. My advice... focus more sharply on what works best in each context and strive for greater efficient and effective for all. Avoid dissipating the limited energy and resources of schools and AuSSI project officers on time consuming activities with little return on investment in either student outcomes in EfS or school recognition (e.g. writing unsuccessful grant/award applications; expanding curriculum integration across too many areas so that practice is 'thinly' spread rather than deepening and consolidating what is now known to work well for 'sustainable' EfS).

7. Overview What is it that I really believe? Sustainability underpins our school’s core values – care and respect for one another. Through the sustainable schools initiative as a staff we have become more aware of the incredible value of educating for sustainability – how it will equip our students for life. It engages students through their hands, hearts and minds. It encourages them to problem solve, care, reflect and research. It’s about kids making connections and shifting the control and ownership of learning from the teachers to the students. The teachers act as learning facilitators – providing big picture opportunities – authentic, personalised learning. Creating partnerships with the community. Vicki Hales, Principal of Latrobe Primary School Quote from material provided by AuSSI-Tas