The Activities Matter: Exploring a framework to evaluate the impact of university-led outreach interventions Annette Hayton and Dr Andrew Bengry-Howell Podcast available via

Evaluating successful interventions During an activity such as this you can feel and sense the energy excitement enthusiasm personal transformations BUT - is this enough to say that this summer school was an effective intervention? What is success – it varies according to your position and interests Government, OFFA HEFCE University SMTs interested in the long term impact on progression to HE - longitudinal tracking could provide this but cannot capture the transformative process that appears to take place WP practitioners, students and school teachers very interested in the activity itself and its short term impact – success is seen as a perceived positive experience for participants Both essential but represent a real dichotomy in how we think about WP interventions

2014 Year 12 Summer School -Clear commitment to move beyond the usual generic offerings to include subject-based sessions -Although aims were identified absence of overall framework meant they lacked focus and coherence for example: -Provide an introduction to key aspects of university life -Develop Independent Research and Learning Skills -Raise student attainment -Evaluation ‘added on’ and not integral to event

2014 Year 12 Summer School “I was quite worried that I would just be sitting alone all the time in my room, but the organisers made a really good job of getting everyone to talk to each other and get to know each other.” “I think it has been really good to actually work with the academics, rather than just have someone telling you about the course. The academics are at the cutting edge of the research and are able to tell you about what they are doing right now. That was a great opportunity.” “I liked working in the labs. It’s so different from college, so it was good to know what sort of environment you would be working in if you did a particular course.” “Coming on the summer school has made coming to uni less scary … because I didn’t really want to go to uni. I did, but I was really scared. So this has been really helpful.”

Progression to higher education

Capitals Bourdieu … recognises that social or cultural capitals are expressions of differences in status and power within an unequal social system. In his analysis, economic, social and cultural capitals are all considered in terms of enabling and restricting engagement with education. He also expanded his concept of capitals in relation to education, distinguishing between intellectual or scientific capital (subject expertise), academic capital (understanding of rules and customs within the academy) and social capital (social connections). Whitty, Hayton and Tang (forthcoming)

Field of widening participation

Evaluating widening participation outreach Need to develop: 1.aims for interventions informed by theory and practice 2.content which reflects the aims 3.appropriate methods to generate useful data 4.evidence to demonstrate impact and inform practice and theory

Reflexivity

Evaluation Framework SOCIAL AND ACADEMIC CAPITAL HABITUS EDUCATIONAL CAPITAL? INTELLECTUAL & SUBJECT CAPITAL PROGRESSION CURRICULUM STUDENT IDENTITIES SKILLS CURRICULUM KNOWLEDGE CURRICULUM KNOWCHOOSEBECOMEPRACTISEUNDERSTAND Develop students' knowledge and awareness of the benefits of higher education Develop students' capacity to navigate Higher Education sector and make informed choices Develop students' confidence and resilience to negotiate the challenges of university life Develop students' study skills and capacity for academic attainment Develop students' understanding by contextualising subject knowledge

Reflexivity Framework enables us to share aims and develop activities with -Participants -Parents -School teachers -WP outreach workers -University academic and professional support staff

2015 Year 12 Summer School High intensity intervention Common top level objectives for a Level 3 event inform discussions and planning Specific activity objectives developed within the framework Evaluation approach integral to planning using a range of methods

Common Aims with specific objectives for a Level 3 Intervention 1. Develop students' knowledge and awareness of the benefits of higher education Enable students to investigate course and placement options, and social and leisure opportunities at the University of Bath and other universities 2. Develop students' capacity to navigate Higher Education sector and make informed choices Enable students to evaluate university courses and make informed choices that align with personal interests and future aspirations 3. Develop students' confidence and resilience to negotiate the challenge of university life Enable students to anticipate challenges they will face in Higher Education and make a successful transition to university 4. Develop students' study skills and capacity for academic attainment Enable students to enhance their academic skills through collaborative projects that develop capacity for critical thinking, independent research and self-directed learning 5. Develop students' understanding by contextualising subject knowledge Enable students to: (a)situate their existing knowledge within wider fields of knowledge and apply to other contexts. (b)Situate their existing knowledge and interests within the context of university degree programmes and academic disciplines

2015 Year 12 Summer School Monitoring participants against agreed WP criteria e.g. State school, socio-economic class, LPN, parental education, gender, ethnicity, disability, Local Authority Area Longitudinal tracking – HEAT tracker Measurable indicators for event objectives Follow up

2015 Year 12 Summer School Aim1: Social and Academic Capital: Know e.g. Specific content and entry requirements of courses at University of Bath -Clickers to record students self-assessed learning -Tests to assess knowledge? -Focus groups would illustrate what they know Aim 2: Social and Academic Capital: Choose e.g. Evaluate different types of HEI in terms of personal interests and career aspirations -Facilitated group activity -Reflective discussions conducted by staff

2015 Year 12 Summer School Aim 3: Habitus: Become e.g. A positive immersive experience of HE and University of Bath -Visual documentation -Clickers to record students’ satisfaction feedback -Participant observation, field work, focus groups, staff & student ambassador reflections -Postings on social media -Follow-up online survey Aim 4: Educational Capital?: Practise e.g. Enhance communication and presentation skills using different media -Peer evaluation of verbal presentations -Staff assessing posters and giving feedback -Confident participation in debates

2015 Year 12 Summer School Aim 5: Intellectual Capital: Understand e.g. Enhance understanding through collaborative projects, which extend knowledge and challenge assumptions -Collaboration: active learning -Projects: creation of knowledge -Challenge assumptions: encourage new ways of thinking -A ‘product’ to demonstrate learning and understanding -Ability to put a coherent and robust argument in a debate -Staff and student feedback on capacity of students to apply and develop their understanding -Participant reflection

2015 Year 12 Summer School Aim 1: Develop students' knowledge and awareness of the benefits of higher education Enable students to: Investigate course and placement options, and social and leisure opportunities at the University of Bath and other universities Discover course and placement opportunities at the University of Bath and opportunities at other universities Find out about research areas, expertise and facilities at the University of Bath and new areas of development Explore social and leisure, and extra-curricular opportunities at the University of Bath Discover career benefits of Higher education and the employment opportunities for University of Bath graduates. Find out about academic and information services, facilities and resources at the University of Bath

2015 Year 12 Summer School Aim 2: Develop students' capacity to navigate Higher Education sector and make informed choices Enable students to: Evaluate university courses and make informed choices that align with personal interests and future aspirations Evaluate different types of Higher Education Institution in terms of personal interests and career aspirations Compare degree courses and study options across a range of universities Engage effectively with the UCAS process and generate and submit a strong university application Compare student finance, budgeting support and student employment opportunities across a range of universities

2015 Year 12 Summer School Aim 3: Develop students' confidence and resilience to negotiate the challenge of university life Enable students to: Anticipate challenges they will face in Higher Education and make a successful transition to university Gain a positive first-hand experience of student life and a university environment Become familiar with learning and teaching approaches in Higher Education Engage with academic and personal support mechanisms at the University of Bath Interact with academic staff and other university employees Interact with other students on programme, Student Ambassadors and current University of Bath students. Participate in challenging educational activities which are stimulating and motivating Celebrate in the achievements of students from similar backgrounds who have progressed to the University of Bath Access the information, advice and guidance they need to make a successful transition to the University of Bath or other selective university

2015 Year 12 Summer School Aim 4: Develop students' confidence and resilience to negotiate the challenge of university life Enable students to: Enhance their academic skills through collaborative projects that develop capacity for critical thinking, independent research and self-directed learning Enhance their capacity for independent learning, self-directed study and enterprise Enhance capacity for creative problem solving and decision making Enhance communication and presentation skills using different mediums. Enhance critical thinking skills through experimentation, reflection, analysis, synthesis and evaluation Enhance group work skills and capacity to lead and work collaboratively Enhance revision skills and expertise in a range of revision techniques

2015 Year 12 Summer School Aim 5: Develop students' understanding by contextualising subject knowledge Enable students to: (a)Situate their existing knowledge within wider fields of knowledge and apply to other contexts Extend awareness of the wider applications of knowledge Locate existing knowledge within wider fields of knowledge and other contexts Enhance understanding through collaborative projects, which extend knowledge and challenge assumptions (a)Situate existing knowledge and interests within the context of university degree programmes and academic disciplines Relate existing knowledge and interests to university subject areas and degree programmes Understand how knowledge can be developed within university subject areas and academic disciplines