ArtFULL – finding and using evidence of learning Centre for Education and Industry University of Warwick

Learning Outcomes for Pupils (The DfES criteria) fulfilment and satisfaction from achievement increased learning within the subject area increased understanding of connections between subjects increased learning across subjects increased self-confidence and self-esteem increased cultural understanding and respect and tolerance of others increased ability to work with others increased involvement in class, school community events the ability to make informed choices within and beyond the planned experiences positive attitudes to the experience and a desire for further experiences

DfES Objectives for teachers increased confidence increased expertise increased professional satisfaction DfES Access Objective Ensure that more pupils and, where relevant, teachers, benefit from access to objects and images from museums and galleries during the school day

Scale of MGEP2 130 projects – 16 galleries 165 museums and galleries involved Around 20, 000 pupils participating – 880 in galleries £1 million 9 regional museum, library and archive agencies plus engage Timeframe: October 2002 to March 2004

Methodology Initial Project Description, Quarterly & Final Report from 130 projects Project Portfolio: pupil responses and resources (61) Simple exit questionnaire for pupils (4093 & 1436) (472 in galleries) Selected entry questionnaires (2745) (235 in galleries) Exit questionnaire for teachers, museum staff, assistants (536) (59 in galleries) Observations(71), group interviews (59), teacher and staff interviews as appropriate for 51 projects (85) (7 of which were galleries) Entry/Exit Interviews with regional co-ordinators, observation of training and networking sessions (129)

Evidence of Learning Teacher assessment of student work: quality of work, before & after Comparison with non-project classes: teacher and pupil judgments Observation of activity: response to different learning styles, independence New concepts, skills, knowledge: greater complexity/challenge Observation and recording of participation: inclusion, concentration Developmental work: progress, group self-evaluation Pupil work or response: how personal is response, how complete? Notes, sketchbooks, drafts: creative and original work, development Response or requests for further opportunities: behaviour, decisions Pupils’ judgement and feedback: attitudes, satisfaction

What and how much was learned? How did they learn it? Who and how many learned it?

Analysis learning gain objectives and outcomes associations between outcomes galleries vs. museums character of gallery learning explanation of sustainability

There was substantial aggregate impact on subject learning –64% of entry/exit respondents experienced “learning gain” –70% of teachers judged that pupils had learnt a lot about the specified topic Chart 3.31: Percentages of respondents reporting the extent of learning about “subject” at exit grouped in terms of extent of knowledge at entry. (n=2617). (A1*B4 Cross-tabulation)

The Relative Impact of a Gallery Education Programme Base: MGEP2 – 2923, ArtFULL – 235/379

Activities were enjoyed –61% of all pupils enjoyed activity very much (65% for ArtFULL) –73% of those who reported highest learning also reported highest enjoyment Working with adults –Overall pupils judged relationships with other pupils and adults on projects positively but similar to normal –Teachers judged as more positive –Some groups of pupils reported relative progress –Projects involving artists/performers working intensively with pupils were relatively successful at improving experience of “learning with adults” – 59% of these projects showed a learning gain in this area as against 51% for all projects.

Fulfilment –91% of all pupils were either very pleased or content with the work they did themselves –Across MGEP2 fulfilment levels did not show significant value added –Some identified value added for those least satisfied with their own work at school Chart 3.53 Percentages of respondents expressing satisfaction with their own work at exit grouped in terms of how satisfied they were with their own school work at entry (n=2630)

Impact of Programmes on Fulfilment Base: MGEP2 – 2630, ArtFULL – 235/379

Confidence –58% of pupils reported a gain in confidence (67% for ArtFULL) –77% teachers reported some or general gain for pupils –Some gains in confidence for least confident Chart 3.42: Percentages of respondents reporting changes in confidence at exit grouped in terms of how confident they were at entry (n=2609)

Choices –Experience of choice relates to satisfaction Chart Percentages of Key Stage 2/3/4 pupils expressing satisfaction with their own work in the project grouped in terms of their experiences of making choices on their project at exit (n=3755) (B3 * B11 Cross-tabulation)

Opportunities for Making Choices Base: MGEP2 – 2609/3937, ArtFULL – 235/379

Outcomes for Teachers 94% of teachers reported a gain of satisfaction 84% of teachers reported an increase in expertise in using this kind of project Teachers contributed to design and development of projects Teachers valued collaboration with other teachers and other professionals Teachers valued projects which made curriculum more relevant for their pupils (and for them) No general difference between museum and gallery projects Rare for projects to have wider impact on schools

Characteristics of Museum and Gallery Learning Situated learning/immersion in context Distinctive social context for learning Use of performance Enhanced opportunities for creativity Relatively long blocks of time and an extended treatment of a particular topic Cross curricular learning Learning in special places Object handling and interpretation

Sustainability –Meeting a curriculum need –Existence of ready schools market –Cost and future funding –Charging –Flexibility of project –Professional development –Embedding in school curriculum –Quality of marketing

Issues for Evaluators –Cost and time to collect evidence –Reconciliation of project and programme evaluation –Involvement of schools – use of assessment data –Usefulness of evaluation – scope for more informed discussion about what various stakeholders want to know from evaluation and therefore about what kinds of evaluation should be gathered and analysed –Collaborative evaluation