1. 4 th November 2005 JISC VRE Advisory Board Meeting Formative evaluation workshop: initial feedback and discussion John Kelleher and Kerstin Junge Tavistock Institute, London

2. Formative Evaluation of JISC VRE Nov. 2005 Introduction This workshop Presentation and discussion a.Brief introduction to the Tavistock and its ways of working (10 to 15 minutes) b.The formative evaluation approach and progress to date (15/20 ‘’ ) c.Initial findings: some characteristics of the programme as implemented (20/25 ‘’ )

3. Formative Evaluation of JISC VRE Nov. 2005 Introduction This workshop Workshop d.Emergent issues (20/25 minutes) e.Evaluation concerns and foci (20/25 ‘’ ) f.Implications and consequences (20/25 ‘’ )

4. Formative Evaluation of JISC VRE Nov. 2005 The Tavistock Institute a. The Tavistock and its ways of working Research and consultancy organisation ‘supporting and learning from change and innovation’ Bias towards Action Research Formative and other evaluations of eLib and several other JISC programmes and activities

5. Formative Evaluation of JISC VRE Nov. 2005 Formative evaluation approach and progress to date b. Formative evaluation Learning from innovation Timely ongoing feedback to support steering and development Part of the Action Research cycle (plan – act – evaluate - plan - …)

6. Formative Evaluation of JISC VRE Nov. 2005 Formative evaluation approach and progress to date This formative evaluation Phase I Documentary and literature review Key interviews Project visits (5 so far, completion by late Nov.) Policy map (in progress) Interim report (December, JCSR Feb.)

7. Formative Evaluation of JISC VRE Nov. 2005 Formative evaluation approach and progress to date This formative evaluation Phase II Planning Project and programme support: individual projects; sets of projects; all projects; programme User needs, self evaluation, institutional embedding, collaboration, …

8. Formative Evaluation of JISC VRE Nov. 2005 Initial findings c. Some characteristics of the programme Project types Technically orientated – user orientated Development – research Multiple institutions – single institutions Arts & humanities – natural sciences Single discipline – multiple disciplines Users imagined – users investigated

9. Formative Evaluation of JISC VRE Nov. 2005 Initial findings Some characteristics of the programme Orientations and approaches Tools more than technologies Implicit rather than explicit user models Institutional settings assumed rather than addressed ‘Revolutionaries’ versus ‘evolutionists’ ‘True believers’ (in VREs) v ‘agnostics’ (believers in e-research)

10. Formative Evaluation of JISC VRE Nov. 2005 Initial findings Some characteristics of the programme Memberships Presence of leading edge users in teams Absence of institutional decision makers in projects Sparse organisational and programme design and change expertise

11. Formative Evaluation of JISC VRE Nov. 2005 Initial findings Some characteristics of the programme Initial project issues arising User engagement Environmental embedding Intra-programme exchange and learning Exit and exploitation

12. Formative Evaluation of JISC VRE Nov. 2005 Emergent issues d. What do WE mean by … ‘VRE’? ‘Users’? ‘Embedding’? ‘Products’? ‘Learning’? ‘Policy’?

13. Formative Evaluation of JISC VRE Nov. 2005 Evaluation concerns and foci e. Evaluation concerns and foci? Social, organisational and institutional assumptions Strategies of engagement … of users, of institutional decision makers Emergent agendas for action

14. Formative Evaluation of JISC VRE Nov. 2005 Implications and consequences f. Implications and consequences? For the formative evaluation design … For the programme (VRE1) … For the next programme (VRE2) …