1. What works for Evaluators of Management Education ? Elaine Clark

2. Action Learning for Senior Leaders in the NHS in London Commissioned by NHS London, SHA Aims: Provide support through individual/organizational transition. Facilitate building of personal resourcefulness though creation of a supportive network. Enhance participant’s passion for improving services for patients through learning and reflection. Enhance resilience.

3. Challenges in evaluation Identifying the meaning…illuminating impact What does this mean? Love conquers all...!

4. Or…? Finding surprises…

5. Evaluative challenges

6. Accountability… Commissioner? Participants? Research community? Self?

7. Need for complementarity - relationships Between… p AND q Quantitative (positivistic) and qualitative/interpretative Commissioners and participants

8. Searching for reconciliation

9. Practical implications Causality – mutual, not linear – Causality may not be temporally precedent, but simultaneous or antecedent – Emphasis on relationships, the networks – Need for multiple perspectives - multi-stakeholder interviews, with as many different perspectives as possible Need to ‘leave space’ in design for highly exploratory stage. Uncover unexpected/unintended impact. Complementarity of research methodology – comparative data against a ‘control group’; interviews and participative inquiry.

10. Participation, regular and critical: Stakeholders kept informed as evaluation proceeds, encouraged to challenge, dispute, add to findings. Brings to the fore, tensions, emotions and politics. Focus upon relationships ; Internal and External Reflexive/questioning approach as an evaluator : Are we doing better things as well as doing things better (Burgoyne, 2010) this requires action as a consequence

11. Some findings… Importance of relationships: internal and external Importance of P, as well as Q Growing awareness of participants of politics and emotions within the health care environment of London Enhanced resilience Awareness of importance of, and courage to ask, questions

12. Some findings… “Given me the ability to gain a broader view of organisations outside my normal sphere and given me confidence through a greater knowledge of non exec roles within the organisation.” “Programme has provided opportunity to better understand current issues affecting NHS providers from an operational perspective.” “There was a lot about organisations coming together, with widely different systems, cultures and legacies and starting to understand those different perspectives.”

13. “ Have gained greater awareness of what other people are doing. Also, opportunity for networking and (gaining) general awareness of how other people are impacted on by the system and how they operate within it helps you to reflect on your own practice and your own organization. A practical example is that I am reviewing some of our board papers on the back of it.” “What the programme did was to help me to understand some of the sensitivities around NHS London”

14. What was carried forward? NHS London became Clinical commissioning groups Importance of knowledge of, and ability to build, relationships, networks Enhanced self-belief Enhanced ability to question and therefore to learn – generative learning? Knowledge about the system – the P, but also the politics and sensitivities With a system change like this, is the learning more likely to be carried forward or less?

15. A suggestion… From sociological paradigms to … Action learning as a ‘sociology of freedom’?

16. References Burgoyne, J. G. (2009) Issues in action learning: A critical realist interpretation Action Learning: Research and Practice 6, 2: 12 Burgoyne, J. G. (2010) Evaluating action learning: a critical realist complex network theory approach Action Learning: Research and Practice 7,3: 239-251 Burrell, G and Morgan, G (1979) Sociological Paradigms and Organizational Analysis, London and Exeter: NH. Heinemann Parlett, M; Hamilton, D (1972) Evaluation as illumination: a new approach to the study of innovatory programs Occasional paper: Edinburgh University Available at ed.gov, accessed May 2 nd 2014