Presentation on theme: "Quality & Impact in DE defined: a selection"— Presentation transcript:
1Quality & Impact in DE defined: a selection An overview of selected informantsHarm-Jan FrickeOctober 2011
2DARE Forum so far (May 2011) Q&I of DE depends particularly on Quality of organisation and management of workQuality of ways of working with audiences, partners, beneficiariesSpecific to DE are issues of:Coherence:between values and actions, between actions and relevance to the own organisation and the audience/beneficiaries/partnersPartnerships:explicitly defined and operatedParticipation:In methods used and in participants defining work donePerspectives:Plurality of viewpoints
3Johannes Krause: Global Education quality standards Project/ programme designbased on quality standardsMonitoring& evaluation(a) M&E instruments, (b) M&E practiceOfficial policies:quality standards translated into grant guidelines, funding criteriaOrganisational Quality ManagementSharing good practices/ lessons learnt within the org. & with networks; peer reviews (e.g. Norway), etc.Work on quality frameworks(a) in the organisation, (b) in networks, (c) for policy level
4Irish DEA: Quality and Measuring the Impact of DE Attention to the context in which work takes place: the relationship between the ‘development agenda’ and the ‘learning agenda’Distinguish between process and impact evaluation: show progress against defined milestones or progress markersSuggested criteria and indicators at output level:# of schools/groups/individuals participating in a programme/course/event# of sessions/meetings held% of participants who complete a programme# of publications/resources/webpages produced# of visits to a website# of posts to a blog, facebook page etc.# of people who join an organisationCriteria and indicators at outcome level:% of participants who take an action/ join relevant activist groups or campaigns% of participants who go on to further study/training in the area% of participants who change their consumer behaviour and/or lifestyle choicesProgress towards ‘markers’ of desired behavioural changes
5Irish DEA: example of Q&I at national levels Output criteriaIndicatorCooperation with DES is enhancedNumber of meetings with the DESNumber of times DE is mentioned at the cross-departmental working groupEducational policies contain DE messagesNumber of educational policies containing dev ed messagesQuality ResearchNumber of research articles on DEOutcome criteriaIndicatorCooperation with DES is enhanced(Enhancement is defined as increased awareness and buy-in from people at senior management level and increased commitment to integrate messages)Senior members of DES attend DE events and are open to conversations and dialogueDE becomes visible in DES communications and policiesEducational policies contain DE messagesQuality and depth of dev ed messages contained in policiesQuality ResearchIndicators for research should follow academic standards (peer- reviews, quality of publications, etc.) but also bear in mind that relevance for practice is an important indicator of success for quality DE research.
6ACODEV – COPROGRAM: Quality in Development Education Looking for quality in DE means the following:awareness of the relationship DE action - context, culture, reality, power relationships (society, north-south relationships, institution, funding,…)To act in coherence with our values.Quality on 2 main levels,Level 1 : the 3 main groups of actors involved in a DE action : participants, educators (and the institution to which they belong) and partnersLevel 2 : the 3 main phases in the setting up and development of any DE action: action design, action process or processes, results and outcomes
8Teles and Schmitt: Elusive craft of evaluating advocacy “Successful advocacy efforts are characterized not by their ability to proceed along a predefined track, but by their ability to adapt to changing circumstances”Attribution of effect and of effectiveness is difficultTherefore:Assess the (policy-social) context within which interventions take placeUse a "spread betting" approach to advocacy, and thus to evaluation since success or pay-off is rarely due to any one project/intervention aloneAdopt a Long Time HorizonPay Attention to Policies’ Impact on Politics, Not Just on Target PopulationsEvaluate organisations as a whole, not the single project
9DESS 2011: Q&I in advocacy, campaigns and events Clarify concepts and definitions:Advocacy v. CampaignsAdvocacy v. Awareness RaisingAdvocacy/Campaigning v. EducationQ&I shown through:Strategic approach to do with for instance:Choice of topic/theme/issue;Problem analysisStakeholder analysisDefinition of objectivesCritical path/Intended change processParticipation including to do with:VisionValuesReflectionTheory of change made explicit
10DESS 2011: Q&I in adult education Criteria:Participation of audience/target group and relevance of issues to learners’ needsMethodology to promote engagement, motivation, critical reflectionEngagement as outcome: with skills useful in various situations
11DESS 2011: Q&I in youth work Issues to address include: Plan for impactAddress relevanceShow coherence with visionBe based on explicit valuesUse participatory approachShow different perspectivesStimulate reflection (on what is being gained)Be sensitive to gender and other power relations
12DESS 2011: Q&I in resourcesDoes it engage the user in questioning, discussion and action?Is it relevant and educationalDoes it show different perspectives on the issue – including minority and southernDoes the resource enforce/reinforce stereotypes?Is it relevant to the target groupIs it accessible to the userIs the information accurate and up-to-date
13OECD-DAC & EC: evaluation framework for projects RelevanceEffectivenessEfficiencySustainabilityImpactDG DevCo: the above plusCoherenceAdded value
14Next steps for the Q&I working group? Given the foregoing andgiven the DEEEP objective (a pan-European quality framework on Development Education):What can the Q&I group do next?What is needed?What is useful?