Choosing Questions and Planning the Evaluation

What do we mean by choosing questions? Evaluation questions are the questions your evaluation is meant to answer about your work. Evaluation questions help set the direction of the work, as well as assess its effectiveness. Ideally, choosing evaluation questions is part of the planning of the overall program.

Questions to ask yourself as you choose: What do you want to know? Why are you interested? Is the issue you’re addressing important to the community or to society? How does the issue relate to the field? Is the issue general, rather than specific to your population or community? Who might use the results of your evaluation? Whose issue is it?

Why is it necessary to choose evaluation questions carefully? It helps you understand what effects different parts of your effort are having. It helps you clearly define what it is you’re trying to do. It shows you where you need to make changes. It highlights unintended consequences. It guides your future choices. In participant evaluations, it involves stakeholders in setting the course of the program, thus making it more likely that it meets community needs. It provides focus for the evaluation and the program. It determines what needs to be recorded in order to gather data for evaluation.

When should you choose questions and plan the evaluation? If possible, choosing questions and planning the evaluation should be an integral part of planning your program. If your reality makes that impossible, choosing questions and planning the evaluation should take place as soon as possible after the program starts.

Who should be involved in the process? To the extent possible, the process should involve all stakeholders, including program participants/beneficiaries.

How do you choose questions and plan the evaluation? Choosing questions: – Describe the issue or problem you’re addressing. – Describe the importance of the problem. – Describe those who contribute to the problem. – Assess the importance and feasibility of changing those behaviors. – Describe the change objective. – Make sure that the expected changes would constitute a solution or substantial contribution to the problem.

Planning: – Take into account the issues raised by multiple or very different settings. – Take into account the issues raised by participant groups that differ in culture, ability to complete the program, geographical location, and other factors.

For outside evaluators, specifically: – Choose a setting. – Learn as much as you can about the organization you’ve chosen. – Contact the appropriate person(s) and request an interview. – Plan and prepare for the initial meeting.

For all evaluators: – Find out all you can about the context. – Establish trust with program administrators, staff, and participants. – Aim for a participatory evaluation. – Plan the evaluation, in collaboration with stakeholders.

Consider in your planning all the elements of an evaluation: – Information gathering and synthesis. – Designing an observational system. – Developing and testing a prototype intervention. – Selecting an appropriate experimental design. – Collecting and analyzing data. – Gathering and interpreting ethnographic information. – Collecting and using archival data. – Encouraging participation throughout the research. – Refining the intervention based on the evaluation – Preparing the evaluation results for dissemination.