What’s the best approach? Comes down to: ›What do we want to do? ›How much time have we got? ›Who have we got? ›What resources do we have? ›Who is our target audience? ›Best imaginable outcome for results?
Some techniques ›Interviews ›Observations ›Focus Groups ›Small questionnaires with crisp, manageable questions ›Review of documents, records, media ›Limited archival research ›Performance based methods -Interactive role playing ›Follow up (cohort) surveys to map changing decisions/perceptions ›Participant led data gathering
Pros and cons ›Focus group -Economical but risk getting dominant ‘group think’ ›Observations -less intrusive but takes longer saturation to get to patterns ›Interview format/interactive performance based -flexible and rich but time intensive ›Survey -larger potential population -useful for longitudinal or cohort analyses -not ideal for understanding detail or social complexity -Also dealing with questionnaire-fatigued target groups 5
Sampling: size versus depth ›If want larger sample, must tame the monster: ask mainly yes/no questions; have only one open-ended question ›If want detail, limit the sample ›Most reliable method for young people is face to face interviews and participatory research ›In any case, design data analysis before you administer instrument of choice -Clarify what data you actually want and ask for it, not what you think should be asked -Involve target group in the revisions -Anticipate and design the data analysis tools (the database and coding systems) before going live
Some protocols ›Research shows that poorly worded questionnaires bore young people, or provokes conventional answers – needs to be attractive and easily interpreted ›Also important to avoid having it be another piece of school work ›True consent not possible in power structures ›Thus even more essential instruments are well designed -Conduct a small trial: test the questions – do people respond as you had hoped? Ask young people how they found it. Gauge reactions and modify -Limit time required to 15 minutes or less -Be aware of biased terms – e.g. ‘assistance to the poor’ (positive) versus ‘welfare’ (negative) -Keep it uncluttered – less is more -Think about the order – make it a logical progression, not choppy 7
What are the right questions? ›What is the age group? Gender? Social background? ›What data story do you need to be able to tell and with what ambition (For further funding? Publications?) -Can’t over-state need for clarity about purpose ›Vignettes of transformational stories are excellent -Write with journalistic flair ›Combine these with other impact data -The more ‘factual’ and repeatable over time, the better 8
Youth oriented evaluations that yield convincing data ›Given reality of need to maximise responses in shortest time frame and at least cost, how to get the holy grail of youth-centred evaluations that yield convincing data? ›Many interventions presume a deficit (in esteem, in awareness, in comfort) and rush to fill that, without any primary research ›Foregrounding the students and making the evaluations as student- centred as the interventions vital especially given academic persona as another teacher/adult ›Be creative 9
Widening Participation Are we making an impact? How known? What is meant by/needed for sustainability? Is it worth expanding/scaling? What do we not want to be doing more of?
Brainstorming time ›Group discussion of project status in relation to the timelines and matching methods to questions to desired outcomes ›Contact details: firstname.lastname@example.org; 9351 email@example.com