Presentation is loading. Please wait.

Presentation is loading. Please wait.

RACcER: Re-use and Archiving of Complex Community Based Evaluation Research Irish Qualitative Data Archive Tallaght West Childhood Development Initiative.

Similar presentations


Presentation on theme: "RACcER: Re-use and Archiving of Complex Community Based Evaluation Research Irish Qualitative Data Archive Tallaght West Childhood Development Initiative."— Presentation transcript:

1 RACcER: Re-use and Archiving of Complex Community Based Evaluation Research Irish Qualitative Data Archive Tallaght West Childhood Development Initiative

2 Irish Qualitative Data Archive Funded by the Irish Government under the Programme for Research in Third Level Institutions (Cycle 4) Several major datasets being archived, including – Growing up in Ireland study Life Histories and Social Change in Ireland in 20th Century Ireland But... Limited awareness/understanding of qualitative archiving in Irish research community Lack of literature about challenges of archiving qualitative data generated in evaluation research Re-use & Archiving of Complex Community-based Evaluation Research (RACcER) project explores issues specific to archiving community-based evaluation data, also aims to raise awareness throughout research community

3 RACcER RACcER co-funded by IRCHSS and CDI: demonstrator project for qualitative archiving Objectives: Scope concerns, issues and requirements in qualitative data archiving through interviews Explore issues and challenges for researchers, participants, funders, potential data users Test feasibility of archiving evaluation data Develop protocols and procedures for archiving evaluation data Research Interviews with 30+ individuals from policy and research community, including those involved in TWCDI evaluation Evaluation researchers were all university based

4 Tallaght West Childhood Development Initiative (CDI) 1 of 3 Prevention and Early Intervention Programmes jointly funded by government & philanthropy Aims to improve the education and well-being outcomes for children and families in historically disadvantaged areas of Tallaght West 5 constituent projects: early years, literacy, pro social behaviour, healthy schools, community safety Major emphasis on evaluation Mainly quantitative approach to evaluation of RCT interventions, but qualitative process data collected for all constituent projects and overall evaluation

5 Findings – qualitative data gathered in evaluations Range of qualitative material collected One-to-one interviews, discussion groups, meeting minutes, site observations, project log books Variety of participant groups Service providers, practitioners, school, police, health service personnel, service users (children and parents), community reps Most material recorded, anonymised and transcribed

6 Findings – archiving policy Evaluators research centres didnt generally have archiving policies for qualitative material Range of experience of what happens to qualitative data after project Destroyed; stored privately but not used; returned to commissioner; archived internally No evidence of researchers sharing datasets informally Default ethics position is to destroy after 5 years In some cases 10 years

7 Findings – archiving experience No evidence of formal archiving Limited experience of accessing qualitative archives, One example for use in teaching Some had archived quantitative material Some had accessed quantitative archives Range of views on resource implications of prepare qualitative material for archiving

8 Findings – issues with archiving Difficulties of anonymising in small community Level of anonymisation would reduce re-use value Perception of limited reuse value of material collected for evaluation purposes Secondary analysts wouldnt understand the nuances of evaluation context Whats the added value of raw material vs. final report? Lack of awareness/understanding/trust in newly established archive Compromises fully informed consent Viability of archive, level of resources queried Reluctance to hand over data to be governed by unknown institution

9 Findings – issues(2) Consent not sought Unethical to archive without consent Retrospective consent not feasible Ethically dubious to archive documents like meeting minutes, no consent Constraints of ethics committees limit archiving Loss of control means consent cant be fully informed Prospect of archiving reducing quality of the research a major concern Reduce willingness to participate in research Constrain what participants willing to share There wasnt a hope in hell of anyone speaking to us unless there was anonymity. Archiving seen to equate with loss of anonymity

10 Findings – issues(3) Wide perception of fundamental difference between evaluation research and broader topics Necessarily political At its core it is a judgement on the management and delivery of services Interview concerns 3 rd parties, consent not sought of them Services, jobs, funding are in question, highly sensitive Usually geographically bounded, problems with anonymisation

11 Findings – issues(4) Lack of clarity on who can access data and how it could be used generates fear and reduces inclination to archive Question of what constitutes a bona fide researcher? – How would bona fides be established? What assurances? Issue of policy researchers & communities eligibility to access Concern over data being used out of context, twisted, sensationalising populations, journalists gaining access Archiving seen as a breach of trust between the researcher and researched

12 Findings – issues(5) Researchers concerned about being opened up to scrutiny Sense of archiving being used to open up research to scrutiny makes researchers feel they arent trusted Potential for commissioners/others to access the archive to undermine research when they dont like what it has said Influence of that on evaluation research process And if theyre going to check my work I might as well write what they want to hear. PhD/masters students using archived material could reduce valuable learning opportunities gained from fieldwork Resource implication of archiving Benefits not worth gain if funders arent supporting it

13 Findings – advantages Potential to explore alternative research questions Macro level implementation questions across PEIP sites Language used by different groups in schools Impact of setting on parents willingness to disclose information Intangible factors in project success such as charisma Contributing to cumulative knowledge of implementation of childrens services Facilitate comparative analysis over time or place Although some qual researchers dispute extent to which comparison is appropriate

14 Findings – advantages(2) A valuable teaching resource But disputed by some as being unethical Increase the standing of qualitative research if it is open to external examination, improved validity and reliability But concerns about how it could be used in this way Contribute to corporate memory of commissioning organisation But is this the purpose of archiving?

15 Findings - risks Reputational risk for organisations involved at all levels Particularly if initiatives havent been successful Equally if commissioner is unhappy with research findings Negative publicity for researched community User agreement could be breached Anonymity could be compromised – Longer term implications for researchers accessing certain communities But... Need to consider likelihood of risk occurring and how serious it would be if it happened

16 Findings – mitigating risks Strict controls on user access Ethics approval Sign-off of original researcher on publications Archive to facilitate collaboration between researchers As an explicit function of the archive, builds trust Hierarchy of access, tailored depositor and user agreements Some accept that time delay on access reduces risk Others say time doesnt make a difference, especially in with research on children Time delays reduce reuse-ability in policy context Sunset clause may also be useful in some cases

17 Other project issues Data ownership issue in evaluation context was very sensitive Massive underlying issue in this project – part of the genesis of RACcER Commissioners claim legal ownership, evaluation teams dispute this, some suggest participants have moral ownership Underestimated importance of anonymity to research participants Implications for archiving & reporting–anonymisation burden, higher level unit of analysis Some participants refused archiving

18 Anticipated project outputs Report on RACcER research Development and dissemination of best practice guidance Response to concerns identified in research Workshop with research participants Raise awareness about archive and provide support Archiving of CDI internal material Archiving of RACcER interviews

19 RACcER Team Irish Qualitative Data Archive Jane Gray Julius Komolafe Hazel OByrne Aileen OCarroll Tallaght West CDI Tara Murphy


Download ppt "RACcER: Re-use and Archiving of Complex Community Based Evaluation Research Irish Qualitative Data Archive Tallaght West Childhood Development Initiative."

Similar presentations


Ads by Google