Presentation on theme: "An analysis of evaluation activities planned in NN/LM outreach subcontracts 2001-2006 Mellanye Lackey 2 nd year Associate Fellow National Library of Medicine."— Presentation transcript:
An analysis of evaluation activities planned in NN/LM outreach subcontracts 2001-2006 Mellanye Lackey 2 nd year Associate Fellow National Library of Medicine UNC-Chapel Hill Health Sciences Library firstname.lastname@example.org
2 What I’ll talk about today Gaining background knowledge Literature review and conversations w. experts Barriers Project data Collection methods Analysis Results and graphs Opportunities to learn more
3 Barriers to doing outreach evaluation Prior experiences usefulness motive fear negative results Getting buy-in relationships cooperation from partners support from authority appreciation of evaluation Lack of time money knowledge about evaluation processes - Taut, S. M., & Alkin, M. C. (2003). Program staff perceptions of barriers to evaluation implementation. American Journal of Evaluation. 24, 213-226.
6 Collecting project data Projects included Public Health Information Model Access to Electronic Health Information Subcontract Consumer Health Initiative Subcontract Resource Library Outreach Subcontract Health Professional Information Outreach
7 Project data Funding from $2,000 to $70,000 reporting evaluation plan Reviewed >500 records (1/2 of all NN/LM projects) n = 144
8 Data collection Methods Questionnaire Tally/Log Pre/Post data Interview/Anecdote Evaluation form Focus group Observation/Usability study Approaches Evidence for future use Internal committee Needs assessment Social capital Case study
13 Limits of the data Records created at project inception Data sometimes rich, more often brief How many participants? Focus groups? How much time? Terms vague (eg. evaluation form) Growing emphasis on evaluation with Outreach Evaluation Resource Center (OERC)
14 Further research ideas Compare evaluation planned with final report Get more specifics about each plan Examine formative vs. summative Collect information about “approaches” How get more data?
15 Conclusions At all funding levels: Intent to evaluate Variety of methods Opportunities to learn more Bibliography of resources http://www.hsl.unc.edu/AboutLib/Staff/LiaisonWebPages/PHliaison.cfm Outreach Evaluation Resource Center http://nnlm.gov/evaluation/index.html