1. Effective dissemination and evaluation
Workshop on Teaching and Learning Grants
and Carrick Schemes, 2008
Merrilyn Goos and Clair Hughes

2. Session overview
A project scenario
Dissemination and evaluation defined
Purposes of dissemination and evaluation
Small group development of dissemination and evaluation strategies
Resources

3. A project scenario
“Introduction to University Teaching and Learning” program: aims, objectives, participants, teaching team
New course structure and content from 2008
How to disseminate this innovation - not just to inform potential participants and other stake-holders but to embed good practice?
How to evaluate the course in its first offering in order to monitor and improve its implementation?

4. Dissemination defined
Dissemination is more than distribution of information or making it available in some way. Dissemination also requires that some action has been taken to embed and upscale the innovation within its own context and to replicate or transform the innovation in a new context and to embed the innovation in the new context.
Carrick Institute Dissemination Framework

5. Evaluation defined
Evaluation is the process of determining the merit and/or worth of things for a range of purposes.
Consider funding agency requirements for evaluation:
Ensure that evaluation is undertaken at multiple points throughout the project and is understood and reported within an evaluation framework.
Carrick Competitive Grants program: Guidelines and Supporting Information – 2008 (p.23)
Must provide a strategy for the evaluation of the project with information on outcomes for students, whether there has been a noticeable improvement or not in the expected outcomes for students and how the project outcomes will be disseminated to others within the discipline.
UQ Teaching and Learning Small and Large Grants Scheme Guidelines

6. Purposes of project dissemination
Dissemination for awareness
(scattering the seed to the wind)
Information provision
Distributing information through reports, publishing papers, conference presentations, sending s, developing websites.
One way information process.
Dissemination for understanding
(sowing the seed in prepared soil)
Distributing information in a more purposeful manner, e.g. workshops, visits.
Requires interaction with others who are involved in the project.
Dissemination for action
(propagating, breeding, growing, grafting)
Engaged dissemination
Purposeful, directed, systematic, proactive engagement, evidence based, involves adaptation and implementation, resulting in changes in practice.

7. Purposes of project evaluation
Formative
(to inform ongoing project decision-making)
Monitor project progress
Revise/enhance project processes and outcomes
Engage key stakeholders early and during project to:
demonstrate respect
intensify participation
collaborate in identifying key indicators of success
progressively expose to new perspectives, or information
Make interim reports
Summative
(to determine/support project findings and outcomes)
Report on success/results/value
Demonstrate impact
Account to sponsors
Sustain project outcomes
Support related submissions (eg grants for funding for further related projects; promotion; teaching and learning awards)
Enlightenment
(to learn)
Provide opportunities for project team to maximise the learning developed through participation in the project
Adapted from “The Learning Partnership” and W. K. Kellogg Foundation

8. Discussion In groups of 3 or 4:
address the following dissemination and evaluation questions in the context of the project scenario presented earlier (new “Introduction to University Teaching and Learning” program)
Consider how you would address dissemination and evaluation for a project in your own educational context
Share your responses with the larger group

9. Dissemination questions
Who are the users of and other stakeholders in this program?
What do we want to disseminate (process, ideas, products)?
How could we make users and other stakeholders aware and involved?
How could we engage users and other stakeholders so they use and/or support the program?
How could we obtain feedback from users during the project?
How will we know there has been change in practice?

10. Evaluation questions Who is our project audience (stakeholders)?
Select two stakeholders/groups and list one or more questions each may have in relation to this project.
How could we collect data to answer these questions? (methods)
From whom or where (stakeholders and others) could data be collected? (sources)
What will count as evidence? (consider both qualitative and quantitative)

11. The evaluation framework
EXAMPLE
Evaluation questions
Sources/Methods
Institutional contacts
Expert reference group
Existing records/
artefacts
What is Generic Graduate Attribute (GGA) policy in Australian universities?
Telephone interview (to clarify, address omissions or current activity)
Web search of university policy documents
What are current barriers to effective GGA implementation?
Focus groups to elicit responses to issues paper based on literature review
Written responses to draft issues paper
Literature review (research into implementation experience)
Analysis of AUQA reports

12. Models of planning for evaluation
Implementation
Evaluation
Implementation
Evaluation
Dissemination
Dissemination
Evaluation
Implementation
Implementation
Evaluation
Dissemination
Dissemination

13. References
Carrick Institute for Learning and Teaching in Higher Education. (2006). Carrick Institute Dissemination Framework Retrieved from
Carrick Institute for Learning and Teaching in Higher Education. (2008). Competitive Grants program: Guidelines and Supporting Information – Retrieved from
Evaluating projects resource  A resource to assist applicants planning the evaluation framework component of their Grants Scheme application has been prepared by Murdoch University <
Felton, P., Kalish, A., Pingree, A., & Plank, K. (2007). Towards a scholarship of teaching and learning in educational development. In D. Robertson & L. Nilson (Eds.), To Improve the Academy: Resources for Faculty, Instructional, and Organisational Development (pp ). Boston: Anker.
The Learning Partnership. (2007). Material distributed during Leading and Managing Projects, a workshop sponsored by the Carrick Institute for Learning and Teaching in Higher Education. Melbourne, 17 – 18 September.
TEDI Evaluation Services
The University of Queensland. (2007). UQ Teaching and Learning Small and Large Grants Scheme Guidelines. Retrieved 29 October, 2007, from
W. K. Kellogg Foundation. Executive summary: Introducing Program Logic Models. Material distributed during The Professional Development Program for Medical and Healthcare Educators offered jointly by the University of Queensland and Harvard Medical International August, November, 2006.