© e-ScienceTalk Consortium e-ScienceTalk: Measuring Impact 1 September 2010 – 31 May 2013 Zara Qadir e-ScienceTalk, Dissemination Officer

© e-ScienceTalk Consortium Reaching the audiences Policy makers Business General public Scientists Research community Developers Delegates Scientists Policy makers Funding agencies Journalists

What is impact? Reach Who are our users? Quantitative e.g. Counting subscribers or tracking visitors with Google Analytics. Significance How are people engaging with content? Quantitative e.g. comments, shares How are we helping the e-science community? Qualitative e.g. interviews, open-ended surveys

© e-ScienceTalk Consortium iSGTW reach.. Number of followers Page views/unique visitors Time on site/page Number of subscribers

© e-ScienceTalk Consortium Plan Formative Evaluation Measurement Plan Review Revise/ Improve Measure outcomes Review Revise/ Improve Measure outcomes Do Continually Monitor Outcomes Assess (Metrics) Outputs Quality IMPACT CYCLE Gather feedback at all stages... 5

© e-ScienceTalk Consortium Your toolkit includes.. Quantitative (Reach) Deliverables/Milestones Online surveys Social networks Google analytics Social engagement Webometrics Usability testing Online Klout Scores Qualitative (Significance) Online surveys Interviews Focus groups Unsolicited/personal s Talking to people 6

© e-ScienceTalk Consortium 1. Scientists will be informed of the latest technologies in e-science We conducted a Readership Survey ( ) I use iSGTW to keep up-to-date with technical developments in all areas of e-science/cyber-infrastructure. 68% agreed. 81% of those surveyed have discussed or forwarded an article Brainstorm outcomes..

© e-ScienceTalk Consortium 2. Provide assistance to the community in finding future partners /collaboration/helping with new ideas Readership survey I have found out about tools, services, resources, projects, initiatives, and/or potential collaborators of which I was previously unaware of. 67% said yes Open-ended Question. Received a grant, featured articles in their newsletter, given an interview and had “a new idea for research”. Brainstorm outcomes..

Focus group “There are a lot of layers to getting in touch with other projects. Sometimes, you see that in France, they are doing something with digital humanities libraries, and that makes you think what are we doing here, and why aren’t we doing those type of projects here. That’s my angle when I am reading iSGTW.”

© e-ScienceTalk Consortium 3. Journalists from mainstream media will have established relationships with those within e-science through iSGTW Google alerts, Googling isgtw titles Media pick-ups from HPC Wire, Cosmos Magazine, Wired and Symmetry. ‘The smallest music in the universe’ was published in several publications including Symmetry, Discovery News, Wired UK and Wired US.

© e-ScienceTalk Consortium e-ScienceBriefings Quantitative Q: What is the briefings reach? Total number of downloads AddThis shares Qualitative Q: How do briefings help policy makers, European science projects, government and business? Online survey mailing list Informal chat at conference Personal s 11

© e-ScienceTalk Consortium e-ScienceBriefings Quantitative (Reach) 123 subscribers. Downloaded 6,450 times. Qualitative (Significance) Helps explain e-science topics to those new to the field. “I've been told that on a national level these briefings are used as material to show others as a "read this and understand or hey look at this and what they are doing or what can be done", 12

© e-ScienceTalk Consortium GridCast Quantitative Q: What is GridCast’s audience? YouTube number of subscribers and viewers (GridTalk) Qualitative Q:How does GridCast help bloggers? Focus groups ‘2012 Year-end Summer Update’ 13

© e-ScienceTalk Consortium GridCast Quantitative (Reach) The YouTube channel has attracted 105 subscribers and the cumulative number of video views is 212,549. Qualitative (Significance) Bloggers have received proposals to collaborate and have developed peer-reviewed papers from ideas first discussed in blog posts Used as a test-bed for ideas in an open, informal setting. Blog posts are picked up by other commercial publications. 14

© e-ScienceTalk Consortium GridCast 15

© e-ScienceTalk Consortium Recommendations Identify audiences and brainstorm possible outcomes Gather feedback at all stages using a range of tools Ensure you are using the right tool to measure each outcome Face-to-face can be more effective If one methodology isn’t working, try another 16

© e-ScienceTalk Consortium Collaborations with…

eChallenges paper…

© e-ScienceTalk Consortium Follow us live Blogging live at Follow us on Twitter at Follow iSGTW on Twitter at (also on Facebook and Nature Networks Forum) Sign up to our e-ScienceBriefings

© e-ScienceTalk Consortium Real Time Monitor Quantitative Who is using the Real Time Monitor? IP analysis of top 100 users Qualitative Q: What are people using the RTM for? to suspected users at institution 20

© e-ScienceTalk Consortium Quantitative (Reach) Many institutions are running the RTM. establishments (Roma Tre University, Oxford University, University of Glasgow, Max- Planck-Institut für Physik, the CC- IN2P). Qualitative (Significance) Utilised for both outreach, educational and demonstration purposes over a wide geographical spread The Real Time Monitor was “something we've looked at and been impressed by” (Science Museum, London) Real Time Monitor 21