1. A centre of expertise in digital information managementwww.ukoln.ac.uk Impact Analysis For Web Sites Brian Kelly UKOLN University of Bath Bath Email B.Kelly@ukoln.ac.uk UKOLN is supported by: http://www.ukoln.ac.uk/web-focus/events/conferences/museums-web-2006/ Acceptable Use Policy Recording/broadcasting of this talk, taking photographs, discussing the content using email, instant messaging, Blogs, SMS, etc. is permitted providing distractions to others is minimised. Acceptable Use Policy Recording/broadcasting of this talk, taking photographs, discussing the content using email, instant messaging, Blogs, SMS, etc. is permitted providing distractions to others is minimised. This work is licensed under a Attribution- NonCommercial-ShareAlike 2.0 licence (but note caveat) museums-web-kelly-2006-06 tag used in del.icio.us

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Contents Usage Statistics Strengths and weaknesses Impact Analysis What is it? Impact Analysis tools and techniques Web 2.0 and Impact Analysis Using Web 2.0 tools for impact analysis Web 2.0 is impact Sector Statistics What is it? Would it be useful?

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 Usage Statistics Exploit Interactive example: EU-funded e-journal funded from 1999-2000 Steady growth in usage measured by Sitemeter & usage logs Usage Statistics Deeper analysis: Popular article on "Are You Linking To A Porn Site?", April 1999 Issues: Is popularity of Web site based on "wrong hits" and growth reflects growth in Internet usage? How can we aggregate usage data in a meaningful way? Issues: Is popularity of Web site based on "wrong hits" and growth reflects growth in Internet usage? How can we aggregate usage data in a meaningful way?

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 Usage Statistics Usage statistics: Can provide useful and valuable data … … but can also be flawed Implications: Collect and use as they can be useful for formative statistics e.g. has a new marketing campaign worked; what keywords do users use to find our Web site; … But also: Be honest about limitations and don't over-hype statistics Use as a part of a portfolio of metrics Usage Statistics

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 Impact Analysis Usage statistics: Data on who is using Web site and how they're using it e.g. how many visitors are there? Impact analysis: Attempting to measure the impact of the Web site or the contents of the Web site e.g. how has the Web site visit affected the visitors Challenges: More difficult to measure than simpler 'visits' Requires thought as to aims of Web site Requires Objective and subjective metrics e.g. 'possible' indicators of impact Automated and manual measurements Impact Analysis

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 Types Of Metrics Automated: Usage statistics Link analysis Search analysis Manual: Focus groups, surveys, etc. Hybrid: Blogs, RSS feeds, social bookmarks, etc. Let's use a case study for the Institutional Web Management Workshop (IWMW) 2006 Web site – an annual 3 day event for University Web managers Impact Analysis

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 Link Popularity Why link popularity? A link to my Web site is an indication that A We b author felt strongly about the resource Links help drive traffic (and so can boost usage statistics) Links enhance Google rating (and so can boost usage statistics) http://www.linkpopularity.com/ Note that you can receive a monthly email report, which can provide information on trends It may be desirable to remove links from your own Web site, otherwise growth may just reflect growth in your site Many site analysis tools now provide RSS feeds. This can help Web managers in their monitoring activities

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Search Analysis Why search engine analysis? Search engines help drive traffic (and so boost usage statistics) Impact Analysis Technorati: There's more to search engines than Google Searches Blogs, RSS news feeds, etc. Use of tags can help searching social networking services such as Blogs, Flickr, etc

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 Talking To Users Impact analysis can be carried out by talking to / listening to the user community: Focus groups Visitor books Evaluation forms Anecdotes Media watch … Such techniques are well-known – but how can technologies be used to support such activities? Impact Analysis – Manual Techniques

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 Using Blogs Blogs can make it easier to gather quotes & other impact measures: Impact Analysis Blogs provides good indication of user satisfaction and of impact analysis of event "I've come back with ideas to …" Finding Blogs (e.g. tags, Technorati) http://j4.livejournal.com/2006/06/19/ http://www.meanboyfriend.com/overdue_ideas/ 2006/06/10_years_of_the.html http://www.meanboyfriend.com/overdue_ideas/ 2006/06/10_years_of_the.html Blogs associated with Web site Official Bloggers has proved useful: set a standard for others; claimed a tag in tag space; shown benefits of trust; …

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 Being Open The impact of ideas can be maximising by allowing the ideas to be used freely: Open source software Open standards Creative Commons licence for content CC licences for IWMW 2006 resources allows: Info about event to be used by others Blog articles, news, etc. to be syndicated Impact Analysis http://www.ukoln.ac.uk/web-focus/events/ workshops/webmaster-2006/publicity/ http://www.ukoln.ac.uk/web-focus/events/ workshops/webmaster-2006/publicity/

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 Impact Analysis Providing Mashups You can maximise the impact of your Web site by allowing others to make use of your content e.g. Syndication News feeds Third party applications … Note CC licence for event details permits such reuse http://upcoming.org/event/69469/

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 Building A Community Building a community for your Web site can: Maximise impact by allowing interested parties to discuss their shared interests Provide you with feedback & ideas Allow you to provide targetted information http://www.frappr.com/iwmw2006 Web 2.0 services such as Frappr, Blogger, MySpace, etc. allow Web communities to be easily set up (and may be particularly valuable to the 'Net Generation')

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 Sharing Resources Social bookmarking tools like del.icio.us: Provide an indication of impact (others have felt Web page worth bookmarking Support community building (finding others with similar interests) http://del.icio.us/lisbk/iwmw2006-parallel-kelly-phipps

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 Maximising Impact & Web 2.0 We have seen how Web 2.0 technologies (Blogs, RSS feeds, syndication technologies, third party services, etc) can: Maximise impact by providing additional access mechanisms for users Minimise resource effort needed by making use of 3 rd party services Be used to measure the impact of our Web services Impact Analysis and Web 2.0

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 The Bigger Picture Usage analysis and impact analysis aren't self- contained, but form part of a bigger picture Usage analysis Who's accessing Web site? How are they finding site? What's popular? User testing How do users navigate site? What's difficult to find? Have they achieved their aims? Impact Analysis How has Web site influenced the users? What do the users find interesting? What changes do users make as a consequence? These areas are complementary and should be addressed as part of a strategy for ensuring user needs are being satisfied. The strategy should be hybrid, covering software and processes, real world interaction, etc. Bigger Picture

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 Sector Statistics Another aspect of the 'bigger picture' is statistics for the sector: What are the characteristics of the sector? What legitimate assumptions can we make? What are the benefits to be gained by sector statistics? Bigger Picture UKOLN has worked with Nedstats who have provided sector statistics based on 22 higher educational institutions who were willing to host their (invisible) counter. Summary based on > 25 million page views and > 4 million visitors See UKOLN has worked with Nedstats who have provided sector statistics based on 22 higher educational institutions who were willing to host their (invisible) counter. Summary based on > 25 million page views and > 4 million visitors See

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 Overall: Accept95.93% Decline4.07% 2006 Average Accept96.8% Decline3.2% External: Accept94.64% Decline5.36% Internal: Accept97.90% Decline2.10% Taken from Visitors Accepting Cookies Bigger Picture Cookies needed to monitor user trails

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 How do people find my site? Overall 56.29% 20.93 28.66 Three basic ways Direct Entry External Referrer Search Engines External 39.98 21.62 38.38% Taken from Bigger Picture

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 Number of visitors per day Percentage of visitors from search engines Visitors from search engines – Number of visitors UniversityGeneral Cass CCCU Aberdeen TVU UCL Sheffield

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 Conclusions To conclude: Usage statistics should form a part of an impact analysis strategy A more comprehensive impact analysis strategy is needed This should be part of a user-satisfaction strategy Web 2.0 technologies can help Embedding Web 2.0 culture (openness, trust, etc.) can help the impact of Web sites which aim to maximise engagement with its user community "Impact Analysis For Web Sites", QA Focus briefing document no. 99, provides further advice – see Would sector statistics be useful for the museum sector?