Evaluation of Web Sites: What Works and What Doesn't Sue Ellen Hansen, Survey Research Operations, University of Michigan Matthew Richardson, ICPSR, University of Michigan

1.Is the navigation clear and consistent? 2.Does the site meet accessibility guidelines? 3.Has the content been written for the Web? 4.Does the visual design improve or impair the acquisition of knowledge? 5.Is the search engine a barrier to accessing content? 6.How clear and navigable is the access system? 7.Does the site do an adequate job of getting the user to his/her real goal, data analysis? 8.How well does the site educate the user and encourage further research? Evaluative Principles

Behavioral Risk Factor Surveillance System (BRFSS) Centre for Comparative European Survey Data (CCESD) Information System Council of European Social Science Data Archives (CESSDA) Cultural Policy and the Arts National Data Archive (CPANDA) The Health and Retirement Study (HRS) The Henry A. Murray Research Center Integrated Public Use Microdata Series (IPUMS) International Social Survey Programme (ISSP) Inter-university Consortium for Political and Social Research (ICPSR) National Election Studies (NES) Panel Study of Income Dynamics (PSID) The Roper Center Sites Reviewed

1. Is the navigation clear and consistent? Is the navigation consistent in terms of placement and style? Do the navigation labels make sense? Are they free of jargon? Are there too many navigation items? Are there obvious links for commonly used Web site features? I.e., "contact us", "help", "site map", "search", and "privacy policy". Can the user reach content without searching?

2. Does the site meet accessibility guidelines? Does the site use proper linking behavior, such as matching the link name to the page title and making sure each linked phrase is self-explanatory? Does the site use page positioning or color as a navigational tool, such that the site may be unusable to users with visual impairments? Is the text scaleable? Section 508 – WAI –

3. Has the content been written for the Web? Is the text concise and well-organized? Is the writing simple and oriented towards modular paragraphs and bulleted lists? Is the content free of marketing fluff? Are acronyms spelled out and jargon avoided?

4. Does the visual design improve or impair the acquisition of knowledge? Is there enough color contrast to ease on-screen reading? Are printer-friendly pages enabled? Is there a visual cue for clickable image elements?

5. Is the search engine a barrier to accessing content? Does the search engine offer a basic and advanced option? Can the user narrow his/her search results? Is there a help document on searching that spells out issues such as case sensitivity, phrase vs. word searching, Boolean searching, field searching, etc.? Does the search engine search all site content?

6. How clear and navigable is the access system? How clearly are access restrictions spelled out? I.e., terms of use. To what extent is security/access an impediment to the user? What kind of registration is required? To what extent is the archive wed to Web distribution, or can it accommodate other means of data dissemination? Are confidentiality issues addressed in a meaningful fashion?

7. Does the site do an adequate job of getting the user to his/her real goal, data analysis? How quickly can the user get to data? How easy is it to acquire the full bundle of products? I.e., all files needed to perform analysis, including documentation. Can you browse the documentation separately? Is it clearly labeled? Are multiple software suites supported? How descriptive are the metadata? Does the site provide any form of online analysis?

8. How well does the site educate the user and encourage further research? To what extent does the raw data serve as a gateway to other research? Does it reflect additional steps in the research data life cycle? What does the site offer in term of data literacy education? To what extent does the site attempt to educate novice users?

Other Ways Sites Encourage Research For series data, does the site link the waves to one another? Are aggregate files available? Is it possible to find other research conducted by the investigator? In a particular subject area? Funded by a particular agency? Does the site offer or link to statistical education programs? Does the site direct users to conferences of interest to social science researchers?

Conclusion You should invest more time into designing your site's navigation than you do in dealing with visual design. Familiarize yourself with accessibility guidelines, and make your site accessible. Be wary of search engines. Consult experts when planning your site's search engine. Spell out your terms of use. Streamline the process of distributing data. Make sure that the core function of your site is doable with minimal time and effort. Remember that research doesn't really end with the data download. Analysis is where it gets interesting, and we should be more than just a file transfer utility.