1. Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson, Department of Psychology, The Open University claire.hewson@open.ac.uk

2. A (very) Brief History of Internet-mediated Research (IMR) Pioneers started implementing online data collection methods around the mid-1990s Bordia (1996); Dillman (1991); Gaiser (1997); Hewson (1994). Reports (e.g. validation studies) have since supported the quality of data that can be obtained in IMR Corley & Scheepers (2002); Hewson & Charlton (2005). A new era of IMR (Web 2.0) The web as organic, collaborative, interactive, ever-changing (e.g. Wikipedia); the rise of social media (e.g. Facebook, Twitter, YouTube).

3. IMR Data Collection Methods Obtrusive Approaches Surveys, experiments, interviews, participant and / or disclosed observation approaches. Unobtrusive Approaches Observation using traces and archives, e.g. discussion group archives, browsing behaviours, social networking links (SNA); lurking? 'Big Data' Data sets which become so large as to be difficult to manage; presently a 'hot topic'.

4. The Visualisation of Complex Networks Nike+ involves the placement of a sensor underneath the footbed of your Nike running shoe in order to collect data about where you've run, how long it took and where you can improve over time. http://www.visualcomplexity.com/vc/ Visualisation of a year's worth of runs uploaded to the Nike+ website. It plays back runs throughout three cities: New York, London and Tokyo. The runs showed tens of thousands of peoples' runs animating the city and bringing it to life. The software visualizes and follows individual runs, as well as showing the collective energy of all the runners, defining the city by the constantly changing paths of the people running in it.

5. Web-based Surveys and Experiments There is now a large range of software for helping implement these, particularly survey packages (which can also support simple experimental designs), e.g. 'Surveymonkey', 'Qualtrics'. Dedicated packages to support experimental web-based studies include: WEXTOR (Reips, wextor.org/wextor/en) and WEBEXP (Edinburgh webexp.info).wextor.org/wextor/enwebexp.info

6. Web-based Surveys and Experiments New design issues and principles emerge in this novel context. Useful resources and guidelines include: Exploring Online Research Methods: www.restore.ac.uk/ormwww.restore.ac.uk/orm WebSurveyMethodology: websm.orgwebsm.org Couper (2008). Designing Effective Web Surveys. CUP: New York. Reips & Krantz (2010). Conducting True Experiments on the Web. In Gosling & Johnson (eds) Advanced Methods for Conducting Online Behavioral Research. APA. Fielding, Lee & Blank (2008). The Sage Handbook of Online Research Methods. Sage: London.

7. Recruiting Participants in IMR Sample Bias Issues?, e.g. Bordia (1997). True Volunteers, place adverts in public spaces, e.g. newsgroups, webpages, IMR study clearing houses, Facebook. List-based Sampling Frame, e.g. email lists, online panel. Probability Sampling Methods, e.g. random selection from known list, online probability panel. General Principle: methods which allow some estimation of the sampling frame are preferable where measures of response rate, non-response bias, etc. are important to the research design.

8. Online Sampling Resources ● Online Study Clearing Houses, e.g. Online Psychology Research: http://www.onlinepsychresearch.co.uk http://www.onlinepsychresearch.co.uk ● Newsgroups, SNSs, e.g. Facebook: facebook.com, Newsgroups (e.g. http://uk.groups.yahoo.com; https://www.groups.google.com), Mailing Lists (see the searchable Catalist database: http://www.lsoft.com/catalist.html)facebook.com http://uk.groups.yahoo.comhttps://www.groups.google.comhttp://www.lsoft.com/catalist.html ● Online Panels, e.g. Knowledge Networks: http://www.knowledgenetworks.com/knpanel http://www.knowledgenetworks.com/knpanel Online workforces, e.g. Mechanical Turk: www.mturk.comwww.mturk.com OU Participant Panel, Jim will talk more about this

9. OU ‘Virtual Participant Panel’ URL: http://www.open.ac.uk/socialsciences/psychology/cog psypanel/ People can participate in studies ad hoc and/or… … can sign up to be contacted for future studies. Sign-ups give certain information (e.g. sex, age, location, handedness) which can be used to filter if particular characteristics are needed in a sample. Currently has 1,245 signed-up members. Has a built-in ‘Questionnaire Manager’ to create surveys.

10. OU ‘Virtual Participant Panel’ Instruction text Question text Response text

11. OU ‘Virtual Participant Panel’ Stimulus image (‘question’) Response images (‘answers’ ) Response buttons

12. SurveyMonkey Example ● Computer Attitudes Survey https://www.surveymonkey.com/s/computer_attitudes_study https://www.surveymonkey.com/s/computer_attitudes_study ● Qualtrics – site license for all OU social science faculty students / staff. Visit 'openss.qualtrics.com' and sign up with a valid OU (open.ac.uk) email address.'openss.qualtrics.com Martin discussed using Qualtrics in a radiology expertise study this morning, and Jim will demonstrate an online experiment implementation using Qualtrics. Importing studies from SurveyMonkey into Qualtrics?

13. Qualtrics

14. Stimulus image Response images

15. Ethics in Online Research Some ethics issues require special consideration in an IMR context, the complexities of which are not always obvious to researchers or participants. e.g. Data security, traceability (confidentiality, anonymity).

16. Key feature of IMR: involves acquisition of data from or about individuals in absence of face-to-face co-presence COUPLED WITH greater scope for carrying out quite complex interactive procedures

17. The New BPS Guidelines (November 2013) Document aims to outline some of the key ethics principles for researchers and ethics committees to keep in mind when designing and / or evaluating an IMR study Emphasis on how to apply existing ethics principles in an IMR context “Thinking is not optional”

18. The New BPS Guidelines (November 2013) ● Centred around the four main principles of the BPS 'Code of Human Research Ethics' (2011). 1. Respect for the autonomy and dignity of persons 2. Scientific value 3. Social responsibility 4. Maximising benefits and minimising harm

19. Main Ethics Issues to Consider in IMR Public-private domain distinction Confidentiality* Copyright Consent; withdrawal; debrief* ● Levels of control* ● Disruption of social structures

20. Valid Consent Verifying characteristics (e.g. age, sex) Engagement with consent process (check boxes?) Adequate information – not all risks online are obvious, e.g. potential data leakage (balance between information and overload) Risks will be higher in some contexts than others – key principle of proportionality of ethics procedures to levels of risk of harm re-emerges

21. Withdrawal and Debrief Robust withdrawal procedures may be more difficult to implement online (e.g. volunteer web survey) Detecting withdrawal may be harder (use of partial data?) Ensuring debrief difficult (e.g. exit by closing browser) Enhanced risk in cases of deception (no debrief)? Principle of proportionality of ethics procedures to levels of risk (of harm) re-emerges, e.g. effects of withdrawal without debrief (such as in very sensitive research)

22. Confidentiality, Anonymity ● Enhanced traceability of data, dissemination implications, use of quotes. Other data leakage risks, e.g. email. Traces on personal computing equipment. Sensitivity of data is a factor associated with levels of risk of harm. Researchers should carefully assess risks, and perhaps inform participants of these risks.

23. Data Protection UK Data Protection Act (1998). Issue of storing data on non-UK/EU servers? Popular software packages (e.g. SurveyMonkey, Qualtrics) are US-based. 'Safe Harbour Scheme'.

24. Office of National Statistics SurveyMonkey, Safe Harbour Scheme

25. Levels of control Over procedures, who takes part, participant behaviours... Implications for scientific value (validity) Implications for levels of harm (monitoring participant reactions, presenting debrief info., etc.)

26. Maximising Benefits, Minimising Harm These considerations permeate the other issues Careful assessment needed; trade-offs will be inevitable (e.g. anonymity versus traceability/verifiability; disclosure versus social disruption/harm) “Thinking is not optional”

27. AoIR http://aoir.org/http://aoir.org/

28. Some Examples ● http://www.socialpsychology.org/expts.htm http://www.socialpsychology.org/expts.htm http://www.onlinepsychresearch.co.uk http://psych.hanover.edu/Research/exponnet.html http://www.wexlist.net