1. Online data collection A mixed blessing? Samuel C. Lefever Assistant Professor Iceland University of Education Ásrún Matthíasdóttir Assistant Professor Reykjavik University Michael Dal Assistant Professor Iceland University of Education Anna Ólafsdóttir Supervisor of Distance Education University of Akureyri

2. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 2 Overview Possibilities and limitations of online data collection for research purposes Experience of online data collection in four surveys conducted in Iceland in 2002 A part of the research project LearnICT – using ICT in learning and teaching in Iceland, a project funded by the Icelandic Centre for Research (Rannís) under the Information Technology research program

3. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 3 Online data collection Here the term “online data collection” is used for data collection with the use of a web site to collect real-time data and store in a format suitable for use in statistical programs Online data collection may be replacing paper-and-pencil surveys or questionnaires in the near future Well-suited for quantitative research

4. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 4 Advantages Time saving Cost efficient Population sample can be large, diverse and global Quick and convenient collection of responses Less likelihood of human errors – loss of data Less personal nature of online surveys may give more honest answers Novelty of online surveys may increase participation Online data are easily converted for analysis

5. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 5 Limitations Difficult to define the population Inability of selecting a random, representative sample Generalizing results is questionable Unreliable or unavailable population lists Possibility of imposters or untruthful answers Rejection of e-mail surveys as “junk mail” Technical problems Lack of computer literacy Lack of willingness to participate online Potential breaches of respondent privacy Browser incompatibility

6. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 6 Online data collection at the university and upper secondary level in Iceland 2002 Three universities Fifteen upper-secondary schools

7. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 7 Online surveys Extensive surveys with 4 sections Background information: age, gender, school, etc. General information about computer and Internet access, etc. Attitudes towards ICT use in teaching and learning Experience and attitudes towards distributed and distance learning 28 – 35 questions with several answering possibilities

8. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 8 Online surveys Research sample – upper secondary level Students 2093 (24%) respondents out of 8575 Females 65% (52%) and males 35% (48%) 70% were 16 – 19 years old Teachers 423 (47%) of 906 Females 48% (51%) and males 52% (49%) 62% were 41 – 60 years old

9. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 9 Online surveys Research sample – university level Students 2040 (46%) Females 75% (71%), males 25% (29%) 77% were 20 – 39 years old Teachers 187 (59%) Females 49% (46%), males 51% (54%) 54% were 41 or older

10. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 10 Data collection method Reaching the sample population Outcome – a commercial e-survey system Sends out introduction letter with e-mail Directs participants to a particular website in order to fill in the questionnaire Sends out reminders when requested Prevents repeated answers Compiles the data in an Excel file and Word report

11. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 11 Data collection method Reaching the sample population E-mail was used for contacting participants Permission was received from school authorities Some schools provide students with e-mails In other schools we had to collect e-mails and received only limited numbers of e-mail addresses

12. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 12 Data collection method Reaching the sample population Letters were sent to school administrators to encourage participation Flyers were sent to schools for posting on bulletin boards Notices were posted on school closed-circuit television screens

13. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 13 Findings Older participants responded better Upper-secondary students were most reluctant to respond Returned messages – incorrect emails (up to 7%) Reasons for not responding Multiple reminders caused aggravation Younger students showed negative responses to the survey Rude comments and ‘threats’ Too time consuming Opposition to participation Not applicable to the participant

14. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 14 Unreliability of e-mail addresses Participants have more than one e-mail address and may not use the one the school provides them WebCt e-mail used instead of school-issued e- mail address Who has the ownership of e-mail addresses The school or the student? Discussion

15. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 15 Discussion Limited access to upper secondary schools for outside researchers Disturbs the teaching Schools are inundated with requests for surveys You need ‘connections’ to get permission Information laws – underage participants Need for parental consent?

16. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 16 Discussion Face to face data collection Can give high response rate, but are answers given on a voluntary basis? (reliability of answers) Ways to increase participation Provide incentives Break down larger surveys into a few smaller ones Improve ways of making contact with potential participants

17. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 17 Conclusions Access to large and distributed population Reliable email access - is there a solution? Time and cost efficient for the researcher Less cost and work in data collection Questionnaires must be simple and quick to complete Online surveys no longer as appealing as expected Depends on factors such as age, interest, maturity, duty to the profession, etc. Reliability Are “volunteer” responses more reliable if you can choose when and whether to answer? Need for further research

18. Samuel C. Lefever, Ásrún Matthíasdóttir, Michael Dal, Anna Ólafsdóttir 18 Is there cause for concern regarding data collection and new research methods?