Chapter 12: Surveys Introduction 12.1 The method 12.2 Random samples 12.3 Designing questionnaires 12.4 Pilot studies 12.5 Collecting data 12.6 Processing data 12.7 Getting professional help

Introduction Counts things in a group (population) Members in group with a specific characteristic are counted and compared to members without that characteristic Three aspects Population is often too large to count: Count a sample and generalise If human population, count opinions Surveys are not done to determine current status of population, but to prove theories Problems You might miss target group Poorly formulated questions (ambiguous, etc.)

12.1 The method The method To assist Start with a theory Formulate a hypothesis Select a sample Collect the data Analyse the data Test hypothesis Confirm or modify theory To assist Know the broad problem, e.g. ease of use Know dimensions, e.g. ease of learning, ease of use under pressure, etc. Identify indicators. E.g. ease of learning may be measured by questioning people directly, comparing duration of training courses with that of other systems, obtain statistics on support systems used, etc. Determine relative weight of various indicators

12.2 Random samples Know exactly what the population is Select a sample in such a way that every member of the population has equal chance of being in the population E.g don’t use telephonic interviews / internet if every member of population is not reachable by telephone / internet If necessary, redefine population, e.g. refer to computer users in a specific country/ language/etc.

12.3 Design questionnaires Distinguish between questionnaires and schedules Ask precisely and unambiguously Don’t ask questions that participants are unlikely to know, e.g. “How many minutes did you spend on the Internet during the past month?” Don’t ask questions that participants do not want to answer or will most likely not answer honestly, e.g. “What is the weakest point in your company’s computer security measures?” Keep the questionnaire short and simple. Pay attention to professional appearance.

12.3 Types of questionaires Nominal measures (example p 82) Ordinal measures Using numbers to express preferences but weight of number does not bear any meaning Ratio measures Measurement in a unit where 0 means zero Interval measures Measurement in a unit where 0 does not mean zero Likert scales Specify degree to which specific statement applies Example p 83 LPC scales Respondent indicate preference between two explicit alternatives Open questions Format of response depends on respondent

12.4 Pilot studies Once the survey begins, you will experience problems with respondents not understanding instructions, certain questions that prove to be ambiguous, etc. Do a pilot study Select a small group from the population – need not be representative but it will help Conduct the survey Results are not important - look for hitches Adapt questionnaire If many problems were encountered, repeat pilot study

12.5 Collecting data If you use assistants, ensure that they all collect the data in the same way. If they explain the questions, the explanation might influence the answers Use detailed protocol If you depend on cooperation of people in the sample People might not be honest Pay them? Promise copy of research report Make results available on the internet Create a sense that participants are contributing to something worthwhile Conduct the survey under auspices of an organisation with a reputation of conducting researches Personal contact

12.6 Processing the data Sometimes easy, e.g. you are only interested in the totals or averages for every question. Sometimes you need more statistical analysis (chapter 13).

12.7 Getting professional help Statistical consultation service Don’t contact them only after you have collected the data The data might have an inherent flaw and all the effort, time, cost is wasted Contact them prior to setting up experimental design