Quantitative research – variables, measurement levels, samples, populations HEM 4112 – Research methods I Martina Vukasovic

Variables (1) Independent and dependent –Independent – suspect for cause –Dependent – outcome of interest Types (measurement levels): –Nominal/categorical Dichotomies as special type –Ordinal –Interval –Ratio

Variables (2) Why is the choice important? –If not in line with the concept – then jeopardizing construct validity –Use of statistical tools depends on types of variables How to choose? –What are the attributes of the particular concept?

Variables (3) 1.Are there more then two categories? a)NO  variable is dichotomous b)YES  go the next question 2.Can the categories be rank ordered? a)NO  variable is nominal/categorical b)YES  go to next question 3.Are the distances between categories equal? a)NO  Variable is ordinal b)YES  go to next question 4.Does a zero value of the variable make sense? a)NO  Variable is interval b)YES  Variable is on the ratio measurement level

Variables (4) - exercise Determine measurement level of the following variables: –Age –Gender –Education attainment –Occupation –Duration of studies –Research productivity –Approach to teaching Discuss your choices

Sources of data (1) Sources: –statistical data bases  already collected data –questionnaires/surveys  you are collecting the data Often from a trustworthy source –e.g. Ministry, national statistical bureau, UIS –But are they always trustworthy? Sometimes you have to collect data by yourself

Sources of data (2) - tips Make sure you understand the definitions of indicators Make sure the indicators are comparable (if doing a comparative study) Check who is the actual source, esp. for international data bases If you are collecting the data on your own, reliability of data depends on –how good the questionnaire/survey is –how representative is the sample

Operationalisation (1) From concepts to indicators Example 1: –Concept – intelligence –Indicator – IQ –Tool for measuring intelligence: tests that yield IQ Example 2: –Concept – social intelligence –Indicator – social IQ –Tool for measuring social intelligence: tests that yield SIQ (?) Example 3: –Concept – intelligence –Indicator – several different IQs –Tool for measuring intelligence: tests that yield IQ Sometimes you can use several indicators for one concept –BUT you need to have a reason to do so –AND you need to be clear how you combine these indicators

Operationalisation (2) Can be presented as a 2-step proces: –Development of an indicator –Development of a tool to measure the indicator Sometimes already defined (if using data bases) –make sure you understand the definition and the tool –be critical about them Sometimes you define it –be careful about construct validity!

Operationalisation (3) IQ – good indicator of intelligence? IQ test – good measuring tool? How would you operationalise quality in HE? –Why?

Sampling (1) Sampling –A sample is a representative part of the population you are interested in Important for generalization! An assumption for using statistical tools in the first place Different techniques for sampling –Depends on your research topic

Sampling (2) Random sampling –In some computer programmes (e.g. SPSS/PASW) this can be done automatically Generators of random numbers –Can be done in various ways, although some techniques may introduce bias if not careful –Selection from the population is done entirely at random  no bias (?)

Sampling (3) - exercise Imagine you need a random sample for a study. Discuss what bias, if any, can be introduced by using the following methods: –Stopping people in public and asking questions: On the street In the theatre –Distributing a questionnaire inside the classroom –Calling people on the phone –Asking people to complete an online survey

Sampling (4) But completely random sampling can in some ways introduce bias even if done correctly For some topics and populations, stratified sampling is more appropriate –E.g. when you know in advance that the distribution is not normal

Sampling (5) Stratified (random) sampling –Divide the population in several groups, or strata –Identify how many respondents or “cases” you need in each group on the basis of their proportion in the entire population –Do random sampling within this strata –Check after data collection if your stratification worked

Sampling (6) - exercise What kind of sampling procedure you need, if interested in the following issues (if you think you need to stratify the sample, also discuss what strata you need): –Female students are more successful than male students; –Students who pay for their education are more concerned with quality of education; –Mobile students have difficulties in obtaining jobs after graduation?

Constructing a questionnaire (1) Which data do you need? –back to conceptual framework (+ research questions) How do you operationalise the concepts? –Use the literature, see what others did before you –Several questions can serve as indicators of one concept –Sometimes “control” questions are used to check the internal consistency of answers Be clear what is the purpose of each question –Useful also to see them terms of how they relate to independent or dependent variables

Constructing a questionnaire (2) Questions MUST NOT be ambiguous –Piloting is necessary –Be aware of language issues Options for answers need to be clear Layout needs to be user friendly –The respondent needs to be able to complete the questionnaire easily –Otherwise, you risk if incomplete questionnaire  incomplete data bases

Practicalities (1) Make sure you do not jeopardize your sample Make sure you allow enough time for responses The collecting procedures needs to be as simple as possible Expect low response rates, not everyone will answer

Practicalities (2) online or paper? Think if this affects your sample If paper questionnaire –putting data into the data base requires time, discipline and concentration –useful to label each completed questionnaire with a unique number and introduce that into the data base as well (for later checks)

Practicalities (3) Analysis –Make a plan beforehand! Statistical packages can be seductive –Be systematic in building the data base, especially in terms of variable labels, types etc. Always make notes of all the manipulations to the data base that you make Keep your data base as well as files with results of analysis safe and make regular backups  SPSS workshop as part of HEM 4113