1. Principle, Ariel Training Consultants
Data
&
Data Collection
by Miles Hamby, PhD
Principle, Ariel Training Consultants
Source: Wadsworth, Thomson (2005). So you want to use a Likert scale? from the Learning Technology and Babbie, Earl R. (2005). The Basics of Social Research. p ISBN and Likert, Rensis (1932), "A Technique for the Measurement of Attitudes", Archives of Psychology 140: pp. 1-55

2. Problem Statement drives the research question
Research Question drives the methodology
Methodology drives the data collection

3. Data Collection Instrument
Driven by the statistical or analysis procedure intended (descriptive, correlation, regression, ANOVA, non-parametric, t-test. etc)
Type of data (nominal, ordinal, interval, ratio)
Scale and values

4. { Levels of Data RATIO INTERVAL ORDINAL NOMINAL Meaningful ‘0’
e.g., age, height
RATIO
Quantitative __
Meaningful X {
e.g., Fahrenheit scale
INTERVAL
Qualitative or Quantitative
e.g., Order of finish in a race
ORDINAL
NOMINAL
Qualitative only
e.g., eye color, jersey numbers

5. Variables
Before composing a survey, identify the dependent variable and independent variables.

6. Variables
Dependent Variable – the characteristic (variable) of an hypothesis that is dependent upon the action of other variables.
Independent Variables – those characteristics (variables) that effect the dependent variable.
e.g. “The purpose of this research is to determine if daily exercise will reduce a person’s weight.”

7. Variables
e.g. “The purpose of this research is to determine if pay is a significant factor in job satisfaction.”
e.g. “The purpose of this research is to determine if increasing oil prices has changed people’s lifestyles.”

8. To collect specific data for testing
Purpose of a Survey
To collect specific data for testing
an hypothesis

9. Data Measurement Scales - Likert
Pronounced 'like-urt‘
Named after Rensis Likert, who published a report describing its use (Likert, 1932).
Type of psychometric response scale
Widely used in survey research.
Respondents specify level of agreement to a statement.

10. Likert Scale – cont’d
Respondent asked to indicate his or her degree of agreement with the statement or any kind of subjective or objective evaluation of the statement.
The five response categories represent an Interval level of measurement.
Traditionally a five-point scale is used,
Many psychometricians advocate using a seven or nine point scale - some recommend even-numbered

11. “Pay is important to my job satisfaction.”
Likert Scale – Cont’d
Example.
Please indicate the strength of your agreement or disagreement with the following statement.
“Pay is important to my job satisfaction.”
Strongly Neither Agree Strongly
Agree Agree Disagree Disagree Disagree

12. Likert Scale – Scaling
Bipolar - measures positive or negative tendencies toward a statement.
Even-point scale used to force respondent ‘off the fence’
e.g., middle option "Neither agree nor disagree" not available.

13. “Pay is important to my job satisfaction.”
Likert Scale – Cont’d
Example.
Please indicate the strength of your agreement or disagreement with the following statement.
“Pay is important to my job satisfaction.”
Strongly Strongly
Agree Disagree

14. Likert Scale – Scaling Likert scales subject to distortion
central tendency bias – Respondent avoids extreme response categories
acquiescence bias – respondent agrees with statements as presented
social desirability bias – respondent tries to be portrayed in favorable light

15. Likert – Analysis Summative Analysis
Items summed to create score for a group
Treated as interval data measuring a latent variable
With more than 5 responses, summed responses fulfil relevant assumptions, parametric statistical tests such as the Analysis of Variance (ANOVA) can be applied.
Nominal data - combining all agree and disagree responses into two categories of "accept" or "reject".
Commonly used statistical procedures - Chi-Square, Cochran Q, or McNemar

16. Ipsative Scale
Rank-order scale - particular rank used only once - opposite of Normative.
Example – “Rank the following ten movies from best to worst, in your opinion.”
With multiple raters , mean, median, and standard deviation is the same.

17. Normative Scale
With multiple raters, the mean, median, and standard deviations usually different from one iteration to the next.
Vogt, W. Paul (2005). Dictionary of Statistics and Methodology: A Non-technical Guide for the Social Sciences, Sage Publications, Inc. ISBN

18. Survey Example

19. Satisfaction Survey
This survey is being conducted as a part of coursework for a graduate degree at XYZ University and is in no way connected to your place of employment. Your responses are anonymous and information will be confidential.
Please indicate your response to the following questions:
1) Overall, how satisfied are you with your current job?
VERY STAISFIED 4
SATISFIED 3
NOT SATISFIED 2
VERY DISSATISFIED 1
2) How significant a factor is pay in your job?
VERY SIGNIFICANT
SOMEWHAT SIGNIFICANT
SIGNIFICANT
NOT SIGNIFICANT
Please indicate the strength to which you agree or disagree with the following statements:
Strongly Agree Strongly Disagree
3) Management keeps me informed.
4) The work is interesting.
5) My pay is commensurate with my job requirements.
6) My work schedule is flexible.
7) The environment is conducive to work.  
8) Please indicate your work status.
9) Please indicate your age (write in )______________
10) Please indicate your ethnic origin.
11) Please indicate your gender.
12) Please indicate your family status.
13) Please indicate your income level.
WHITE HISPANIC AFRICAN AMERICAN
AFRICAN ASIAN/PACIFIC ISLANDER NATIVE AMERICAN/ALASKAN
MALE FEMALE
MARRIED, NO CHILDREN MARRIED W/ CHILDREN
SINGLE, NOT SUPPORTING SINGLE PARENT
< $10, $10,000 to $30, $30,000 to $50, $50,000 to $80, >$80,000
Please return the completed survey in the SASE to: Researcher, PO Box 459, Alexandria, Va, 22306

20. Survey Guidelines
Compose questions with research DV, statistical procedure, and ease of data entry in mind
Group questions by type ~ eg, Strength of Agreement, Level of Perception, Demographics
State questions in the affirmative
Code higher numbers affirmative, low numbers unfavorable
Insure visually asthetic ~ e.g., alignment, font size & style, consistency in capitalization, numbering
Strive for 1 page max

21. Survey Hosts
MS Excel
Survey and questionnaire forms can be composed on an MS Excel spreadsheet and ed or sent electronically to potential respondents. The respondent can open the Excel spreadsheet, simply click on the appropriate response (Check-box or Option button).
1. Open Excel, click on View/Toolbars/Forms
Write the survey question in a cell
To insert a response item, click on ‘Check Box’ or ‘Option Button’ – position the box to the appropriate place next to the question and type in the text for the appropriate response. Continue to add and position response boxes as required. ‘Group’ the question and response boxes with the ‘Select Objects’ arrow (from the Draw toolbar) and Draw/group.

22. Many providers/hosts of online surveys
Online Survey Hosts
Many providers/hosts of online surveys
Search keywords ‘online survey providers’
Many offer limited use free or full short-term use for reasonable cost.
Before selecting an online survey host - identify the research dependent and independent variables, measurement scales, statistical method, and specific questions.

23. Data
&
Data Collection

24. Likert Scale – Analysis
Single Item Analysis
Responses to single item treated as ordinal data
With only 5 levels, cannot assume respondents perceive difference between adjacent levels as equidistant.
As ordinal data, Likert responses analyzed using non-parametric tests, e.g., Mann-Whitney test, the Wilcoxon Signed-rank test, or Kruskal-Wallis test.

25. Consensus Based Assessment (CBA)
Likert – CBA
Consensus Based Assessment (CBA)
Used to create objective standard for Likert scales in domains where no generally accepted standard or objective standard exists
CBA can be used to refine or validate generally accepted standards.

26. Likert – Rasch Model
Used to obtain interval level estimates on a continuum
Polytomous Rasch model permits testing of hypothesis that the statements reflect increasing levels of an attitude or trait, as intended.
e.g., application of the model often indicates neutral category does not represent a level of attitude or trait between the disagree and agree categories.