1. Error Why do we care?

2. Reliability Degree to which measures are free from random error and, therefore, provide consistent data. There are three ways to assess reliability Test-retest, equivalent forms, and internal consistency (see the next slide)

3. Assessing the Reliability of a Measurement Instrument Test-retest reliability:Use the same instrument a second time under as nearly the same conditions as possible. Equivalent form reliability:Use two instruments that are as similar as possible to measure the same object during the same time period. Internal consistency reliability:Compare different samples of items being used to measure a phenomenon during the same time period.

4. Validity Extent to which a measurement instrument actually measures the attribute it was intended to measure. Validity can be examined from a number of different perspectives, including: Face, content, criterion-related, and construct validity (see next slide)

5. Assessing the Validity of a Measurement Instrument Face validityResearchers judge the degree to which a measurement instrument seems to measure what it is supposed to. Content validityThe degree to which the instrument items represent the universe of the concept under study. Criterion-relatedThe degree to which a measurement instrument can validitypredict a variable that is designated a criterion. (a) Predictive ability; (b) Concurrent validity Construct validityThe degree to which a measure confirm a hypothesis created from a theory based upon the concepts under study. (a) Convergent validity; (b) Discriminate validity

6. Illustrations of Possible Reliability and Validity Situations in Measurement............................ Situation 1Situation 3Situation 2 Neither reliable nor Valid Highly reliable but not valid Highly reliable and valid...

7. Sample Survey Blueprint Research Question or Objective Survey Question Describe the characteristics of our current customers preferred cookie? Describe the “perfect” chocolate chip cookie. Who is the “end consumer” of the purchased cookie? You bring the cookies home from the store. Who eats them?

8. Measurement

9. The Concept of Measurement and Measurement Scales Measurement Process of assigning numbers or labels to things in accordance with specific rules to represent quantities or qualities of attributes. Rule: A guide, method, or command that tells a researcher what to do. Scale: A set of symbols or numbers constructed to be assigned by a rule to the individuals (or their behaviors or attitudes) to whom the scale is applied.

10. Types of Measurement Scales Nominal Scales Scales that partition data into mutually exclusive and collectively exhaustive categories. Ordinal Scales Nominal scales that can order data.

11. Interval Scales Ordinal scales with equal intervals between points to show relative amounts; may include an arbitrary zero point. Ratio Scales Interval scales with a meaningful zero point so that magnitudes can be compared arithmetically.

12. Nominal Ordinal Interval Ratio WinPlace Show 1 length2 lengths 40 to 1 long-shot pays $40

13. Type of ScaleNumerical Operation Descriptive Statistics NominalCountingFrequency; Percentage; mode OrdinalRank ordering(plus…)Median Range; Percentile IntervalArithmetic operations on intervals bet numbers (plus…) Mean; Standard deviation; variance RatioArithmetic operations on actual quantities (plus…) Geometric mean; Co-efficent of variation

14. Selecting appropriate univariate statistical method ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Nominal Scale Identify sex of key executives Is the number of female executives equal to the number of males executives? Chi-square test

15. ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Nominal Scale Indicate percentage of key executives who are male Is the proportion of male executives the same as the hypothesized proportion? T-test

16. ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Ordinal scaleCompare actual and expected evaluations Does the distribution of scores for a scale with categories of poor,good, excellent differ from an expected distribution? Chi-square test

17. ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Interval or Ratio scale Compare actual and hypothetical values of average salary Is the sample mean significantly different from the hypothesized population mean? Z-test (sample is large) T-test (sample is small)

18. Questionnaire design A survey is only as good as the questions it asks

19. What should you ask? The questions asked are a function of previous decisions The questions asked are a function of future decisions (such as statistical analysis)

20. Key criteria Questionnaire relevancy No unnecessary information is collected and only information needed to solve the problem is obtained. Be specific about your data needs; tie each question to an objective Questionnaire accuracy Information is both reliable and valid

21. Phrasing Questions Open ended response versus fixed alternative questions “?” Decision criteria: type of research; time; method of delivery; budget; concerns regarding researcher bias

22. Avoid Leading questions Overly complex questions Use of jargon Loaded questions (can use a counterbiasing statement) Ambiguity Double barreled questions Making assumptions

23. Order? Order bias results from an alternative answer’s position in a set of answers or from the sequencing of questions Funneling technique: general to specific helps understand the frame of reference first Anchoring effect: the first concept measured tends to become a comparison point from which subsequent evaluations are made

24. Types of questions

25. Single dichotomy or dichotomous-alternative questions “Are you currently registered in a course at the University of Lethbridge? Yes____ No____” Respondent chooses one of two alternatives (yes/no; male/female) What scale would this data create? Types of fixed alternative questions…

26. Multi-choice alternative Respondent chooses from several alternatives Many types… Types of fixed alternative questions…

27. Determinant choice Choose only one from several possible responses “Which faculty are you currently registered in at the University of Lethbridge? Management ___ Education ____ Arts/Science____ Health sciences____ Combined degree____ What type of scale would these data create? Multi-choice alternative questions…

28. Frequency determination Asks for an answer about frequency of occurrence In a typical week, how often do you purchase chocolate chip cookies? __never __ once __ 2 or more times What type of scale would these data create?

29. Check list Provide multiple answers to a single question Should be mutually exclusive and exhaustive “What brands of chocolate chip cookies have you, to the best of your memory, purchased in the past month (check all that apply?)” __ Dare __ Chips A’hoy __ Presidents Choice Decadent etc. etc. What type of scale would these data create?

30. Attitude rating scales Attitude: An enduring disposition to consistently respond to various aspect of the world, including persons, events and objects Typically seen as having three components: Cognitive Affective Behavioural

31. Attitude Scales: Scaling Defined The term scaling refers to procedures for attempting to determine quantitative measures of subjective and sometimes abstract concepts. It is defined as a procedure for the assignment of numbers to a property of objects in order to impart some of the characteristics of numbers to the properties in question.

32. Unidimensional Scaling Multidimensional Scaling Procedures designed to measure only one attribute of a respondent or object Procedures designed to measure several dimensions of a respondent or object

33. Attitude measuring process Ranking Rating Sorting Choice

34. Types of attitude scales Simple attitude scales Most basic form – respondent responds to a single question Do not allow for fine distinctions or placement on continua You are at a company party and are feeling nervous, but you are obligated to be there. Do you: __ find someone you know to buddy up with __ take it as an opportunity to meet new people What type of scale would these data create?

35. Category scales More sensitive; provides more information Overall, how satisfied are you with the high speed performance of your Mercedes: __ very satisfied __ somewhat satisfied __ neither satisfied nor dissatisfied __ somewhat dissatisfied __ very dissatisfied If you could choose, how long would each term be? ___26 weeks __ 13 weeks __ 6 weeks ___4 weeks What type of scale would these data create?

36. Summated rating scales – the Likert scale Respondents indicate their attitudes by checking how strongly they agree or disagree with statements Chocolate chip cookies are my preferred variety of cookie Strongly disagree Disagree UncertainAgreeStrongly Agree (1) (2) (3) (4) (5) What type of scale would these data create?

37. Semantic Differential Rating scale An attitude measure consisting of a series of seven-point bipolar rating scales allowing response to a “concept” Think of your favorite type of cookie. Rate it on each of the following continua: Hard------------------------------------------------------Soft Lots of chips---------------------------------------Fewer chips Crispy---------------------------------------------------chewy What type of scale would these data create?

38. Numerical Rating scale Similar to a semantic differential except that it uses numbers as response options to identify response positions instead of verbal descriptions Think of your favorite type of cookie. Rate it on each of the following continua: Hard------------------------------------------------------------------------Soft 87654321 This scale is called an 8 point numerical scale, why? What type of scale would these data create?

39. Constant Sum Scales Attributes based on their importance to the person. Respondents are asked to divide a constant sum to indicate the relative importance of attributes Example: Suppose the photocopy budget per professor was $100 per month. How much should be allocated to the following. Divide the $100 according to your preference: ____ photocopying for student needs; ____ photocopying for research needs; ____ photocopying for committee needs. ==== $100 TOTAL

40. Stapel Scales An attitude measure that places a single adjective in the center of an even-number range of numerical values Example: Research Methodology +3 +2 +1 Exciting -2 -3

41. Graphic Rating Scales An attitude measure consisting of a graphic continuum that allows respondents to rate an object by choosing any point on the continuum

42. Rank-Order Scales Scales in which the respondent compares one item with another or a group of items against each other and ranks them. Example: handout

43. Most important skills Adaptability to change Problem identification Listening skills Written communication Leadership Informal Oral communication Analytical thinking/problem solving Time management Coping with stress/job pressures Interpersonal relations Formal oral presentations

44. Most important skills Adaptability to change Problem identification Listening skills Written communication Leadership Informal Oral communication Analytical thinking/problem solving Time management Coping with stress/job pressures Interpersonal relations Formal oral presentations 8961 24 4235 710 117 98 1011 Managers Business grads

45. Paired Comparison Scales Respondent is presented with two objects and is asked to pick the preferred. Example: Which type of cookie do you prefer __ chocolate chip __ oatmeal __ I do not have a preference between these two

46. Sorting Respondent indicates their attitudes or beliefs by arranging items. Example: Please sort the following cards with pictures of cookies into the following categories Like Dislike Neither like nor dislike

47. Ranking, sorting, rating or choice? How many categories or response positions? Balanced or unbalanced? Forced choice or nonforced choice? Single measure or index? Decisions