Presentation on theme: "Scales and Measurement Chapters 11, 12. Constructs and Measurement Construct Development Identifying and defining what is to be measured A construct is."— Presentation transcript:
Constructs and Measurement Construct Development Identifying and defining what is to be measured A construct is a hypothetical variable composed of different elements that are thought to be related (e.g., 5 questions tapping brand loyalty) Measurement Figuring out how to measure what you want to measure Measure needs to be reliable and valid
Different Types of Reliability Internal Reliability Extent to which items on a scale “hang together” or are correlated with one another Cronbach’s alpha (covered in last class) Split-half reliability (split measure into halves, correlate) Test-Retest Reliability Extent to which scores are stable over time Have people complete questionnaire twice and correlate scores
Validity: Overview of Key Definitions Validity (in general) The extent to which conclusions drawn from a study are true Internal Validity When a researcher can clearly identify cause and effect relationships (i.e., there are no confounds) External Validity The extent to which what you find in your study can be generalized to your target population Construct Validity Extent to which your constructs of interest (e.g., sensation seeking) are accurately and completely identified (measured) In other words, the extent to which you are actually measuring what you say you are measuring (your sensation seeking scale really does measure the true construct of sensation seeking)
Other Forms of Validity Content Validity (Face Validity) Extent to which a measure is appropriate according to experts in the domain of interest Concurrent Validity (Convergent Validity) Extent to which one measure of a construct overlaps with other similar measures of that construct Discriminant Validity Extent to which a measure of one construct does not overlap with measures of different constructs Predictive Validity Extent to which a measure of a construct can predict theoretically- relevant outcomes Nomological Validity How a construct fits within a broader set of related constructs
Key Idea To develop reliable and valid measures which we can subsequently (and appropriately) use in statistical analyses, we must understand: Properties of scales How to design good questions that do not lead to biased or inconsistent responses
Measurement/Scaling Properties Assignment You can assign objects to categories Order (Magnitude) You can order objects in terms of having more or less of some quality Distance (Equal Intervals) The distance between adjacent points on the scale is identical Origin (Absolute Zero Point) Zero “means something” (absence of a given quality)
Types of Scales Nominal Scale Has Assignment Only (Political Party) Ordinal Has Assignment, Order (Rank Order of Finish in a Race) Interval Has Assignment, Order, Equal Intervals (Temperature) Hybrid Ordinally-Interval Scale Like an ordinal scale, but researcher “pretends” it is an interval scale (e.g., assumes 1 to 7 scale is an interval scale); commonly used in questionnaires Ratio Has Assignment, Order, Equal Intervals, Absolute Zero (Number of Cars)
What Type of Scale? Number of Sweaters Purchased This Year? _______ What is Your Ethnicity? To what extent do you agree or disagree that Congress should have approved the $700 bailout? (1 = strongly disagree to 7 = strongly agree) Please rank the following issues from most to least important (Iraq, Health Care, Economy, Environment) What is your income? (5-10k; 11-15k; 16-20k; 20-25k; 25-30k)
On the Importance of Attitudes I think McCain has a good grasp of national security issues I feel a strong connection with Obama I am inclined to vote for McCain I’m planning on voting for Obama I think Obama’s Plan for Health Care Reform is a good one I believe both candidates bring strengths to the table I feel McCain would be a good leader
Three Components of Attitudes Cognitive Component How a person thinks about an attitude object (product, issue, candidate, idea) Affective Component How a person feels about an attitude object Behavioral – A person’s behavioral predisposition to respond to an attitude object in a certain way
Three Components of Attitudes? I think McCain has a good grasp of national security issues I feel a strong connection with Obama I am inclined to vote for McCain I’m planning on voting for Obama I think Obama’s Plan for Health Care Reform is a good one I believe both candidates bring strengths to the table I feel McCain would be a good leader
Measuring Attitudes While attitudes not perfect predictors of behavior, still very important We need to understand how to measure attitudes accurately Today, we’ll look at some creative ways to measure attitudes And, some standard approaches taken in marketing research As we do this, think about the types of questions we might want to ask in our restaurant concepts study
Creative Measures of Attitudes Projective techniques (partially structured) Physiological (GSR; Heart Rate; Blood Pressure) Functional magnetic resonance imaging (fMRI) Flushing toilets Bogus pipeline Lost letters and emails* Implicit association test (IAT)*
You’ve Got Mail! Dear Peter Price, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We are happy to inform you that you have been selected to receive a Glassner Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail! Dear Mohammed Hameed, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We are happy to inform you that you have been selected to receive a Glassner Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail! Dear Mohammed Hameed, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition four four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We regret to inform you that you have not been selected to receive a Glasser Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail! (Method) Bushman & Bonacci (2004, JESP) Pretested on Arab-American Prejudice 2 weeks later, receive email intended for a different person 2 IVs Intended recipient had European-American (Peter Price) vs. Arab- American name (Mohammed Hameed) Intended recipient won or didn’t win a scholarship (4 years support) DV = willingness to return the email to the sender to indicate it was incorrectly delivered (must be done in 2 days)
GOOD NEWS! YOU WON!! P (returning email) went down as prejudice went up, especially when the intended recipient had an Arab-American name BAD NEWS…YOU DIDN’T WIN. European-American: P (returning email) went down as prejudice went up Arab-American: P (returning email) went up as prejudice went up. People with Arab-American prejudice were actually more likely to return the bad news than the good news email when recipient was Arab American WHAT MIGHT EXPLAIN SUCH ARAB-AMERICAN PREJUDICE?
Terror Management Theory Basic Assumption Humans have developed “anxiety buffering mechanisms” against fear of death (cultural worldview, self-esteem, relationships) Typical Method Write about own death (mortality salience) or dental pain Filler task Judge others from ingroup vs. outgroup Typical Result We judge worldview threatening others more harshly under conditions of mortality salience Support for Bush (vs. Kerry) goes up after MS prime
Implicit Association Task (IAT) An implicit attitude is an attitude which people are not conscious of (or would not want to admit to) but which can be assessed via the associations people hold in their minds between evaluative words (good, bad) and attitude objects (Caucasian, Arab) The IAT is a reaction time (RT) task in which subjects categorize words (positive, negative) and attitude objects (pictures of Caucasians, Arabs) on the left or right side of screen To simplify, some trials involve categorizations that are consistent with an implicit attitude, others are inconsistent with the attitude If there is an implicitly negative attitude toward Arabs, then reaction times should be slower to the inconsistent trials
The following set of trials is consistent with an implicitly negative attitude toward Arabs Reaction times should be relatively fast if there is an implicitly negative attitude toward Arabs
Good Word or Caucasian Bad Word or Arab If you see a good word or a Caucasian, hit “e” (left) If you see a bad word or an Arab, hit “i” (right)
Now Change Categorization Now Change Categorization (The following trials are inconsistent with implicit negative attitude toward Arabs) Reaction times should be slower on these trials if there is an implicitly negative attitude toward Arabs
Bad Word or Caucasian Good Word or Arab Categorization Switched
What are we saying?! Sensitive attitudes (e.g., racial prejudice) are sometimes hard to assess with a self-report scale (due to social desirability concerns) The IAT measures associations (automatic attitudes) Implicitly negative attitudes are not uncommon, but… People can also override these automatic responses with controlled processes (so behavior is not discriminatory) For more information, visit https://implicit.harvard.edu/implicit/ where you can take an IAT for the 2008 election (or other issues)https://implicit.harvard.edu/implicit/
Verbal Rating Scales How satisfied were you with today’s meal? 1234567 ExtremelyDissatisfiedSomewhatNeitherSomewhatSatisfiedExtremely DissatisfiedDissatisfiedSatisfiedSatisfied
Issues in Designing Verbal Rating Scales Many measures taken by marketing researchers are verbal ratings What do we need to consider when we develop verbal rating scales? – Number of categories – Forced vs. unforced scale – Balanced or unbalanced scale – Extent of verbal description – Should response categories be numbered or not – Comparative vs. noncomparative scale – Scale direction
Number of Response Categories? To what extent are you satisfied with your current MP3 player? Most researchers suggest between 5 and 7 categories; for example: 1234567 ExtremelyDissatisfiedSomewhatNeitherSomewhatSatisfiedExtremely DissatisfiedDissatisfiedSatisfiedSatisfied Too few does not give you enough information Too many and it will be hard for people to discriminate between the options (e.g., a 100-point scale)
Forced vs. Unforced Scale? How likely would you be to buy a car manufactured in Brazil? Forced Scale (even number of options forces the respondent to lean one way or the other): 123456 VeryUnlikelySomewhatSomewhatLikelyVery UnlikelyUnlikelyLikelyLikely Unforced scale gives people a neutral option: 1234567 VeryUnlikelySomewhatNeitherSomewhatLikelyVery UnlikelyUnlikelyLikelyLikely
Balanced vs. Unbalanced Scale? How satisfied are you with your current hair stylist? Balanced scale (same number of positive and negative options): 1234567 ExtremelyDissatisfiedSomewhatNeitherSomewhatSatisfiedExtremely DissatisfiedDissatisfiedSatisfiedSatisfied Unbalanced scale (here all options are positive): 1234567 SomewhatVery SatisfiedSatisfied Unbalanced scale can give biased results; unless distribution is naturally skewed to one side of the scale, should use balanced scale
Extent of Verbal Description? The U.S. should invest in wind powered energy Label endpoints or label all options? 1234567 StronglyStrongly DisagreeAgree 1234567 StronglyModeratelySlightlyNeither AgreeSlightlyModeratelyStrongly DisagreeDisagreeDisagreeor DisagreeAgreeAgreeAgree Labeling all options can aid in interpretation.
Should Categories be Numbered? Toyota is an Environmentally Friendly Company StronglyModeratelySlightlyNeither AgreeSlightlyModeratelyStrongly DisagreeDisagreeDisagreeor DisagreeAgreeAgreeAgree 1234567 -3-2-10123 Numbers can help respondents understand scale 1 to 7 scale quite common But -3 to +3 can help interpretation of scale (disagree is negative, agree is positive); it may, however, overemphasize negativity Judgment call; pretesting both scales could help identify problems Should we have numbers here?
Comparative vs. Noncomparative? Noncomparative question How would you evaluate Mintifresh toothpaste? Comparative question Compared to your current brand, how would you evaluate Mintifresh toothpaste? Comparative questions establish the referent and can be useful if you need to know how your product compares to a specific competitor or the customer’s current brand Noncomparative have the advantage of allowing the respondent to create their own referent, which can potentially improve accuracy
Direction of Scale? Typical direction (lower values, negative connotation on left): StronglyModeratelySlightlyNeither AgreeSlightlyModeratelyStrongly DisagreeDisagreeDisagreeor DisagreeAgreeAgreeAgree 1234567 Some scales are not valenced, so must be careful about positioning. Here we see a semantic differential scale, with amusing positioning: Unpleasant-2-1012Pleasant Flimsy-2-1012Sturdy Male-2-1012Female Hmmm…this arrangement suggests that males are to be evaluated negatively; must be careful in designing scales so as not to bias results
Other Types of Scales Rank order the restaurants above where 1 = most preferred, 3 = least preferred
Rank-Order and Paired Comparison Techniques Rank-order technique Advantages: easy to understand, typically what we do in real- life (vs. ratings) Disadvantage: it may be that a person dislikes all of the options, so ranking not that informative Paired comparisons Take n products, compare each one to every other (in each pair, pick the one you prefer) Gives you direct comparisons, but # paired comparisons can be very large as number of products to be compared increases
Semantic Differential Scale Please provide your impression of Politician X: Dumb______________________X__Smart Cold_________________X_______Warm Boring______________________X__Funny Mean_________________X_______Nice Please provide your impression of Politician Y: Dumb_______X_________________Smart Cold__X______________________Warm Boring____________X___________Funny Mean_______X_________________Nice Profile Analysis of Politician X vs. Y: Dumb_________________________Smart Cold_________________________Warm Boring_________________________Funny Mean_________________________Nice X Y
Stapel Scale Please provide your impression of Shoe Store X. (Use negative numbers if you feel it is inaccurate; positive numbers if you feel it is accurate): -5-4-3-2-1Fast Service12345 -5-4-3-2-1Friendly12345 -5-4-3-2-1Knowledgeable12345 Can draw comparative profile analysis (e.g., of various shoe stores) as we did with semantic differential scale (previous slide)
Questionnaire Design Chapter 13 Some Key Issues
Questions: Common Pitfalls Incomprehensible When you consider a new stadium, is it possible that part of your determining factors might rest on the fact that you sometimes choose to forgo entertainment in favor of more pedestrian activities like walking your dog? Unanswerable Will building a new stadium in Boise cost too much? Leading Wouldn’t you agree that it is a great idea to build a new multiuse facility in Boise? Double-barreled Do you think the old stadium needs to be replaced and a new stadium should be built downtown?
Purpose of Questions Descriptive Describe characteristics of your sample Predictive (Hypothesis Testing) What demographic factors are associated with support for the stadium? Every question should be designed to provide useful information What is our primary goal in the stadium study? What questions are we trying to address? How do our questions stack up? Good, bad questions? Do they each answer a question? Descriptive? Predictive?
Arrangement of Questions: Flowerpot Approach Exhibit 13.5