Slides to accompany Weathington, Cunningham & Pittenger (2010), Chapter 8: Creating and Using Assessments, Surveys, and Objective Measures 1

Objectives Purpose of Measurement Creating a Measurement Scale Constructing Interviews, Questionnaires, and Attitude Surveys Question Response Formats Writing Good Questionnaire and Survey Items Determining the Sample Size for a Survey Naturalistic Observation 2

Purpose of Measurement To work with our operational definitions To facilitate consistency in research Issues to overcome: –Mistrust in measurement –Excessive trust in measurement Reification Missing important information 3

Creating Measures Follow the sequence: 1.What questions are you asking? Hypotheses 2.How can you collect the best data? Operational definition + method plan 3.What measurement approach would be most accurate? 4

Interviews Several varieties (Table 8.1) Pros: –Encourage participation, richer answers –Ability to clarify questions Cons: –Expensive (time, $) –Training required –Interpersonal issues may arise 5

Self-Report Surveys Paper-pencil, phone- or internet-based Pros: –Cheap –Easy distribution Cons: –Data quality may suffer –Lack of control over data collection 6

Survey Strategies Use a sufficiently large sample Can increase retention by: –Using a “captive” audience –Making multiple contacts and reminders –Using creative labeling/packaging –Offering incentives/gifts –Facilitating quick completion (good timing, easy format) 7

Question Types Open-ended –Can gather rich data –Responses often incomplete or difficult to interpret –“What symptoms or signs do your recognize in yourself when you are experiencing a great deal of stress?” 8

Question Types Closed-ended/response –Researcher supplies response options: Nominal categories Forced choice Likert Guttman 9

Question Writing Strategies Use an existing measure Single questions/statements Be specific and clear –K.I.S.S. Write with neutrality (avoid bias) Don’t embarrass/anger the participant 10

Question Writing Strategies Make it easy to answer Ask more than one question Try to avoid a response set Avoid full transparency (obviously correct) 11

Determining Survey Sample Size Using formulas, you can estimate the optimal sample size for surveys Formulas differ depending on the type of responses you are gathering (yes/no, Likert, open-ended responses) Your textbook presents formulas for estimating N for surveys with binary outcomes (e.g., “Yes/No” type questions) 12

Naturalistic Observation A.K.A. Field studies, observation, natural experiment Follow same strategies as with survey construction: –What behaviors to observe? –How defining these behaviors? –What data would be best to gather? 13

Participant Observation Joining the group to learn about its functioning or about phenomena in that environment Ethnographic approach Can be difficult to stay objective Can be risky May lead to criticism 14

Observational Data Frequency –Counting repetition in specified time span Duration –Length of time behavior lasts Interval –How long between behaviors Intensity –How strong was the behavior or stimulus 15

Observational Research Issues Interrater reliability –Index of consistency across multiple raters Cohen’s kappa for level of agreement (nominal/ordinal scales) r for duration or interval (interval and ratio scales) –Improved by using “blind” raters, training, video recording, avoiding reactivity 16

What is Next? **instructor to provide details 17