1. Research design

2. Four Steps Formulate a research purpose Design the study
Select instrumentation for Y and operationalize X
Conduct the study

3. Experimental Design
In experiments, researchers give treatments in order to observe their effects on participants’ behavior Classic design for exploring cause-and-effect relationships Pretest-posttest randomized control group design

4. Pretest-Posttest Randomized Control Group Design
Group A
(Experimental Group)
Experimental Treatment
Assign participants at random to groups according to stimulus
PRETEST
Collect demographic data
TEST
Show stimulus
Manipulate cause (X)
POSTTEST
Measure effect (Y)
Group B
(Control Condition)
Control Condition

5. Experimental Design BASIC TERMS Randomization Control Reliability
Validity

6. Randomization
The assignment of subjects to conditions or levels of an independent variable by the investigator or by a natural process in the field
Treatment or control group assignment
Each subject in the study had an equal chance or probability of falling into each group

7. Control
When the values of the independent variable are held at a base or comparison level.
Used in reference to the control group. Often it is untouched or untreated in an investigation, although they may receive some standard or currently existing treatment or condition.

8. Given these operational definitions, …
Nice guy—someone who has a good personality, is agreeable and eager to please others and waits for sex
Jerk guy—someone who is exciting, attractive, charming and sexually aggressive

9. And these treatment groups ….
Scenario 1
Scenario 2

10. What should the Control Scenario look like?
Thursday evening you meet a friend you have not seen since middle school at a mutual friend’s party and spend the entire evening engaged in lively and interesting conversation. Before the two of you leave the party, your friend asks for your telephone number and you give it to her. The next day, your friend calls to let you know how great it was to spend time catching up the night before and asks you to join her for pizza. You agree to meet her at a neighborhood place the following evening (Saturday night). You have a ball. The food is great and you and your friend enjoy lots of humorous conversation. At the end of the night, you walk together to your cars, exchange social media contact information and promise to keep in touch.

11. Reliability When tests yield consistent results/scores
Statistic is called the reliability coefficient
Seek instrumentation with values >=.70

12. Validity Accurate description of your variables
Content validity—based on expert’s judgments of the appropriateness of the contents of a scale/test for a particular purpose (i.e., is it contemporaneous?)
Construct validity—the extent to which an instrument yields scores/results that are consistent with what is known about what the instrument is designed to measure (i.e., is it multi-dimensional?)

13. Reliability and Validity

14. INSTRUMENTATION

15. Basic Design for Pretest-Posttest Experiment
Measure demographics
Measure media consumption (heavy/light user)
POSTTEST
Manipulation check (measure the experimental/control treatment)—attention, retention and resonance
Measure the dependent variable—media perceptions, self efficacy and self regulation

16. Where do you find existing instrumentation?
Look in tables in the METHODS section of an article
Look in the APPENDICIES of articles, masters theses or dissertations
Look in METHODS section for a reference, then check the BIBLIOGRAPHY
Check MEASURES links on my website
Check books like Communication Research Measures

17. Creating your own instrument
DO’s
KEEP IT SIMPLE, STUPID in language and construction
Focus on the current, the specific and/or the real
Maximize variation
All questions must be exclusive; all options must be exhaustive
DON’Ts
Avoid double negatives
Avoid biased or leading questions
Avoid “Agree--Disagree” options
Avoid ranking—use ratings instead
Avoid a middle option
Use “I don’t know” and “N/A” sparingly

18. Given these operational definitions, …
Nice guy—someone who has a good personality, is agreeable and eager to please others and waits for sex
Jerk guy—someone who is exciting, attractive, charming and sexually aggressive

19. How would you create a manipulation check?
On a scale of 1-10, where 1=NOT AT ALL and 10=VERY WELL, how well do the following words describe the hypothetical person you went out with on Saturday night?
ATTRACTIVE
AGREEABLE
FUN
TALKATIVE

20. Modifying your instrument
Changing the wording of the items
Changing the scaling of the items—note reliability and validity
NOMINAL
ORDINAL
INTERVAL
RATIO
Dependent variable must be interval or ratio (continuous)

21. Questions from ESS main questionnaire
A1. On an average weekday, how much time, in total, do you spend watching television? 00 01 02 03 04 05 06 07 88
No time
at all
Less than
½ hour
to 1 hour
More than
1 hour, up
to1½ hours
1½ hours,
up to 2 hours
2 hours, up
to 2½ hours
2½ hours, up
to 3 hours
More than 3 hours
Don’t know
A2. And again on an average weekday, how much of your time watching television is spent watching news or programs about politics and current affairs?
Questions also for listening Radio (A3, A4) and reading Newspaper (A5,A6)

22. Modified EES Questions for Nominal Variable
I spend LESS THAN 3 hours a day
(Check as many as apply)
updating my Facebook wall
tweeting on Twitter
posting on my blog
Posting on Instagram
I spend MORE THAN 3 hours a day
(Check as many as apply)
updating my Facebook wall
tweeting on Twitter
posting on my blog
Posting on Instagram

23. Modified EES Questions for Ordinal Variable
On an average weekday, how much time to you spend on Facebook?
(Circle one below)
Less than hour 1 hour 2 hours 3 hours hours
On an average weekday, how much time do you spend on Facebook reading your news feed? (Circle one below)
On an average weekday, how much time do you spend on Facebook posting on your friends’ walls? (Circle one below)

24. Modified EES Questions for Interval Variable
On an average weekday, I spend ___ hours a day updating my blog.
(Circle the number of your best estimate)
On an average weekday, I spend ___ hours a day reading other people’s blogs.
On an average weekday, I spend ___ hours a day commenting on other people’s blogs.
On an average weekday, I spend ___ hours a day looking for other blogs to follow.

25. Modified EES Questions for Ratio Variable
On an average weekday, how much time to you spend on Twitter? _______________hours a day (Fill in the blank with whole numbers)
On an average weekday, how much time do you spend on Twitter tweeting? _______________ hours a day (Fill in the blank with whole numbers)
On an average weekday, how much time do you spend on Twitter reading your followers’ tweets? _________________ hours a day (Fill in the blank with whole numbers)