Evaluation After presenting the results, I asked students to rate the classroom activity according to whether it (1) was enjoyable to them, (2) was helpful.

Slides:



Advertisements
Similar presentations
Project VIABLE: Behavioral Specificity and Wording Impact on DBR Accuracy Teresa J. LeBel 1, Amy M. Briesch 1, Stephen P. Kilgus 1, T. Chris Riley-Tillman.
Advertisements

A Room With a Cue: Personality Judgments Based on Offices and Bedrooms
Lab 3: Correlation and Retest Reliability. Today’s Activities Review the correlation coefficient Descriptive statistics for Big Five at Time 1 and Time.
Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If.
Personality and Life Satisfaction: A Facet-Level Analysis Ulrich Schimmack Shigehiro Oishi R. Michael Furr David C. Funder.
1]Knoll, N., Burkert, S., & Schwartzer, R. (2006). Reciprocal support provision: Personality as a moderator? European Journal of Personality, 20,
Part II Knowing How to Assess Chapter 5 Minimizing Error p115 Review of Appl 644 – Measurement Theory – Reliability – Validity Assessment is broader term.
STANDARDIZED TESTING MEASUREMENT AND EVALUATION  All teaching involves evaluation. We compare information to criteria and ten make judgments.  Measurement.
Introduction Method  Evaluation of ability to provide social support yielded scores with good internal consistency reliability.  There was moderate agreement.
Social network and support influences on perceived control for exercising 2, 4 or 6 days per week S.N. Fraser 1, T.C. Murray 1,2, W.M. Rodgers 2, & C.
Introduction to Central Tendency One of the most important measures of personality created is The Big Five Inventory. Psychologists in the 1960s first.
Classroom Assessment A Practical Guide for Educators by Craig A
Social Science Research Design and Statistics, 2/e Alfred P. Rovai, Jason D. Baker, and Michael K. Ponton Internal Consistency Reliability Analysis PowerPoint.
Measurement Concepts & Interpretation. Scores on tests can be interpreted: By comparing a client to a peer in the norm group to determine how different.
Measures of Central Tendency
Personality. Defining Some Terms Personality = Psychologists define personality as the reasonably stable patterns of emotions, thoughts, and behavior.
Reliability, Validity, & Scaling
How to Write an APA-Style Research Paper
Standardized Testing (1) EDU 330: Educational Psychology Daniel Moos.
The Psychology of the Person Chapter 7 Trait Approach Naomi Wagner, Ph.D Lecture Outlines Based on Burger, 8 th edition.
Student Engagement Survey Results and Analysis June 2011.
Foundations of Educational Measurement
McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. Educational Research: Fundamentals.
LECTURE 06B BEGINS HERE THIS IS WHERE MATERIAL FOR EXAM 3 BEGINS.
The Dynamic Analog Scale: Using a Single Item to Measure Personality INTRODUCTION When measuring personality traits, personality psychologists typically.
The Genetics Concept Assessment: a new concept inventory for genetics Michelle K. Smith, William B. Wood, and Jennifer K. Knight Science Education Initiative.
User Study Evaluation Human-Computer Interaction.
Tonya Filz & Regan A.R. Gurung University of Wisconsin – Green Bay Abstract As class sizes increase due to stagnating budgets, and as colleges and universities.
Thinking About Psychology: The Science of Mind and Behavior 2e Charles T. Blair-Broeker Randal M. Ernst.
Social Cognitive & Trait Theories
Module 20 Social Cognitive & Trait Theories. SOCIAL COGNITIVE THEORY Definition –says that personality development is shaped primarily by three forces:
Variables, sampling, and sample size. Overview  Variables  Types of variables  Sampling  Types of samples  Why specific sampling methods are used.
Research Methodology Lecture No :24. Recap Lecture In the last lecture we discussed about: Frequencies Bar charts and pie charts Histogram Stem and leaf.
Psychology’s Statistics Module 03. Module Overview Frequency Distributions Measures of Central Tendency Measures of Variation Normal Distribution Comparative.
◦ 125 adolescents (56% girls; 75% Caucasian) and their mothers from the Adolescent Adjustment Project (AAP) ◦ Adolescents’ mean age = (SD=.70) ◦
Attractive Equals Smart? Perceived Intelligence as a Function of Attractiveness and Gender Abstract Method Procedure Discussion Participants were 38 men.
Quantitative SOTL Research Methods Krista Trinder, College of Medicine Brad Wuetherick, GMCTE October 28, 2010.
Psychology 3051 Psychology 305A: Theories of Personality Lecture 2 1.
Appraisal and Its Application to Counseling COUN 550 Saint Joseph College For Class # 3 Copyright © 2005 by R. Halstead. All rights reserved.
Personality and Substance Use Behaviors Associations of the Big Five traits with adolescent substance use, demonstrated through analysis of the MOFAM data.
Thinking About Psychology The Science of Mind and Behavior 3e Charles T. Blair-Broeker & Randal M. Ernst PowerPoint Presentation Slides by Kent Korek Germantown.
Learning Objectives Copyright © 2002 South-Western/Thomson Learning Using Measurement Scales to Build Marketing Effectiveness CHAPTER ten.
Introduction Relationship between Extraversion and Delay Discounting of Social Interactions Sara L. Daugherty and Daniel D. Holt University of Wisconsin-Eau.
Research Methodology and Methods of Social Inquiry Nov 8, 2011 Assessing Measurement Reliability & Validity.
Investigating service user ethical priorities in psychological research Rachael Carrick.
Psychometrics. Goals of statistics Describe what is happening now –DESCRIPTIVE STATISTICS Determine what is probably happening or what might happen in.
Measurement MANA 4328 Dr. Jeanne Michalski
Experimental Research Methods in Language Learning Chapter 12 Reliability and Reliability Analysis.
©2011 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Chapter 7 Measuring of data Reliability of measuring instruments The reliability* of instrument is the consistency with which it measures the target attribute.
Psychology 3051 Psychology 305A: Theories of Personality Lecture 4 1.
Chapter 14: Affective Assessment
Chapter 6 - Standardized Measurement and Assessment
◦ th and 11 th grade high school students (54% girls) ◦ 63% Caucasian; 24% African-American; 13% Hispanic; remaining were Asian or “other” ◦ Mean.
CLASS 10. Trait Theories Recall the three definitions of personality lay definition: friendly, interesting, etc. grand theory of psychology (e.g. Freud)
Hypothesis Testing with z Tests Chapter 7. A quick review This section should be a review because we did a lot of these examples in class for chapter.
Measurement Chapter 6. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts.
Project VIABLE - Direct Behavior Rating: Evaluating Behaviors with Positive and Negative Definitions Rose Jaffery 1, Albee T. Ongusco 3, Amy M. Briesch.
Improving Student Engagement Through Audience Response Systems
Midwest Nice, Southern Hospitality, and Coastal Cool
A Preliminary Study of Music Experience, Creativity and Personality
Measuring Athlete Coachability
Reliability and Validity of Measurement
Assessment of Personality
Personality An individual’s characteristic pattern of thinking, feeling, and acting.
Using statistics to evaluate your test Gerard Seinhorst
Thinking About Psychology The Science of Mind and Behavior 3e
Method Results Discussion
Trait perspective.
Humanistic Perspective
Presentation transcript:

Evaluation After presenting the results, I asked students to rate the classroom activity according to whether it (1) was enjoyable to them, (2) was helpful in providing understanding of personality research, and (3) should be used in future classes. Each question was rated on a 7-point Likert-type scale (1 = “not at all,” 4 = “neutral,” 7 = “very much”). Student evaluations of the activity were largely favorable. Mean ratings for enjoyment, understanding, and future use were 5.7, 5.6, and 5.9, respectively. The median ratings were 6 on all items. The distributions of ratings on each question are shown in Figure 2. I left room for open-ended comments or suggestions on the evaluation form. All open- ended comments that were given are shown below Figure 2. Below are a few benefits of the Facebook activity (in my opinion): 1.Most students are involved in social networking, so they are likely to be intrinsically interested in the question of how we use online information to make inferences about each other (and whether our inferences are right). 2.The activity helps reinforce student understanding of the nature of the Big Five traits (i.e., the particular facets that are reflected in each global trait). 3.It provides an opportunity to clarify the distinction between rater consistency and rater accuracy in personality assessment (i.e., agreement does not imply accuracy or truth). 4.The activity was fun and generated a lot of interesting discussion. During the activity, students spontaneously began discussing whether the subject was using “impression management” that might not reflect a true picture of his personality. This gave me an opportunity to discuss Gosling’s distinction between deliberate identity statements and inadvertent behavioral residues. 5.The activity was fairly easy to implement. Results Presented to Class I first presented data on rater consistency in judgments of the Big Five factors. Because only one target subject was rated, rater consistency could not be calculated for each of the traits separately (as is typically done in published research). Instead, I computed Cronbach’s alpha coefficients for ratings over the five traits. Essentially this involves treating each rater as a test item, and each trait as a “subject” to be rated. In this way, Cronbach’s alpha coefficient provides an estimate of the degree to which raters were consistent in their relative rankings of the five traits (e.g., whether the subject was higher in conscientiousness than in openness, etc.). For “close other” ratings, Cronbach’s alpha was.94; for the class ratings, alpha was.96. These results indicated that in both sources of ratings there was high consistency in judgments of the target subject’s relative standing on the five traits. Next I presented data on rater accuracy. Again, because only one target subject was rated, the typical quantitative measures of accuracy could not be used. Instead, I presented error bar charts for each Big Five factor showing (1) the target subject’s self-ratings, (2) the mean “close other” ratings, and (3) the mean class ratings. The standard deviations of the ratings were used to define the error bars. A reference line was displayed at the neutral point (rating of 4). If all 3 sources of ratings were above the neutral point or below the neutral point (within the error limits), this could be viewed as a crude assessment of accuracy. The error bar charts for each of the Big Five are shown in Figure 1. On all factors, “close other” ratings were closer to the target subject’s self-report than were class ratings (which was to be expected). Class ratings were fairly accurate for extraversion, conscientiousness, and openness (all sources of ratings consistently above or below the neutral point), but less so for emotional stability and agreeableness. I showed the class the accuracy data for Facebook ratings provided in Gosling et al. (2007), which is also shown in Figure 1. Their results (based on ratings of 133 target subjects) also indicated less accuracy of ratings for agreeableness and neuroticism (reverse of emotional stability). So the results of the classroom rating activity were broadly compatible with published research. Abstract I describe a classroom activity designed to reinforce student understanding of the nature of the Big Five personality traits and to illustrate recent research on how people make personality judgments on the basis of environmental cues, including cues provided in online (virtual) spaces. Students rated the personality of a subject (unknown by them) based solely on his Facebook page. Class ratings were compared to the target subject’s self-ratings and to ratings of five people who know the target subject well. Most students enjoyed the activity, thought it helped them understand personality research and concepts, and believed that the activity should be used in future classes. A Classroom Activity Illustrating Big Five Personality Judgments from Facebook Cues Jack W. Berry, Samford University Introduction Trait theories are central in research-based approaches to personality psychology. The dominant trait theory today is the Big Five personality theory, which centers on the measurement and predictive value of five broad personality dimensions: extraversion, agreeableness, neuroticism, conscientiousness, and openness to experience. When I teach about trait theories, students seem particularly interested in recent research on the sources of information in everyday life that reflect a person’s personality, such as handshakes, musical preferences, occupational choices, and cues from a person’s living spaces such as offices and bedrooms. Personality researchers have begun to examine whether a person’s “online presence,” such as personal websites and social network sites, provide useful cues for making valid judgments about personality (Back et al., 2010; Gosling, Gaddis, & Vazine, 2007). In these studies, research participants make personality ratings on many target subjects based solely on the cues evident in the targets’ online spaces. Personality ratings are assessed for rater consistency (agreement) and accuracy. Accuracy is typically assessed by comparison to target subjects’ self-reports and to proxy ratings by people who know the targets well. In my Theories of Personality class in the Fall 2012 semester, I adapted methods used by Gosling et al. (2007) in their Facebook research to create a classroom activity to give students the experience of making their own personality judgments about a person unknown to them based on the person’s Facebook page. Although 36 students were enrolled in the course, 32 students were present for the activity. Figure 1. Personality Ratings by Source Figure 2. Student Evaluations of Classroom Activity Student Comments This activity was a great way to get students involved and better understand the concepts we’re studying in class. I enjoyed this activity and liked the idea of comparing the results to Gosling’s research. I thought it was a great activity and something you should definitely do in future classes. ☺ It was very enjoyable but wasn’t extremely helpful. It was helpful because it gave us practice doing something we do on a normal basis. Expand it. A one-off study is less useful than doing 5. If we did more it would be better. Get students to indicate what influenced their decisions (what parts of Facebook page).. Description of Classroom Activity After describing previous research on personality judgments based on living spaces and online cues, I asked the class whether they would like to try making ratings of someone’s personality based only on the person’s Facebook page. The students seemed excited by the prospect. I explained that I had already obtained the self-report ratings of the target subject (my nephew, a recent college graduate), who completed the Ten Item Personality Inventory (TIPI; Gosling, Rentfrow, & Swann, 2003) of the Big Five traits. The TIPI consists of 10 items, each rated on a Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree), with 4 representing the neutral point. Each of the Big Five traits is estimated by the average of two items. Students were already familiar with the TIPI, having completed it themselves earlier. The TIPI is moderately correlated with longer scales of the Big Five but takes only a few minutes to complete and score, which made it ideal for completing in a single class period. I had also obtained TIPI ratings of the target subject from five people who know the subject well (his fiancé, brother, sister, mother, and me, his uncle). I took the average of these ratings as a “close other” estimate of the target’s personality profile. In class, I projected my computer screen so students could view the target subject’s Facebook page. One student volunteered to navigate the page for the class. After 15 minutes, students rated the target subject using the TIPI, which they completed, scored, and returned to me. I conducted several analyses on the data and presented the results the next class meeting. Contact: Jack W. Berry, Department of Psychology, Samford University, 800 Lakeshore Drive, Birmingham, AL 35229; ;