2009 Pearson Prentice Hall, Salkind. Chapter 6 Methods of Measuring Behavior

2009 Pearson Prentice Hall, Salkind. CHAPTER OVERVIEW Tests and Their Development Types of Tests Observational Techniques Questionnaires

2009 Pearson Prentice Hall, Salkind. SOME IMPORTANT FIRST CONSIDERATIONS Is the tool you propose to use reliable and valid? Base your choice of research tools on how you have asked the research question.

2009 Pearson Prentice Hall, Salkind. CHAPTER OBJECTIVES - STUDENTS SHOULD BE ABLE TO: List five reasons why tests are useful. Discuss the various types of tests and how they are used. Conduct an item analysis identifying the discrimination and difficulty indices for each item in a test. Explain the difference between the discrimination index and the difficulty index. List the various techniques used to record behavior. Write questions using a Thurstone scale and a Likert scale. List the factors to consider in order to make questionnaires successful.

2009 Pearson Prentice Hall, Salkind. TESTS AND THEIR DEVELOPMENT

2009 Pearson Prentice Hall, Salkind. TESTS AND THEIR DEVELOPMENT A test should measure the nature and extent of individual differences A good test differentiates people based on their true scores

2009 Pearson Prentice Hall, Salkind. WHY USE TESTS? To help determine outcomes of experiments To diagnose strengths and weaknesses To assist in placing individuals in appropriate programs To assist in selecting applicants To evaluate a program’s effectiveness

2009 Pearson Prentice Hall, Salkind. TYPES OF TESTS

2009 Pearson Prentice Hall, Salkind. TYPES OF TESTS What It Does Achievement TestAssesses an individual’s knowledge of a specific area Attitude TestAssesses an individual’s feelings about an issue Personality TestAssesses stable individual behavior patterns

2009 Pearson Prentice Hall, Salkind. ACHIEVEMENT TEST SOURCES Standardized  Commercially prepared for wide use  Scoring instructions included Researcher-made  Designed by user for specific purpose  Scoring instructions specific to test

2009 Pearson Prentice Hall, Salkind. ACHIEVEMENT TESTS REFERENCING What Comparisons Do Tests Make? Norm-referenced—individual’s scores are compared to results from a larger group Criterion-referenced—individual’s scores are compared to defined performance standards

2009 Pearson Prentice Hall, Salkind. MULTIPLE-CHOICE ACHIEVEMENT ITEMS Anatomy of a Multiple-Choice Item 12. Intelligence tests that are given to preschool children a. favor middle-class children b. have questionable construct validity c. are based on motor skills d. are no fun at all STEM Clearly written CORRECT ANSWER DISTRACTERS Should be plausible (b & c), not easily ruled out (d)

2009 Pearson Prentice Hall, Salkind. TO USE OR NOT TO USE? Advantages  Versatile  Easy to score  Simple to take  Poor writers not penalized  Good items used again  Good distracters are diagnostic  Hard to fake correct answer Disadvantages  Limit student’s options  No opportunity to practice writing  Some people don’t do well on them  Limits content to be assessed  Items must be well written

2009 Pearson Prentice Hall, Salkind. ITEM ANALYSIS: HOW TO TELL IF YOUR ITEMS WORK Questions should discriminate those who know the material from those who don’t Item analysis provides two measures of a question’s ability to discriminate  Difficulty index  Discrimination index

2009 Pearson Prentice Hall, Salkind. COMPUTING INDICES First Steps 1. Rank scores from highest to lowest 2. Choose top 27% of scores for “high” group 3. Choose bottom 27% of scores for “low” group

2009 Pearson Prentice Hall, Salkind. COMPUTING INDICES 12. Intelligence tests that are given to preschool children a. favor middle-class children b. have questionable construct validity c. are based on motor skills d. are no fun at all Item AlternativeABCDTotal High Group (n = 41) Low Group (n = 41) Total

2009 Pearson Prentice Hall, Salkind. COMPUTING INDICES Difficulty index Proportion who answered item correctly D = NC h + NC l T Discrimination index Proportion in high group who answered correctly minus proportion in low group who answered correctly D = NC h - NC l (.5)T NC h = number of people in high group answering correctly NC l = number of people in low group answering correctly T = total number of people in high and low groups

2009 Pearson Prentice Hall, Salkind. RELATIONSHIP BETWEEN ITEM DISCRIMINATION AND DIFFICULTY 0% % 100% 0 Difficulty Level Discrimination Level Perfect discrimination when: ½ gets item right, ½ gets item wrong & ½ right is in upper half, ½ wrong is in lower half

2009 Pearson Prentice Hall, Salkind. ATTITUDE TESTS Assess an Individual’s Feelings About a Topic ItemAgreeNo Strong Feeling Disagree The day before Thanksgiving should be a holiday. ____ ____ ____ Final exams should be elective.____ The dining room should serve gourmet food. ____ ____ ____ My parents don’t appreciate how smart I am. ____ ____ ____ My professors don’t appreciate how smart I am, either. ____ ____ ____

2009 Pearson Prentice Hall, Salkind. THURSTONE SCALES Method of Equal Appearing Intervals Statements are written indicating an attitude toward a topic Judges rank the statements from least favorable to most favorable Statements receiving consistent ratings are given the average score A set of statements is selected that covers the entire range of attitudes

2009 Pearson Prentice Hall, Salkind. THURSTONE SCALES Administration Respondents check items with which they agree  Well-formed attitudes are indicated by consistently checking either high or low items  Poorly-formed or inconsistent attitudes are indicated by inconsistent patterns or by checking off many neutral items

2009 Pearson Prentice Hall, Salkind. LIKERT SCALES Statements are written indicating an attitude toward a topic Items with clearly positive or negative attitudes are selected Statements are listed with a space for respondent to indicate degree of agreement

2009 Pearson Prentice Hall, Salkind. ItemRating Government has no business funding child care programs. SDDUASA Child care should be supported by federal, state, and local tax dollars. SDDUASA A LIKERT SCALE Directions: Indicate to what extent you agree or disagree with the statements listed below by circling one of the following: SA means that you strongly agree with the statement (value = 5) A means that you agree with the statement (value = 4) U means that you are undecided about the statement (value = 3) D means that you disagree with the statement (value = 2) SD means that you strongly disagree with the statement (value = 1)

2009 Pearson Prentice Hall, Salkind. SCORING LIKERT RESPONSES Method of Summated Ratings Items are weighted Weights of unfavorable items are reversed Average score is computed ItemRating Government has no business funding child care programs. SDDUASA Child care should be supported by federal, state, and local tax dollars. SDDUASA

2009 Pearson Prentice Hall, Salkind. PERSONALITY TESTS Projective tests  Present respondent with ambiguous stimulus Structured tests  Questions are objective

2009 Pearson Prentice Hall, Salkind. OBSERVATIONAL TECHNIQUES

2009 Pearson Prentice Hall, Salkind. OBSERVATIONAL TECHNIQUES Researcher observes and records Does not interfere with behavior

2009 Pearson Prentice Hall, Salkind. TECHNIQUES FOR RECORDING BEHAVIOR TechniqueHow it WorksExample Duration recordingThe researcher records the length of time that a behavior occurs. How much time is spent in verbal interaction between two children? Frequency recordingThe researcher records the number of times a behavior occurs. How often are questions asked? Interval recordingThe researcher observes a subject for a fixed amount of time. Within a 60-second period, how many times do members of the group talk to another person? Continuous recordingThe researcher records everything that happens. During a 1-hour period, all the behavior of a 6-year- old boy is recorded.

2009 Pearson Prentice Hall, Salkind. OBSERVATIONAL TECHNIQUES? BE CAREFUL! Pitfalls to Avoid Observer effects Observer bias Fatigue Changing definitions

2009 Pearson Prentice Hall, Salkind. QUESTIONNAIRES

2009 Pearson Prentice Hall, Salkind. QUESTIONNAIRES What they are  Paper and pencil tests with structured questions  Self-administered

2009 Pearson Prentice Hall, Salkind. QUESTIONNAIRES Advantages  Can be mailed out  Survey broad geographic area  Cheaper than one-on-one interview  Respondents may be more honest  Data easy to share with other researchers Disadvantage  Low return rate

2009 Pearson Prentice Hall, Salkind. BASIC ASSUMPTIONS OF A QUESTIONNAIRE Does not make unreasonable demands upon the respondent Does not have a hidden purpose Requests information that respondents presumably have Contains interesting questions Does not request information that could be obtained by other means

2009 Pearson Prentice Hall, Salkind. THE QUESTIONS The questionnaire contains questions that can be answered The questionnaire contains questions that are straightforward

2009 Pearson Prentice Hall, Salkind. THE FORMAT The presentation is attractive, professional, and easy to understand Questions and pages are clearly numbered Directions are clear and explicit Questions are objective Questions are ordered sensibly Transitions are used from one topic to the next

2009 Pearson Prentice Hall, Salkind. THE IMPORTANCE OF USING A COVER LETTER Informs the recipient about the research Establishes the importance of the research Makes the recipient a part of the research

2009 Pearson Prentice Hall, Salkind. HAVE WE MET OUR OBJECTIVES? CAN YOU: List five reasons why tests are useful? Discuss the various types of tests and how they are used? Conduct an item analysis identifying the discrimination and difficulty indices for each item in a test? Explain the difference between the discrimination index and the difficulty index? List the various techniques used to record behavior? Write questions using a Thurstone scale and a Likert scale? List the factors to consider in order to make questionnaires successful?