Presentation is loading. Please wait.

Presentation is loading. Please wait.

Using Assessment Items to Study Students’ Difficulty Reading and Interpreting Graphical Representation of Distributions Robert delMas, Joan Garfield, and.

Similar presentations


Presentation on theme: "Using Assessment Items to Study Students’ Difficulty Reading and Interpreting Graphical Representation of Distributions Robert delMas, Joan Garfield, and."— Presentation transcript:

1 Using Assessment Items to Study Students’ Difficulty Reading and Interpreting Graphical Representation of Distributions Robert delMas, Joan Garfield, and Ann Ooms University of Minnesota

2 OVERVIEW Motivation for paper-and-pencil items to assess understanding of graphs ARTIST Project Development of 11 topic scales and CAOS test Spring 2005 study 3 types of Literacy and Reasoning items Student responses to items Implications for Instruction

3 Assessment of Understanding About Distribution Research to date has used observation, interview, tasks,and problems These studies provide rich data on reasoning about Distribution These approaches not easily administered by instructors in the classroom Need for paper-and-pencil assessments

4 Assessment Resources Tools for Improving Statistical Thinking ARTIST Project Past year and a half: Develop and validate assessments Advisory group identified 11 topic areas and CAOS 8 to 12 multiple choice items for each topic 37 multiple choice items for CAOS Advisors met twice to review assessment items March 2004: Suggested changes, identified missing areas Summer 2004: Items revised, new items generated August, 2004: Reviewed second time, judged validity Final versions of topic scales and CAOS produced Piloted topic scales and CAOS fall 2004 Started collecting data spring 2005

5 Spring 2005 PARTICIPANTS Instructor and Student Characteristics 11 Topic Scales (555 Students) 4 High Schools, 205 AP Students 15 Colleges and Universities, 350 Students CAOS (909 Students) 3 High Schools, 97 AP Students 15 Colleges and Universities, 812 Students Advanced Placement (AP) students are high ability; College more heterogeneous

6 Items on Literacy and Reasoning about Distribution Three types Literacy (2 tasks) Read a graph Interpret a graphic display Reasoning (2 tasks) Match graph to description Match graphs of same data Literacy and Reasoning

7 Literacy: Reading a graph

8 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? Literacy: Reading a graph

9 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. 1 b. 2 c. 3 d. 4 Literacy: Reading a graph

10 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d Literacy: Reading a graph

11 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d Literacy: Reading a graph

12 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d ITEM 2: How many people took the test and have scores represented in the graph? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. 5 b. 10 c. 20 d. 30 Literacy: Reading a graph

13 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d ITEM 2: How many people took the test and have scores represented in the graph? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d Literacy: Reading a graph

14 ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d ITEM 2: How many people took the test and have scores represented in the graph? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d Literacy: Reading a graph

15 ITEM 3: What do the numbers on the vertical axis represent? Literacy: Reading a graph

16 ITEM 3: What do the numbers on the vertical axis represent? Literacy: Reading a graph ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. Independent variable b. Scores on the test c. Dependent variable d. Number of Students

17 ITEM 3: What do the numbers on the vertical axis represent? Literacy: Reading a graph ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. Independent variable b. Scores on the test c. Dependent variable d. Number of Students

18 Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints)

19 Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints) ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. 6 b. 7 c. 12 d. 13 e. Can’t be determined

20 Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints) ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a b c d e. Can’t be determined

21 Literacy: Interpreting a Graphic Display

22 ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class?

23 Literacy: Interpreting a Graphic Display ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. A majority of women in the class do not feel that they will succeed in statistics although a few feel confident about succeeding. b. The women in the class see themselves as having lower aptitude for statistics than the men in the class. c. If you remove the three women with the highest ratings, then the result will show an approximately normal distribution.

24 Literacy: Interpreting a Graphic Display ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. A majority of women in the class do not feel that they will succeed in statistics although a few feel confident about succeeding b. The women in the class see themselves as having lower aptitude for statistics than the men in the class c. If you remove the three women with the highest ratings, then the result will show an approximately normal distribution

25 Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable.

26 Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable. ResponseHigh School (N = 97) College- Level (N = 796) All (N = 893) a. The bars go from 3 to 10, increasing in height to 7, then decreasing to 10. The tallest bar is at 7. There is a gap between three and five. b. The distribution is normal, with a mean of about 7 and a standard deviation of about 1. c. Most students seem to be getting enough sleep at night, but some students slept more and some slept less. However, one student must have stayed up very late and got very few hours of sleep. d. The distribution of hours of sleep is somewhat symmetric and bell-shaped, with an outlier at 3. The typical amount of sleep is about 7 hours and overall range is 7 hours.

27 Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable. ResponseHigh School (N = 97) College- Level (N = 796) All (N = 893) a. The bars go from 3 to 10, increasing in height to 7, then decreasing to 10. The tallest bar is at 7. There is a gap between three and five b. The distribution is normal, with a mean of about 7 and a standard deviation of about c. Most students seem to be getting enough sleep at night, but some students slept more and some slept less. However, one student must have stayed up very late and got very few hours of sleep d. The distribution of hours of sleep is somewhat symmetric and bell-shaped, with an outlier at 3. The typical amount of sleep is about 7 hours and overall range is 7 hours

28 Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond?

29 Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. Agree, it looks pretty symmetric if you ignore the outlier. b. Agree, most distributions are bell-shaped. c. Disagree, it looks more skewed to the left. d. Disagree, it looks more skewed to the right. e. Disagree, it looks more bimodal.

30 Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond? ResponseHigh School (N = 197) College- Level (N = 345) All (N = 542) a. Agree, it looks pretty symmetric if you ignore the outlier b. Agree, most distributions are bell-shaped c. Disagree, it looks more skewed to the left d. Disagree, it looks more skewed to the right e. Disagree, it looks more bimodal

31 REASONING: Matching a Graph to a Description

32 Match each description to the appropriate histogram below. III III IV

33 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 8: A set of quiz scores where the quiz was very easy.

34 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 8: A set of quiz scores where the quiz was very easy. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

35 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 8: A set of quiz scores where the quiz was very easy. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

36 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below.

37 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

38 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

39 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below.

40 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

41 REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III III IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below. ResponseHigh School (N = 97) College- Level (N = 797) All (N = 894) I II III IV

42 REASONING: Matching Graphs of the Same Data

43 ITEM 11: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. On the right is the distribution of this variable for the 30 women in the class. Which of the following boxplots represents the same data set as the histogram shown above?

44 REASONING: Matching Graphs of the Same Data ITEM 11: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. On the right is the distribution of this variable for the 30 women in the class. Which of the following boxplots represents the same data set as the histogram shown above?

45 REASONING: Matching Graphs of the Same Data ITEM 12: The following graph shows a distribution of hours slept last night by a group of college students. Which box plot seems to be graphing the same data as this histogram?

46 REASONING: Matching Graphs of the Same Data ITEM 12: The following graph shows a distribution of hours slept last night by a group of college students. Which box plot seems to be graphing the same data as this histogram?

47 LITERACY and REASONING

48 ITEM 13: A local running club has its own track and keeps accurate records of each member's individual best lap time around the track, so members can make comparisons with their peers. Here are graphs of these data. Which of the above graphs allows you to most easily see the shape of the distribution of running times?

49 LITERACY and REASONING ITEM 13: A local running club has its own track and keeps accurate records of each member's individual best lap time around the track, so members can make comparisons with their peers. Here are graphs of these data. Which of the above graphs allows you to most easily see the shape of the distribution of running times? High School (N = 197) College- Level (N = 345) All (N = 542) A B C All of the above

50 LITERACY and REASONING ITEM 14: A baseball coach wanted to get an idea of how well his team did during the past baseball season. He recorded the proportion of hits obtained by each player based on their number of times at bat as shown in the table below.

51 LITERACY and REASONING ITEM 14 continued: Which of the following graphs gives the best display of the distribution of proportion of hits in that it allows the coach to describe the shape, center and spread of the data?

52 LITERACY and REASONING ITEM 14 continued: Which of the following graphs gives the best display of the distribution of proportion of hits in that it allows the coach to describe the shape, center and spread of the data?

53 Summary of Results Most students were able to: Interpret simple histograms Match different graphs of the same data Difficulty reading histograms when Bin widths covered several values End points not marked for each bar Tendency to interpret histograms as bar graphs Represent each case with an individual bar Bar Height = Magnitude

54 Implications for Instruction Activities that helps students map individual cases to histogram bars “fuse” command in Tinkerplots Dot plots from magnitude bars (Cobb & McClain) Opportunities to create different types of graphs of the same data Map information from one type to the other From bar graphs to histograms Draw attention to changing role of axes

55 What have we missed? What is missing from our interpretations and explanations?


Download ppt "Using Assessment Items to Study Students’ Difficulty Reading and Interpreting Graphical Representation of Distributions Robert delMas, Joan Garfield, and."

Similar presentations


Ads by Google