1. Robert delMas (Univ. of Minnesota, USA) Ann Ooms (Kingston College, UK) Joan Garfield (Univ. of Minnesota, USA) Beth Chance (Cal Poly State Univ., USA) ASSESSING STUDENTS’ STATISTICAL REASONING

2. https://app.gen.umn.edu/artist/

3. Statistical Literacy, Reasoning and Thinking Statistical literacy involves understanding and using the basic language and tools of statistics: knowing what statistical terms mean, understanding the use of statistical symbols, and recognizing and being able to interpret representations of data. Statistical reasoning is the way people reason with statistical ideas and make sense of statistical information. Reasoning means understanding and being able to explain statistical processes, and being able to fully interpret statistical results. Statistical thinking involves an understanding of why and how statistical investigations are conducted. This includes recognizing and understanding: The entire investigative process. How models are used to simulate random phenomena. How data are produced to estimate probabilities. How, when, and why existing inferential tools can be used. Utilizing the context of a problem to plan and evaluate investigations.

4. ARTIST Resources Page - Alternative Assessments - Projects

5. Alternative Assessments - References on Student Projects

6. Alternative Assessments - Examples of Student Projects

7. ARTIST Resource Page - References on Assessment

8. References on Assessment in Statistics Education

9. ARTIST Topic Tests There are 11 scales, each consisting of 7-15 multiple-choice items, that can be administered online. Reports of student performance can be downloaded. Our goal was to develop high quality, valid and reliable scales that can be used for a variety of purposes (e.g., research, evaluation, review, or self-assessment). TOPICS Data Collection ( data types, types of study, study design ) Data Representation (choose appropriate graphs, interpret graphs) Measures of Center (estimate, when to use, interpret, properties) Measures of Spread (estimate, when to use, interpret, properties) Normal Distribution (characteristics, empirical rule, areas under the curve) Probability (interpret, independence, relative frequency, simulation) Bivariate Quantitative Data (scatterplots, correlation, descriptive and inferential methods, outliers, diagnostics, influential observations) Bivariate Categorical Data (two-way tables and chi-square test, association) Sampling Distributions (types of samples, sample variability, sampling distributions, Central Limit Theorem) Confidence Intervals (interpret, confidence level, standard error, margin of error) Tests of Significance (hypothesis statements, p-values, Type I and II error, statistical and practical significance)

10. ARTIST Test Report

11. Comprehensive Assessment of Outcomes in Statistics (CAOS) Forty item test that can be administered as an online test to evaluate the attainment of desired student outcomes. CAOS items are designed to represent the big ideas and the types of, literacy and reasoning skills deemed important for all students across first courses in statistics. Unifying focus is on reasoning about variability: in univariate and bivariate distributions, in comparing groups, in samples, and when making estimates and inferences. Not intended to be used exclusively as a final exam or as the sole assessment to assign student grades. CAOS can provide very informative feedback to instructors about what students have learned and not learned in an introductory statistics course (e.g., administered as pretest and posttest). An analysis of responses from a large national sample of students was presented at AERA 2006 (Click the link for ARTIST Publications, Presentations, and Workshops to download a copy of the paper).

12. Accessing the ARTIST Item Database

13. Assessment Builder: Conducting a Search for Items

14. Assessment Builder: Search Results

15. Assessment Builder: Downloading a Test

16. Evaluation of ARTIST Evaluations consisted of Survey of 98 instructors who used ARTIST resources 89 instructors who had not used ARTIST resources Observations of 5 instructors as they used the ARTIST website Interviews with 7 non-users and 7 frequent users Results Content is of high quality and navigation was user-friendly Non-users were not aware of ARTIST or of its purpose ARTIST assessment items were judged to be of the same or higher quality than items from other sources Frequent users of ARTIST resources indicated that they are starting to rethink instruction in order to have a greater impact on students’ statistical reasoning and thinking.

17. Future Plans Create large data sets, along with variable descriptions, that can be downloaded for use in open-ended assessments Provide an online version of the Student Attitudes Towards Statistics (SATS) survey (Shau, 1994) that assesses six components of attitudes toward statistics (affect, cognitive competence, value, difficulty, interest, and effort). Develop a Statistical Thinking Assessment (STA) to assess students’ ability to select appropriate procedures, check conditions and assumptions, and draw correct conclusions. Develop a Statistics Teaching Inventory (STI) to gather information on teachers’ own beliefs about teaching, their instructional practices, and the constraints in which they teach (school and student variables). Collaborate with CAUSE (Consortium for the Advancement of Undergraduate Statistics Education) to use ARTIST assessments and resources in multi-institution statistics education research projects.

18. ARTIST Website https://app.gen.umn.edu/artist Please contact the ARTIST team with any comments and suggestions you have regarding this presentation, or any questions you have about the materials at the ARTIST website. Thank you for your attention.