Data Quality and Education Sean Fox SERC, Carleton College

Quality = Utility in meeting learning goals Educational uses of data take place in wildly diverse circumstances “Quality” data for one educator may be useless for another No single metric can capture “good for education”.

Data as Illustration Typical textbook visualization, or data table One goal is to help the learner understand the science. Also trying to tell a convincing story; demonstrate that scientific understanding is underpinned by data. Often clean and direct in a way that may be misleadingly atypical?

Data Exploration by Student Mimic ‘real’ process: frame problem, find/model data, ask questions, probe and analyze, revise questions, bring in different data, analyse. Repeat. In education this is usually scaffolded: data selected/formatted/subsetted, questions framed, analysis outlined, tool use explicated. Focus student energy on the bits that matter (which depends on goals) in time available matched to student abilities and knowledge.

Why Do Data Exploration? Prepare students to address real-world complex problems; Develop students’ ability to use scientific methods, including consideration of the values and ethics of working with data; Teach students how to critically evaluate the integrity and robustness of data or evidence and of their consequent interpretations or conclusions; and Provide training in scientific, technical, quantitative, and communication skill Using Data in the Classroom

What Facilitates Success? Data site allows curriculum developers to find and use appropriate data of interest easily. Level of knowledge for use is clear Information is provided on relevance of data to problems of societal, educational or scientific significance Site points to examples of educational use of data Data are reliably available and archived for long term access as appropriate. Interface to, and organization of, the data is end-user friendly Descriptive information (e.g. metadata) should be audience appropriate. Data site allows curriculum developers to ascertain the quality of data and determine the impact of data quality on the certainty of their conclusions. Information is provided about overall data collection, quality, reduction, and limitations. Data site includes sources of error and limitations of collection process as well as inaccuracies/ uncertainties from models/ particular choice of representations. Information about accuracy of individual data sets/points/ analyses is provided Data is provided in ways that facilitate manipulation through a variety of tools. This may include: providing the data in common formats that are accessible via desktop tools (spreadsheets, GISs, image analysis programs) providing tools that are directly integrated into or downloadable from the site exposing the data via protocols that make it accessible through 3rd party tools. These tools should be easily acquired, easy to use, and reliable. Data site supports, through these tools, data manipulation to answer questions by using data contained within the site or combined with data from other sites generating appropriate visualizations comparing student's own data to that in the site Data Access Working Group Recommendations:

Datasheets: examples of education- facilitating contextual information:

Data Exploration from the Using Data Report Finding and accessing data relevant to the topic they are investigating. Evaluating the quality of this data. Using appropriate tools and interfaces to manipulate and render data to answer questions. Combining multiple and diverse data sets to solve a central problem. Generating visualizations and representations that communicate interpretations and conclusions. Contributing student data to larger data sets. Viewing individual student data in the context of larger data sets.