ORGANIZING AND STRUCTURING DATA FOR DIGITAL PROJECTS Suzanne Huffman Digital Resources Librarian Simpson Library

DATA AND DIGITAL PROJECTS

Nothing really important is ever headlined “Here is some data. Hope you find something interesting.” Annotation is critical. Editing is critical. -Amanda Cox, New York Times Graphics Editor

The importance of context

Add value to your data Regardless of website functionality, annotation and guidance are important Look at other digital projects and test out similar sites to see what types of discovery and analysis activities you intuitively want to do with similar types of data Follow web design methodology by creating user stories and acting them out with mockups, wireframes, or simple spreadsheets

ORGANIZING AND STRUCTURING DATA

Metadata Metadata is structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource. Metadata is often called data about data or information about information.

The metadata universe

Choosing metadata parameters Ask yourself why you are collecting the information you want to collect (perform a needs assessment) Focus on the outcomes and analysis tasks you want your site’s users to be able to perform with your data Depending on your audience, don’t assume users know what each metadata field or category means Choose data fields and parameters and do a few test runs before analyzing the data to determine if you need to add, change, or edit any of your fields

What is good metadata? According to Understanding Metadata by the National Information Standards Organization, good metadata… Should be appropriate to the materials in the collection, users of the collection, and intended, current, and likely use of the digital object Supports interoperability Uses standard controlled vocabularies to reflect the what, where, when, and who of the content Includes a clear statement on the conditions and terms of use for the digital object Should have the qualities of archivability, persistence, and unique identification, and should be authoritative and verifiable Supports the long-term management of objects in collections

Choose and use standard terminology Use a controlled vocabulary that provides preferred keywords and terminology for specific items Create a data dictionary and be consistent in applying it Example data dictionary entry: Dates are displayed in the yyyy-mm-dd format; i.e., March 15, 2015 would appear as Helps prevent inconsistencies in data entry and analysis Example: when "T", "temp", and "t" are all used interchangeably within a single dataset to refer to temperature measurements

Metadata dos Select your keywords wisely and think about the many ways someone might search for your data Use your data dictionary whenever possible to create keywords to establish a controlled vocabulary Use descriptive and clear writing Ensure that all data fields are independent and that they could exist on their own

Metadata don’ts Do not use jargon; define technical terms and acronyms and put them in your data dictionary Remember that a computer will read the information in the metadata record, so do not to use tabs, indents, or special characters like # % { } | / \ ~ that may be misunderstood Do not copy and paste content from word documents or other sources into your metadata record (use a text editor as a middle step to prevent unnecessary characters and errors being introduced)

Metadata is structured information Example Dublin Core record in XML

Data structure Structuring your data is important to ensure your site functions well and that the dataset can be used in a variety of ways Ways to structure data: For Excel or Google spreadsheets, save your data as CSV files in plain text format XML documents can be easily created through online data-entry forms and contain your metadata within a structured framework

Data best practices Make sure your data is portable Saved in an additional location outside your site in machine-readable, non-proprietary format Portable data is flexible, sharable, and can be harvested by a variety of tools for usage in future projects

Quality assurance and control Restrict what information can be entered into the dataset Limit the use of free text fields for metadata Use lookup tables or drop-down menus for data entry Use validation tools Do manual review Clean up and normalize messy data with tools like Open Refine

DATA MANAGEMENT

What is data management? Data management refers to all aspects of creating, housing, delivering, maintaining, archiving, and preserving data. A data management plan accounts for every activity within the data life cycle.

Contents of data management plan (DMP) Data Type and Format Data Storage Data Standards Data Security Data Sharing Long-term Access Check out VCU Libraries’ Research Data Management Guide at

Data citation and preservation Citation Dataset Citations should have (at a minimum): Creator (PublicationYear): Title. Publisher. Identifier. The Identifier could be a DOI or just the website’s URL Preservation Good documentation on data provenance (the origin and history of a dataset) is crucial. If data cannot be recreated or if it is costly to reproduce, it should be saved. Datasets that have significant long-term value may be contributed to a repository for preservation.

Data repositories These repositories can be used to find data for reuse or to deposit your research data for preservation and sharing:

Questions? Comments? Thank you! Please contact me if you need assistance with managing and organizing data in your research or teaching projects.

References and Resources data-at-the-new-york-times data-at-the-new-york-times df df pdf pdf