Introduction to Metadata

Introduction to Metadata  What is metadata?  When is metadata created?  What is included in a metadata record?  What is the value of metadata?

Introduction to Metadata After completing this lesson, the participant will be able to:  Define “data set” and “metadata”  Identify metadata in provided examples  Identify and list the types of information included in a metadata record  Demonstrate understanding as to the value of metadata with different types of data users, developers, and organizations.

Introduction to Metadata The Data Life Cycle Contributes to the Metadata Record Contributes to the Metadata Record

Introduction to Metadata  Metadata is all around… Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction

Introduction to Metadata Definition: A collection of related data  Generally data sets can be defined as: ◦ Spatial – a collection of logically related features arranged in a prescribed manner such as GIS map layers, water features, etc ◦ Tabular – a file, spreadsheet, data in a table ◦ Many tabular datasets are inherently “spatial”, e.g. water-quality samples associated with stream collection sites  Elements in a dataset can include : ◦ Values, measures, points, coordinates, conditions, qualities, frequencies, or attributes that are a result of an observational study

Introduction to Metadata

Introduction to Metadata SpeciesAverage Temperature Temperature Standard Deviation Number of Observations Minimum Temperature Maximum Temperature Northern Red-legged Frog Tailed Frog Arizona Toad Strecker's Chorus Frog Oregon Spotted Frog New Jersey Chorus Frog Wood Frog Spring Peeper Average Temperature of Observation for Each Species

Introduction to Metadata

Introduction to Metadata  When you provide data to someone else, what types of information would you want to include with the data?

Introduction to Metadata  Providing data: ◦ Why were the data created? ◦ What limitations, if any, do the data have? ◦ How should the data be cited if it is re-used in a new study?  Receiving data: ◦ What are the data gaps? ◦ What processes were used for creating the data? ◦ Are there any fees associated with the data? ◦ In what scale were the data created? ◦ What do the values in the tables mean? ◦ What software do I need in order to read the data? ◦ What map projection are the data in? ◦ Can I give these data to someone else?

Introduction to Metadata  Metadata is: Data ‘reporting’ ◦ WHO created the data? ◦ WHAT is the content of the data? ◦ WHEN were the data created? ◦ WHERE is it geographically? ◦ HOW were the data developed? ◦ WHY were the data developed?

Introduction to Metadata A Standard provides a structure to describe data with: ◦ Common terms to allow consistency between records ◦ Common definitions for easier interpretation ◦ Common language for ease of communication ◦ Common structure to quickly locate information In search and retrieval, standards provide: ◦ Documentation structure in a reliable and predictable format for computer interpretation ◦ A uniform summary description of the dataset

Introduction to Metadata

Introduction to Metadata  Dublin Core Element Set ◦ Emphasis on web resources, publications  FGDC Content Standard for Digital Geospatial Metadata (CSDGM) ◦ Emphasis on geospatial data ◦ Biological Data Profile (BDP) of the CSDGM ◦ Profile to the CSDGM emphasis on biological data (and geospatial)  ISO 19115/19139 Geographic information: Metadata ◦ Emphasis on geospatial data and services  Ecological Metadata Language (EML) ◦ Focus on ecological data  Darwin Core ◦ Emphasis on museum specimens  Geography Markup Language (GML) ◦ Emphasis on geographic features (roads, highways, bridges)

Introduction to Metadata Data developers Data users Organizations Metadatahelps…

Introduction to Metadata  Metadata allows data developers to: ◦ Avoid data duplication ◦ Share reliable information ◦ Publicize efforts – promote scientist’s work and contributions to a field of study ◦ Reduces workload

Introduction to Metadata  Metadata gives a user the ability to: ◦ Search, retrieve, and evaluate dataset information from both inside and outside an organization ◦ Find data. Determine what data exists for a geographic location and/or topic ◦ Determine applicability. Decide if a dataset meets a particular need ◦ Discover how to acquire the dataset you identified; process and use the dataset

Introduction to Metadata  Metadata helps ensure an organization’s investment in data: ◦ Documentation of data processing steps, quality control, definitions, data uses, and restrictions ◦ Ability to use data after initial intended purpose  Transcends people and time: ◦ Offers data permanence ◦ Creates institutional memory  Advertises an organization’s research ◦ Creates possible new partnerships and collaborations through data sharing

Introduction to Metadata DATA DETAILS Time of data development Specific details about problems with individual items or specific dates are lost relatively rapidly General details about dataset are lost through time Accident or technology change may make data unusable Retirement or career change makes access to “mental storage” difficult or unlikely Loss of developer results in loss of remaining info TIME (From Michener et al 1997)

Introduction to Metadata TIME DATA DETAILS Sound information management – including metadata development can arrest the loss of dataset detail Information Entropy

Introduction to Metadata  Metadata can support: ◦ data distribution ◦ data management ◦ project management  If it is: ◦ considered a component of the data ◦ created during data development ◦ populated with rich content derive classify collect planimetricimagery analysis alternative committee review PLAN charette meta

Introduction to Metadata  The descriptive content of the metadata file can be used to identify, assess, and access available data resources. online access order process contacts use constraints access constraints data quality availability/pricing keywords geographic location time period attributes

Introduction to Metadata  A metadata collection can be published to the Internet via: ◦ website catalog ◦ web accessible folder (for example, FTP) ◦ Z39.50 metadata clearinghouse ◦ metadata service ◦ geospatial data portal Internet Metadata CollectionUser Query Internet / Intranet dataset

Introduction to Metadata  Federal and National Portals: ◦ Geo.Data.gov (USA) Geo.Data.gov  “largest web-based access at a single-point for maps, government data, and geospatial services” ◦ GeoConnections (Canada)  ◦ ArcGIS Online  Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.

Introduction to Metadata  Data Maintenance: ◦ Are the data current?  Do we have data older than ten years?  Or prior to the hurricane? ◦ Are the data valid?  prior to most current source data  prior to most current methodologies  Data Update: ◦ Contact information ◦ Distribution policies, availability, pricing, URLs ◦ New derivations of the dataset

Introduction to Metadata  Helps you find your data and others discover your data  Find your data by: ◦ themes / attributes ◦ geographic location ◦ time ranges ◦ analytical methods used ◦ sources and contributors ◦ data quality

Introduction to Metadata  Metadata is an exercise in data accountability. It requires you to assess: ◦ What do you know about the dataset? ◦ What don’t you know about the dataset? ◦ What should you or others know about the dataset? ◦ What methods did you use? Are you willing to associate yourself with the metadata record?

Introduction to Metadata  Metadata is a declaration of: Purpose ◦ the originator’s intended application of the data Use Constraints ◦ inappropriate applications of the data Completeness ◦ features or geographies excluded from the data Distribution Liability ◦ explicit liability of the data producer and assumed liability of the consumer What to do… What not to do…

Introduction to Metadata  Metadata records can serve as a project design document: ◦ descriptions & intent of project ◦ geographic and temporal extent of project ◦ source data of project ◦ attributes requirements of project  Benefits : ◦ expectations are clearly outlined ◦ metadata is integrated into the process ◦ provides a medium to record progress

Introduction to Metadata  Use metadata to monitor progress: ◦ data development status ◦ QA/QC assessments ◦ needed changes Requires that the metadata be actively maintained and reviewed time milestones

Introduction to Metadata  Multiple participants in a project can have a common reference for: ◦ descriptions & parameters ◦ keywords, vocabularies, thesauri ◦ contact information ◦ attributes ◦ distribution information  If reviewed regularly by all participants, metadata created early and updated during the project improves opportunity for coordinating: ◦ source data ◦ analytical methods ◦ new information

Introduction to Metadata  As a key component of the data, metadata should be part of any data deliverable  For best quality metadata from a deliverable, provide: ◦ classification and/or validation schema ◦ metadata content template ◦ metadata specification manual

Introduction to Metadata  Metadata is documentation of data  Metadata records capture critical information about a dataset  Metadata allows data to be discovered, accessed, and re-used  A metadata standard provides structure and consistency to data documentation  Metadata is of value to data developers, data users, and to organizations

Introduction to Metadata We want to hear from you! CLICK the arrow to take our short survey.