Spy Pond Partners Frances D. Harrison Chair, TRB Data and Information Technology Section Subcommittee on Metadata January 22, 2006 Current Perspectives on Transportation Metadata Needs 2006 TRB Annual Meeting

2 Agenda Metadata – what and why Metadata in transportation Library Geospatial ITS – Archived Traffic Data ITS – Road Weather Data TransXML – road and bridge design, construction, safety TRB Data Section Subcommittee on Metadata Mission and Role Activities and Next Steps

3 Don’t you hate it when… You ask five people in your agency the number of highway system miles and get five different answers You spend an hour a day answering the same questions about your agency’s data Your “data guru” quits and leaves gigabytes of partially cleaned data with cryptic documentation You spend thousands of dollars building a data set only to discover that it already existed in another agency You try to combine data gathered from five different corridor studies sponsored by different agencies but can’t fit the pieces together You can’t use half of the data you find because you don’t know what it means or how good it is

4 Wouldn’t it be great if… Agencies had policies in place to require good, complete documentation of all data sets – and devoted the resources to make this happen There were standard templates for documenting every major kind of transportation data There were tools to produce and validate this documentation There was an easy way for agencies to share their data and move towards standardized data definitions and coding schemes There was agreement about how to measure and describe data quality for different types of data There were better and quicker ways of finding data on the web than Google

5 The Solution: Metadata! Training Workflow Repositories Tools Policies Standards

6 Definition of Metadata “Information about a thing, apart from the thing itself” -Ned Batchelder “Structured data about an object that supports functions associated with the designated object” -J. Greenburg ThingMetadata BookTitle Author Publication Date ISBN Number PhotographSubject Photographer Date Web PageURI Title Language GIS CoverageLocation Accuracy Format Data ElementElement Name Data Type Length Possible Values

7 Card Catalogue

8 Document Properties

9 Geospatial Metadata

10 Data Dictionary – Attribute Information Source: Traffic Management Data Dictionary

11 Metadata for Sharing Photographs – Flickr Folksonomies

12 Metadata Maximizes and Preserves Data Value Discovering Understanding Sharing

13 But Metadata is Hard 1. Too abstract 2. Too complicated 3. Not sexy 4. Not easy 5. Tedious 6. Time consuming 7. Expensive 8. Hard to get agreement 9. Not on the critical path 10. Not enforced 11. Low priority 12. Stepchild of the stepchild

14 Do we need a 12 step program for metadata? 1. Recognize there is a problem 2. Believe there is a way to solve it 3. Commit to doing something about it 4. Examine your current practices 5. Publicly admit that things need to change 6. Prepare to change the status quo 7. Get help from colleagues and other experts 8. Identify specific practices to be improved 9. Correct them 10. Institutionalize the process in your agency 11. Periodically reaffirm why you are doing this 12. Help others by sharing your successes and lessons learned

15 Metadata in Transportation - examples Library Geo-spatial ITS – Archived Traffic Data ITS – Road Weather Data TransXML - data exchange formats for road and bridge design, construction, safety

16 Existing Standards (to name a few…) Dublin Core – cross-domain resource description MODS, MARC, METS –metadata standards for digital libraries FGDC CSDGM – Metadata for geospatial datasets ISO 19115/19139 Geographic Information - Metadata ISO – Metadata Registries IEEE 1489 – ITS Data Dictionaries ISO – Meta Attributes for ITS/TICS Data Dictionaries IEEE 1487 – Metadata Registries for ITS (based on ISO and IEEE 1489) DDI – Data Documentation Initiative (social science survey data documentation)

17 National Transportation Library – Modified Dublin Core

18 Transportation Research Thesaurus

19 FGDC Metadata Executive Order (1994) Mandated creation of metadata standards in conjunction with the National Spatial Data Infrastructure (NSDI) Federal Geographic Data Committee (FGDC) – interagency group charged with creation and maintenance of the standards Widely adopted and used Wealth of resources: tools, tutorials, training Enabler of metadata clearinghouses – access to wide variety of geospatial datasets Harmonization with ISO 19115

20 FGDC Content Standard for Digital Geospatial Metadata

21 Archived ITS Data ASTM E Standard Guide for Archiving and Retrieving ITS-Generated Data Published June 2003 General description of metadata types: archive structure, processing documentation and data collection system ASTM Task Group E : “Standard Specification for Metadata Content for ITS-Generated Data Based on FGDC – given wide acceptance and tool availability Tackling specific issues related to traffic data – e.g. quality, aggregation methods

22 Clarus National repository of road weather data – network of networks Separation of Producers and Consumers - Metadata Essential 80 elements defined Based on quality checking and user requirements Data exchange formats Need to address data sharing between “Weather World” and “ITS World”

23 TransXML Transportation Data Exchange Schema – NCHRP Four areas Roadway Design (integration of GIS data, construction cost estimates) Construction (bidding, payments, status tracking) Safety (crash data and highway safety analysis) Bridge Design (verification of structural analysis) Definition of data structures – entities, attributes and relationships (UML) XML Schema posted for comment: Sample applications end of January Final Report – June 2006 Presentation at AASHTO IS (June 4-7 – Ft Worth, TX)

24 Example: Data Exchanges for Construction Progress Progress Payment System Installed Quantities Pay Item Quantities Materials Material Test Results Placement Location Material Samples Tests Performed Test Outcomes Project Engineer Progress Reporting Laboratory Systems Field Data Collection Systems

25

26 TransXML – Issues Leverages existing schema and standards – different styles, overlapping elements, moving targets GML LandXML Global Justice NHTSA Crash Records (MMUCC and FARS) AASHTOWare data schema aecXML Balance between consistent framework and niche schemas Stakeholder participation lower than anticipated Never enough resources for communication, education and liaison Chicken and egg considerations “Vendors will only act if agencies demand it” “Agencies will come on board if the vendors support it”

27 Conclusions Needs and technology converging to enable advances in practice Good models exist – foundation for continued work More standards and models needed Dublin Core or FGDC extensions and refinements for different data objects Data dictionaries and schemas for common data exchanges Taxonomies, controlled vocabularies Code lists and format rules for shared attributes Standards are not enough - tools, training, policies, practices Great need for coordination and information sharing across diverse efforts Crosswalks – to promote understanding and enable harmonization

28 TRB Metadata Metamorphosis Metadata Subcommittee operated under Urban Data – led by Marcus Wigan Research Education Coordination Need for broader approach Metadata is a cross-cutting issue – not appropriate as a subcommittee under Urban Data Metadata is key to data sharing – but requires leadership and coordination Metadata is abstract, not widely understood - hard to get people’s attention How to get to the next level? Working Group – February 2005 Joanna Zmud chair Mission: Recommend plan of action for the Data and Technology Section by summer meeting in August Outcome: Section subcommittee immediately – morph into Task Force

29 TRB Metadata: The Next Generation Section Subcommittee – August 2005 Members from most Data Section Committees + Library Science Mission and Roles Promote common understanding and appreciation of metadata value Share information about metadata related activities across transportation data communities Identify priority needs for new metadata standards Promote research that provides technical groundwork for standards Promote coordination and consistency across standards efforts Promote best practices in use of metadata Promote metadata training and tools development

30 TRB Metadata Subcommittee – Activities To Date Monthly Conference Calls Compiling Resource Lists: Standards, Tools, Implementation Examples Draft Research Agenda – Eight Initiatives White Paper State of the Practice Survey – help to set priorities Best Practice Synthesis End User Needs Investigation MPO Use of Metadata for Modeling and GIS-T Gap Closure Feasibility Study for Metadata Registry or Clearinghouse Crosswalk and Code List Catalogue Comments and Input Welcome –

31 Future Task Force Product: Strategic Plan for Metadata Establish need for and benefit of metadata standards Recommend and prioritize types of standards for development Identify potential barriers to acceptance Identify coordination needs across standards efforts Identify stakeholders and participants Recommend methods to promote use

32 Acknowledgements Kuo-Ann Chiao (NYMTC) Scott Dahl (US Census Bureau) Ralph Gillmann, FHWA James Hall (University of Illinois, Springfield) Joyce Koeneman (RITA-NTL) Guy Rousseau (ARC) Jack Stickel (Alaska DOT) Xiaduan Sun (University of Louisiana) Shawn Turner (Texas A&M) Anita Vandervalk (CSI) Marcus Wigan, Oxford Systematics Eric Ziering (CSI) Joanna Zmud, NuStats