Presentation is loading. Please wait.

Presentation is loading. Please wait.

Metadata Kim Owens – NOAA’s Ocean Service Mike Moeller – NOAA Coastal Services Center Understanding the Value and Importance of Proper Data Documentation.

Similar presentations


Presentation on theme: "Metadata Kim Owens – NOAA’s Ocean Service Mike Moeller – NOAA Coastal Services Center Understanding the Value and Importance of Proper Data Documentation."— Presentation transcript:

1 Metadata Kim Owens – NOAA’s Ocean Service Mike Moeller – NOAA Coastal Services Center Understanding the Value and Importance of Proper Data Documentation

2 First things first Introductions Logistics Issues/questions

3 Presentation Outline The What and the Why The Value of Metadata The FGDC Content Standard for Digital Geospatial Metadata (CSDGM) Writing Quality Metadata

4 What is Metadata?

5 Metadata is information about your data Therefore, the metadata describes the characteristics (content, location, structure, quality, condition, etc.) of the data set.

6 This is the metadata for this. What’s Missing? Emily and Madison

7 This is the metadata for this. While the card-catalog entry is a form of metadata, it does not address topics such as quality, accuracy, or scale. Well-written geospatial metadata describes these and many more aspects of the data. Rodale's illustrated encyclopedia of herbs ISBN: 087596964x (pbk.) : $17.95 ISBN: 0878576991 : $24.95 Title: Rodale's illustrated encyclopedia of herbs / Claire Kowalchik & William H. Hylton, editors ; writers, Anna Carr... [et al.]. Publication info: Emmaus, Pa. : Rodale Press, c1987. Physical descrip: vi, 545 p. : ill. (some col.) ; 24 cm. General Note: Includes index. Subject term: Herbs. Subject term: Herbs--Utilization. Subject term: Herb gardening. Subject term: Herbs--History. Subject term: Herbals. Added author: Kowalchik, Claire. Added author: Hylton, William H. Added author: Carr, Anna, 1955- Added author: Rodale Press. Rodale's illustrated encyclopedia of herbs ISBN: 087596964x (pbk.) : $17.95 ISBN: 0878576991 : $24.95 Title: Rodale's illustrated encyclopedia of herbs / Claire Kowalchik & William H. Hylton, editors ; writers, Anna Carr... [et al.]. Publication info: Emmaus, Pa. : Rodale Press, c1987. Physical descrip: vi, 545 p. : ill. (some col.) ; 24 cm. General Note: Includes index. Subject term: Herbs. Subject term: Herbs--Utilization. Subject term: Herb gardening. Subject term: Herbs--History. Subject term: Herbals. Added author: Kowalchik, Claire. Added author: Hylton, William H. Added author: Carr, Anna, 1955- Added author: Rodale Press.

8 This is Identification_Information Citation Citation_Information Originator: NOAA, NESDIS Publication_Date: 20030929 Title: Hurricane Isabel Storm Surge Geospatial_Data_Presentation_Form: Remote Sensing Image/Map Publication_Information Publication_Place: Camp Springs, MD Publisher: NOAA, NESDIS, SSD Larger_Work_Citation Citation_Information Identification_Information Citation Citation_Information Originator: NOAA, NESDIS Publication_Date: 20030929 Title: Hurricane Isabel Storm Surge Geospatial_Data_Presentation_Form: Remote Sensing Image/Map Publication_Information Publication_Place: Camp Springs, MD Publisher: NOAA, NESDIS, SSD Larger_Work_Citation Citation_Information the metadata for this.

9 Metadata A Component of Data

10 Proper data documentation provides vital information to interested parties. A Component of Data

11 Metadata is that component of data which describes it. Environmental Sensitivity Index Data Metadata RARNUM - unique combination of species, concentration, and seasonality CONC (concentration) = Density species is found at location Season_ID = seasonality code like to the seasonal table Element - Biology group A Component of Data

12 It’s data about a data set. Title Scale Source Content Location Publication Access Title Scale Source Content Location Publication Access MetadataMetadata GIS files Imagery Geospatial databases GPS data GIS files Imagery Geospatial databases GPS data Data set A Component of Data

13 Because metadata provides vital information about a dataset, it should never be viewed or treated as a separate entity. Metadata Non-spatial or attributes Spatial Take Home Message Metadata is a critical and integral component of any complete data set. Metadata is a critical and integral component of any complete data set.

14 The Value of Metadata Why Bother with Metadata?

15 The Value of Metadata The Current Concept Primary external value Discovery Assessment Access Use

16 The Value of Metadata The Current Concept Primary internal value “Inheritance” “Properly documenting a data set is the key to preserving its usefulness through time.”

17 The Value of Metadata An Emerging Concept An aid to data management Internal value Discovery Assessment Access Use

18 Additional data management benefits The Value of Metadata Data Currency Date of last edit/update Age of source files Data Utility Track source file usage Track distribution frequency

19 The Value of Metadata Monitoring Data Development Data processing steps Status of development Estimate Development Costs Data processing – time and extent Source file availability Additional data management benefits

20 The Value of Metadata To realize the full potential of metadata under this new concept, metadata creation must become integral to the data development process. The question is “How?” Make metadata part of the process

21 Approach metadata development from a business perspective Build administrative support The Value of Metadata Preserves data investment Limits liability Helps manage data resources Aids in external data acquisition Facilitates data access and transfer Provides for efficient data distribution

22 Stress the individual benefits of metadata Build technical support The Value of Metadata Reduces workload over the long term Field fewer data inquiries Provides a means of documenting personal contributions Facilitates sharing of reliable information

23 Develop strong staff support The Value of Metadata Incorporate metadata expectations into job descriptions and performance standards Build technical support Provide staff development opportunities The three “T’s”  Training  Tools  Time

24 Develop templates to facilitate efficient and consistent metadata creation Build organizational support The Value of Metadata Identify pertinent fields within the metadata structure Populate fixed fields  Use standardized language  Define distribution methods  Cite standards used Build source and contact libraries

25 Map metadata fields to the work flow Establish and assign responsibilities Distribute the effort The Value of Metadata  Technicians - lineage  Analysts – process and methodology  Field Scientists – accuracy assessments  I.T. Managers – tools, automated collection methods, information management

26 Mandate the use of standards and templates. Develop boilerplate metadata deliverable language for data contractors. Require publication of metadata. Create and publish a metadata SOP to document policies and procedures. Establish standard policies The Value of Metadata

27 Why Have a Standard? Federal mandates and legislation Standardized Metadata

28 Mandates, Policy, and Legislation The Federal Geographic Data Committee (FGDC) Organized in 1990 under the Office of Management and Budget (OMB) Promotes the coordinated use, sharing, and dissemination of geospatial data on a national basis Background

29 Mandates, Policy, and Legislation “ All Federal agencies must document all Geospatial data that they collect or produce, either directly or indirectly, using the FGDC Content Standard for Digital Geospatial Metadata (CSDGM), and to make that standardized documentation electronically accessible to the FGDC Clearinghouse network.” President Clinton, 1994 Executive Order 12906:

30 “ All Federal agencies that collect, produce, acquire, maintain, distribute, use, or archive analog or digital spatial data in the fulfillment of their mission, financed directly or indirectly, in whole or part, by Federal funds are covered by this requirement.” OMB Circular A-16 (revised) http://www.whitehouse.gov/omb/circulars/a016/a016_rev.html OMB Circular A-16 (revised) Mandates, Policy, and Legislation

31 The Data Quality Act Secion 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 directs OMB to issue government-wide guidelines that: “... provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies.” http://www.noaanews.noaa.gov/stories/iq.htm Mandates, Policy, and Legislation

32 Why Have a Standard? The standard for metadata ensures a level of consistency in data documentation. Standards ensure consistency.

33 Why Have a Standard? Think for a moment how hard it would be to…. … bake a cake without standard units of measurement … put gas into your car without standard nozzle sizes … plug a lamp into a socket without standard electrical outlets

34 The Content Standard utilizes... Common terms Common definitions Common language Common structure Access constraints Citation currentness entity attribute domain lineage Process step Establishing a Standard

35 The Content Standard helps the user determine... If a set of geospatial data is available, fit for a particular use. How to access and transfer the data set. Establishing a Standard

36 Who Who collected the data? Who processed the data? Who wrote the metadata? Who to contact for questions? Who to contact to order? Who owns the data? Where Where were the data collected? Where were the data processed? Where are the data located? What What are the data about? What project were they collected under? What are the constraints on their use? What is the quality? What are appropriate uses? What parameters were measured? What format are the data in? When When were the data collected? When were the data processed? How How were the data collected? How were the data processed? How do I access the data? How do I order the data? How much do the data cost? How was the quality assessed? Why Why were the data collected? Metadata written using the Content Standard answers these important questions: Establishing a Standard

37 Details About the Sections and Terms of FGDC Metadata Standard FGDC Metadata Standard

38 All About the Standard FGDC Content Standard for Digital Geospatial Metadata (CSDGM) Defines the 334 metadata elements and their associated production rules. “The Workbook”

39 The Content Standard is organized using numbered chapters called “sections.” There are 7 main sections 3 supporting sections. Each section is organized into series of elements that define the content required to document your geospatial data set. Organization of the Content Standard Section Data Element

40 Warm up Exercise Tagging the Sections of the Standard

41 CSDGM- 7 Main Sections 1.Identification_Information: (p. 34) General bibliographic information about data set: title, originator, data contact, status, date, abstract, purpose, keywords, geographic location 2. Data_Quality_Information: (p. 44) Lineage and data assessments sources, process methods, accuracy, data processing contact

42

43

44 CSDGM- 7 Main Sections 3. Spatial_Data_Organization_Information Data format: (p. 56) vector, point, raster 4. Spatial_Reference_Information Coordinate system parameters: (p. 60) horizontal / vertical coordinate system, projection, datum

45

46

47 CSDGM- 7 Main Sections 5. Entity_and_Attribute_Information: (p. 75) Database design entities, attributes, domains, description of data values 6. Distribution_Information: (p. 81) How to acquire the data distribution contact, available formats, online distribution website, liability, costs

48

49

50 CSDGM- 7 Main Sections 7.Metadata_Reference_Information: (p. 88) General information about the metadata record itself metadata contact, metadata standard used, metadata creation date, metadata review date

51

52 CSDGM- 3 Supporting Sections 8. Citation_Information: (p. 91) originator, title, publication date, publisher, online linkage, larger work 9. Time_Period_of_Content: (p. 95) single date, multiple dates, range of dates 10. Contact_Information: (p. 96) contact person/organization, address, phone, email

53

54 Exercise 1 Reading A Metadata File

55 Rules of the Metadata Game Learning how to read the structure of the standard

56 Data Quality Information Spatial Data Organization Information Spatial Reference Information Entity and Attribute Information 4526731 Metadata The Three Supporting Sections 9 Time Period Information 10 Contact Information 8 Citation Information Distribution Information Metadata Reference Information Identification Information Organization of the Content Standard The Seven Main Sections

57 Interpreting the Graphical Production Rules The workbook uses graphics to illustrate the production rules of the standard. These graphics include most of the information provided by the production rules, including:  How elements are grouped  What is mandatory and what is not  What elements can repeat and how many times they can repeat

58 Interpreting the Graphical Production Rules Section Sections are depicted by this symbol. Compound Element Compound elements are depicted using a 2-dimensional box. Data Element Data elements are depicted using a 3-dimensional box with shadow.

59 Interpreting the Graphical Production Rules Data Element A data element is a logically primitive item of data. Data elements are the things that you “fill in.” The form for a data element is: Data element name -- definition. Type: (choice of “integer”, “real”, “text”, “date”, or “time”) Domain: (describes valid values that can be assigned) An example of a data element is: Abstract -- a brief narrative summary of the data set. Type: text Domain: free text Note: Data element definitions are contained in the text of the Content Standard, not in the graphical production rules.

60 Interpreting the Graphical Production Rules Turn to page 17 in workbook

61 Mandatory - must be provided. Meaning Data Element Compound Element What’s Mandatory? What’s Not? Mandatory if Applicable - must be provided if the data set exhibits the defined characteristic. Optional - provided at the discretion of the data set producer.

62 If an element can be repeated independently from other elements, it will be indicated as such below the element name. Repeating Elements Compound Element 1 (can be repeated unlimited times) Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 This group of elements would repeat. Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 See page 34, under Keywords

63 Using the Graphics to Make Decisions All elements are colored yellow, so all are mandatory and must be reported. Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2

64 Compound Element 1 is mandatory. Compound Element 1.1 is optional. If yes, Data Elements 1.1.1 and 1.1.2 are mandatory. If no, do not report Compound Element 1.1, Data Element 1.1.1 or 1.1.2, and skip to Data Element 1.2. Data Element 1.2 is mandatory. Compound Element 1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Compound Element 1.1 Using the Graphics to Make Decisions

65 Compound Element 1 is mandatory. Compound Element 1.1 is mandatory. Data Element 1.1.1 is mandatory. Data Element 1.1.2 is mandatory if applicable. Data Element 1.2 is optional. Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Using the Graphics to Make Decisions

66 Compound Element 1 is mandatory if applicable. If not applicable to the data set, do not report any elements. If applicable, it is mandatory and: Compound Element 1.1 is mandatory. Data Element 1.1.1 is mandatory if applicable. If not applicable, do not report it. If applicable, it is mandatory. Data Element 1.1.2 is mandatory. Data Element 1.2 is optional. Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Using the Graphics to Make Decisions

67 Exercise 2 Using The Workbook

68 The FGDC Metadata Clearinghouse Metadata as a Data Discovery Tool

69 The FGDC metadata clearinghouse is a decentralized system of Internet servers you can use to search for available geospatial data. Discovering Data Through Metadata Client FGDC Gateway Servers housing metadata

70 A Brief Look at the FGDC Clearinghouse The FGDC has 6 gateways to its clearinghouse, with access to over 300 spatial data servers. www.fgdc.gov/clearinghouse/clearinghouse.html

71 A Brief Look at the FGDC Clearinghouse Searches can be performed by place names or by using a map interface.

72 The new NSDI Search Wizard bins servers by the types of metadata they house. A Brief Look at the FGDC Clearinghouse

73 Searches can be performed using a map interface that allows the user to define an area of interest. A Brief Look at the FGDC Clearinghouse An area of interest can be defined by dragging an area of interest box on the map interface.

74 The selected area defines the bounding coordinates that will be used in the search. A Brief Look at the FGDC Clearinghouse

75 You can search all the servers listed, or you can select only those that interest you. A Brief Look at the FGDC Clearinghouse

76 Individual servers are selected by holding the Ctrl key down and selecting with the mouse. A Brief Look at the FGDC Clearinghouse

77 Search criteria can be further refined by time period of content and keywords. A Brief Look at the FGDC Clearinghouse

78 The status of each selected node is displayed as search is conducted.

79 A Brief Look at the FGDC Clearinghouse When the search is complete, the status window lets you know if you were successful in discovering metadata that matched your search criteria.

80 A Brief Look at the FGDC Clearinghouse Select a server to see what metadata is available.

81 A Brief Look at the FGDC Clearinghouse Metadata found by the search is displayed by title.

82 A Brief Look at the FGDC Clearinghouse Metadata record returned in HTML format. Links take you to each of the seven main sections of the record.

83 A Brief Look at the FGDC Clearinghouse

84 Similar to the FGDC Clearinghouse Searches for servers that house metadata of a “Coastal” nature Coastal Information Directory (CID) – NOAA/CSC http://www.csc.noaa.gov/CID

85 For more information on the clearinghouse system, visit the FGDC Web site (www.fgdc.gov). Here you can find information on how to establish your own clearinghouse node using free Isite  software. On-line tutorials provide assistance for setting up and configuring this software. A Brief Look at the FGDC Clearinghouse

86 Exercise 3 Search for metadata www.fgdc.gov/clearinghouse/clearinghouse.html www.csc.noaa.gov/CID/

87 Writing Metadata

88 It’s not THAT bad! First records are the hardest. Not all fields may need to be filled in. Tools are available. Can often be produced automatically. Can (and should) be reviewed for updates.

89 Document your data as you go. Writing Metadata

90 Before you begin writing, get organized. Writing Metadata

91 Write so others can understand. Writing Metadata

92 Always review your document. Writing Metadata

93 Write simply but completely. Document for a general audience. Be consistent in style and terminology. Keep your readers in mind. Writing Metadata

94 Define all acronyms. Avoid using jargon. Clearly state data limitations. Writing Metadata Keep your readers in mind.

95 Write a complete Title that includes: What Where When Scale Who Writing Metadata

96 The title is critical in helping others find your data. Greater Yellowstone Rivers from 1:126,700 Forest Visitor Maps (1961-1983) Writing Metadata

97 Be specific. Quantify when you can. Vague: We checked our work and it looks complete. Specific: We checked our work using 3 separate sets of check plots reviewed by 2 different people. We determined our work to be 95% complete based on these visual inspections. Writing Metadata

98 Select your key words wisely. Use unambiguous words. Use descriptive words. Fully qualify geographic locations. Writing Metadata

99 Have someone else read it. If you’re the only reviewer, put it away and read it again later. Check for clarity and omissions. Review your final product. Writing Metadata

100 Can a novice understand what you wrote? Are your data properly documented for posterity? When you review your work, ask: Writing Metadata

101 Does the documentation present all the information needed to use or reuse the data? Are any pieces missing? When you review your work, ask: Writing Metadata

102 Your audience may be very diverse. Consider writing your metadata to reflect this diversity.

103 Metadata Creation and Validation Tools of the Trade

104 Metadata Tools Some available tools for metadata creation, validation, and publication. CNS and MP “Chew ‘n spit”- checks and corrects structural errors, and “Metadata Parser”- checks for errors in element compliance; “mp batch” and “mp online tool” Template tools CSC’s metaScribe create large numbers of similar records. ArcView Tools Extension for ArcView 3.x ArcCatalog for ArcGIS 8.x TKME Text editor used for metadata entry.

105 TKME, CNS and mp are FREE downloads! http://geology.usgs. gov/tools/metadata See document, “Downloading and Installing CNS and MP”

106 Tools of the Trade TKME - An editor for formal metadata, TKME is intended to simplify the process of creating metadata that conform to the FGDC Standard. - Hierarchical structure - Proper arrangement Maintains: of elements

107 Tools of the Trade – Creating metadata “Add” element names from dropdown menu When right side turns white, you know to input info (import, cut & paste, etc.)

108 Tools of the Trade – Creating metadata Help Menu: - Version (how to use, helpful hints - Element (definition of element) - Output (final output look) - Fonts (choose different fonts Help Menu

109 Tools of the Trade – Creating metadata Double click on Tkme icon from desktop Go to File, Open, and navigate to C:\Metadata\ benthic_bad.txt

110 Tools of the Trade- Creating metadata NOAA CSC ArcView ® Metadata Collector

111 ArcGIS metadata collector Found in ArcCatalog, this tool allows the user to write metadata within the Arc environment. Tools of the Trade

112 MetaScribe This new tool was also developed by NOAA CSC to aid in the creation of multiple sets of metadata that exhibit a high degree of redundancy.

113 CNS (“Chew ‘n Spit”) A pre-parser for formal metadata designed to assist metadata managers convert records that cannot be parsed by mp into records that can be parsed by mp. Tools of the Trade - Validation Tools MP (Metadata Parser) A compiler to parse formal metadata, checking the syntax against the FGDC Content Standard for Digital Geospatial Metadata and generating output suitable for viewing with a web browser or text editor.

114 TKME, CNS, and MP are available as free downloads from the United States Geological Survey (USGS) Website. (geology.usgs.gov/tools/metadata) TKME will run from a shortcut on the desktop Both MP and CNS can be run from: command line in MS-DOS or UNIX MP Batch tool and MP online interface Tools of the Trade

115 MP Batch, Integraph® Check multiple records at one time for CNS and MP compliance http://imgs.intergraph.com/smms/default.asp SMMS Metadata Software Variety of metadata tools (but it cost) http://www.intergraph.com/gis/support/ > Free Utilities/Tools MP Online Tool, Peter Schweitzer (USGS) User friendly interface for MP (no command line)

116 http://www.nature.nps.gov/im/units/mwr/gis/metadata/metadata_tools.htm NPS Metadata Tools/Extensions - ArcGIS ESRI Metadata Customizations http://imgssupport.intergraph.com/Tools.asp - ArcCatalog extension; 5 new buttons (MP, editor, organizational, input/export capabilities) - Spell checker: understands element names, underscores - Advanced synchronization: turn on/off different sections so it’s not “automatically updated” (Entity & Attributes) Tools of the Trade

117 Optional Exercise MP exersise MP Batch MP online tool

118 Tools of the Trade http://www.fgdc.gov/standards/status/textstatus.html NBII Biological Data Profile Remote Sensing Extension FGDC Profiles and Extensions to the CSDGM Shoreline Data Profile http://www.csc.noaa.gov/metadata/shoreline_profile.html http://www.fgdc.gov/standards/status/sub5_2.html http://www.fgdc.gov/standards/status/csdgm_rs_ex.html http://www.fgdc.gov/standards/status/sub5_6.html http://www.nbii.gov/datainfo/metadata/standards/index.html

119 Keyword Lists / Controlled Vocabularies / Thesauri Global Change Master Directory's (GCMD) http://gcmd.gsfc.nasa.gov/Resources/valids/index.html Integrated Taxonomic Information System http://www.itis.usda.gov/ Master Environmental Library http://mel.dmso.mil/docs/metadata_guide/section_6.htm Aquatic Science and Fisheries Abstracts http://www4.fao.org/asfa/asfa.htm Geographic Names Information System (GNIS) http://geonames.usgs.gov/

120 Resources: CSC’s Metadata Website: http://www.csc.noaa.gov/metadata Metadata Standards How to start writing metadata with Metadata Bob Metadata tools Metadata Forum Metadata Training Materials... and much, much more! Featuring:

121 Resources: NOS Internal Website NOS Metadata Program Metadata in our Everyday Lives Metadata: What and Why? The FGDC Metadata Standard The FGDC Clearinghouse Metadata Tools... and much, much more! Featuring: https://inside.nos.noaa.gov/foremployees/it/metadata/welcome.html

122 Finally... Remember, metadata is a legacy document that concisely sums up your data or data set. Without metadata, your data set is incomplete.

123 Optional Exercise: Get Started! Create a record using Tkme or ArcCatalog

124 Kim Owens Kimberly.Owens@noaa.gov Mike Moeller Mike.Moeller@noaa.gov


Download ppt "Metadata Kim Owens – NOAA’s Ocean Service Mike Moeller – NOAA Coastal Services Center Understanding the Value and Importance of Proper Data Documentation."

Similar presentations


Ads by Google