Presentation on theme: "FEMA’s Flood Map Modernization 1 MIP R3 Requirements – Content Manager Metadata and Data Submissions August 2006 MIP Studies Workflow R3 Deployment Training."— Presentation transcript:
FEMA’s Flood Map Modernization 1 MIP R3 Requirements – Content Manager Metadata and Data Submissions August 2006 MIP Studies Workflow R3 Deployment Training MIP R3 Requirements
FEMA’s Flood Map Modernization 2 Existing Federal Metadata Policy Executive Order 12906 Coordinating Geographic Data Acquisition and Access: The National Spatial Data Infrastructure Signed 4/11/94, by President Clinton Section 3, Development of a National Geospatial Data Clearinghouse “Standardized Documentation of Data,... each agency shall document all new geospatial data it collects or produces, either directly or indirectly, using the standard under development by the FGDC, and make that standardized documentation electronically accessible to the Clearinghouse network. http://www.fgdc.gov/metadata/contstan.html
FEMA’s Flood Map Modernization 3 Existing Federal Metadata Policy Circular No. A-16 Coordination of Geographic Information and Related Spatial Data Activities 8/19/02, Office of Management and Budget Section 2.b.2, “All spatial data collected or derived directly or indirectly using federal funds will have FGDC metadata.” Section 6.b, “As examples, this Circular applies to, but is not limited to… FEMA's Flood Plain Mapping program and other federal activities that involve national surveying, mapping, remote sensing, spatially referenced statistical data, and Global Positioning System (GPS).” http://www.whitehouse.gov/omb/circulars/a016/a016_rev.html#1
FEMA’s Flood Map Modernization 4 Current MIP Metadata Policy DFIRM Database Deliverable Submission Appendix L, section L.2.8 “To facilitate the use of these data and the transfer of data files between users, the assigned Mapping Partner shall prepare and submit a metadata file with all digital data submittals.” Workflow tasks Develop DFIRM Database (draft DFIRM DB) Produce Preliminary Map Products (prelim DFIRM DB) Produce Map Products (final DFIRM DB)
FEMA’s Flood Map Modernization 5 Current MIP Metadata Policy DCS Task Submissions Appendix N Section N.2.3.1, Deliverables - Metadata FEMA Procedure Memorandum 35, Additional Requirements for Inclusion in FY05 Study contracts Section 3, “A metadata file that complies with the FEMA Metadata Profiles in XML-format shall be included with each submittal.” FEMA Metadata Profiles were distributed 1/17/06.
FEMA’s Flood Map Modernization 6 Current MIP Metadata Policy Non-DCS/Non-DFIRM Task Submissions Metadata must be submitted for data archiving and retrieval purposes. Metadata ensures data integrity for all spatial data used for the DFIRMs.
FEMA’s Flood Map Modernization 7 Content Manager Archive for all data (spatial and non-spatial) associated with DFIRM, LOMR, and LOMA production and submissions Only authenticated MIP users can access the Search & Retrieve portlet Tools & Links > Search & Retrieve Data Metadata profiles ensure data discovery and proper ingestion into Content Manager Data is archived when RMCs pass the Store Content task; data is not stored in Content Manager upon upload except for Pre-scoping and Scoping tasks. Pre-Scoping and Scoping submissions do not require validation, and there aren’t metadata profiles for those tasks.
FEMA’s Flood Map Modernization 8 Archiving Data Users upload data submissions with metadata. Data is archived for all spatial submissions based on the metadata profiles. Data is archived for all non-spatial submissions (e.g., Scoping meeting minutes) based on key metadata elements; not a complete FGDC-compliant metadata file. Placekeys Title Abstract Access Restriction Effective Date
FEMA’s Flood Map Modernization 9 Archiving Data Metadata Profiles ensure that the metadata elements necessary to discover the data submissions are populated correctly. Placekeys have distinct headings to define the type of location STATE VA COUNTY Fairfax COMMUNITY Vienna FEMA-CID 510053 Publication Date and Calendar Date cannot be “Unknown” or “yyyy”. Values are stored in a numeric field; query for date ranges
FEMA’s Flood Map Modernization 10 Querying Archived Data The search for archived data is based upon metadata elements (e.g., placekeys, themekeys, originator). General Search All tasks, All workflow types (i.e., Studies, Revisions, and Amendments) DFIRM/DCS Data Search Spatial-related tasks, Studies workflow only MapViewer returns bounding coordinates for spatial searches
FEMA’s Flood Map Modernization 11 Search & Retrieve General search for spatial and non-spatial data submissions for ‘Lonoke’ County Discovered data submissions for ‘Lonoke’ County
FEMA’s Flood Map Modernization 12 Discovering Archived Data Excerpt from metadata file… Headings drive the location searches; Searched for ‘Lonoke’ in the County search Access the individual data submission via the ‘Link’ Excerpt of data files in the uploaded data submission
FEMA’s Flood Map Modernization 13 Querying for Archived Data Coordinates auto-populate in the query based on the map extents in MapViewer Query returns specific data type submissions in which the bounding coordinates in the metadata file are located within the queried extent
FEMA’s Flood Map Modernization 14 Retrieving Archived Data accconst: None If the metadata file declares no access constraints, then all data files in the submission can be downloaded. accconst: Restricted If the metadata file declares any access constraints, then none of data files in the submission can be downloaded
FEMA’s Flood Map Modernization 15 Required Files for Studies Workflow The Studies workflow will not advance if required files are not uploaded to the correct folder with the correct naming convention at specific tasks. Metadata Files All data development tasks Produce Preliminary Map Products task Produce Map Product (post-preliminary) task DCS QA Report All DCS data development tasks
FEMA’s Flood Map Modernization 16 Required Files for Studies Workflow The Studies workflow will be stopped if the required notification from HDM has not been received by the MIP. Data is sent to HDM for Independent QC via the Studies workflow for the draft, preliminary, and final deliverable submissions. Develop DFIRM Database task Produce Preliminary Map Products task Produce Map Products task Workflow tasks will not advance if the passing Notification from HDM is not present. Validate Content Submission Perform Map Products QA
FEMA’s Flood Map Modernization 17 Required Files for Studies Workflow Workflow TaskMetadata FileDCS QA ReportHDM Notification Acquire Base Map Develop DFIRM Database (Validate Content Submission task) Develop Hydraulic Data Develop Hydrologic Data Develop Topographic Data Perform Alluvial Fan Analysis Perform Coastal Analysis Perform Field Survey Perform Floodplain Mapping Perform Redelineation Produce Preliminary Map Products (Validate Content Submission task) Produce Map Products (Perform Map Product QA task)
FEMA’s Flood Map Modernization 18 Creating Metadata Digital Metadata Profile Using a digital version of the NFIP Metadata Profiles Guidelines, copy/paste/edit appropriate the metadata profile into a.txt document. Change all underlined text. The underlined text indicates a dynamic required metadata element. DFIRM Tools Use DFIRM Metadata Builder tools in DFIRM Tools These tools create/edit a metadata.txt file according to the DFIRM metadata profile.
FEMA’s Flood Map Modernization 19 Naming the Metadata Files Naming conventions are listed for each metadata profile in the NFIP Metadata Profiles Guidelines. For example, _BaseMap_metadata.xml The names are case-sensitive. The DFIRM Metadata Builder will name the file correctly. Metadata files must be submitted in either.xml or.txt format for all uploads. The following tasks require that the previously uploaded metadata files be in.xml format: Validate Content Submission Store Content Perform Map Products QA Access MetaMan in the Citrix environment to convert the.txt metadata files to.xml format.
FEMA’s Flood Map Modernization 20 Metadata Locations on the MIP DCS metadata Store in the General subfolder within the root\TaskSysID folder for the task (e.g., Develop Topographic Data – Terrain\1200489\General folder) Appendix N specification DFIRM-related metadata Store in the Mapping\ folder (e.g., Develop DFIRM Database - Mapping\Draft_DFIRM_DB folder) Non-DCS/Non-DFIRM metadata Store in the root\TaskSysID folder for the task (e.g., Acquire Base Map – BaseMap\1321086 folder)
FEMA’s Flood Map Modernization 21 Creating the DCS QA Report Use the DCS Validator tool in WISE to generate the QA report, despite whether or not you used WISE to create your DCS data. DCS QA report can be either passing or failing; you will be able to advance the Studies workflow as long as the report has been uploaded and named correctly. Your RMC will verify whether or not the data should be redone and/or does not comply to Appendix N specifications at the Validate Content Submission task.
FEMA’s Flood Map Modernization 22 Naming & Storing the DCS QA Report Files are named according to which folder the task is associated. Names are case-sensitive. Files must be in.xml format. The DCS QA report must be stored in the root\TaskSysID folder for the task; do not upload it in a subfolder. (i.e., Hydrology, Hydraulics, Terrain, and Survey). Workflow TaskDCS QA Report Naming Convention Develop Hydraulic Data _ _Hydraulics.xml Develop Hydrologic Data _ _Hydrology.xml Develop Topographic Data _ _Terrain.xml Perform Field Survey _ _Survey.xml
FEMA’s Flood Map Modernization 23 Generating the HDM Notification The HDM Notification is not a file that you create or upload. Load Data Artifacts portlet You can select to ‘Send Data Contents to HDM’ once you have uploaded all your necessary data files for submission. You cannot complete a DFIRM-related task until you opt to send your data submission to HDM.
FEMA’s Flood Map Modernization 24 Generating the HDM Notification DFIRM Tools When you export your data submission via the Database Exporter tools, you are given the option to send your data submission to HDM. You must have created your metadata prior to exporting your data if you choose to send the submission to HDM.
FEMA’s Flood Map Modernization 25 HDM Notification Results You will be able to complete the DFIRM-related task once the data submission has been sent for Independent QC. QC Results The pass/fail HDM Notification and a weblink to the detailed QC report is automatically submitted back to the MIP by HDM. The weblink to the detailed QA report will be posted on the DFIRM-related task screen, Validate Content Submission screen, and/or Perform Map Products QA screen within the Studies workflow, depending upon the submission type (i.e., Draft, Preliminary, or Final).
FEMA’s Flood Map Modernization 26 HDM Notification Results Advancing the Studies Workflow The RMC will not be able to advance the workflow at the associated Validate Content Submission or Perform Map Products QA task without a passing HDM Notification. Pass data auto-validation - Draft and Preliminary submissions Pass data auto-validation and visual QC – Final submission If a failing HDM Notification is returned, the RMC will only be able to reject the data contents and return the project to the DFIRM-related task lead’s workbench (e.g., DFIRM Database Task Lead)
FEMA’s Flood Map Modernization 27 HDM Notification Results The following graphic depicts the flow of events for a passing and failing HDM notification for the preliminary DFIRM deliverable submission. This assumes that the HDM notification has been received. The flow of events is the same for the Develop DFIRM Database task and Produce Map Products task.
FEMA’s Flood Map Modernization 28 Generating Trial HDM QC Report Tools & Links > HDM Upload portlet Used for trial submissions of your DFIRM database Only the data auto-validation QC review will be conducted on the trial submission; no trial review for visual QC QC report will be emailed to the submitter, not to the MIP There is no functionality for the HDM Upload portlet to ‘connect’ your results to the MIP workflow.
FEMA’s Flood Map Modernization 29 Submitting Data There are 4 ways to submit data to the MIP Copy data from the Production drive (J:) Upload data via the Load Data Artifacts portlet in the Studies workflow tasks Mail your data submission to FEMA Data Depot Exporting data from DFIRM Tools (This is limited to DFIRM-related data submissions.)
FEMA’s Flood Map Modernization 30 Copying Data Submission Data can only be copied from the Production drive (J:) to the Submittal drive (K:) through the Load Data Artifacts portlet. You can copy data only from your FEMA Case Number on the J drive to the root folder (e.g., Draft_DFIRM_DB) and/or subfolders within the root folder (e.g., Terrain\Output\VEC) on the K drive for the current task. If you select a subfolder to copy, the subfolder and all of its contents will be copied to the destination folder.
FEMA’s Flood Map Modernization 31 Uploading Data Submission The Load Data Artifacts portlet is specific to the task in which it appears; the MIP knows to which folder to upload the submission Zip multiple files and subfolders together. The MIP will unzip your file and maintain the subfolder structure. You will be prompted to overwrite existing files with the same name. 300 MB file size maximum; 30 minute connection before timing out
FEMA’s Flood Map Modernization 32 Mailing Data Submission Mail data submission to FEMA Data Depot Send CDs, DVDs, Hard Drives, or Flash Drives; there is no file size restrictions. Include readme.txt file indicating Region, State, County, Community, FEMA Case Number, workflow task(s), and contact information Inform MIPHelp that the submission has been mailed Workflow will not advance until the data/metadata submission has been received and uploaded Must account for postal delivery time and upload time in your task schedule FEMA Data Depot 3601 Eisenhower Avenue Suite 130 Alexandria, VA 22304
FEMA’s Flood Map Modernization 33 Deleting Data from Submission Data can only be deleted from the Submittal drive (K:) for DFIRM- related tasks. Develop DFIRM Database Produce Preliminary Map Products Produce Map Products Once a data file has been uploaded or copied to the K drive for a non DFIRM-related task, you must contact MIPHelp to have the file deleted.
FEMA’s Flood Map Modernization 34 Citrix Windows Explorer The Submittal drive (K:) no longer has write permissions; all previous write permissions have been revoked. Use ‘File Copy’ in the Load Data Artifacts portlet to copy data from the Production drive (J:) to the K drive. Use the Delete functionality in the Load Data Artifacts portlet to delete files for a DFIRM-related task. Contact MIPHelp to delete files for a non DFIRM-related task. Files on your local drives should be uploaded to the K drive via the Load Data Artifacts portlet. Read permissions remain on the K drive; you can copy and paste files to your local drives via Citrix Windows Explorer. The Production drive (J:) still has read and write permissions.
FEMA’s Flood Map Modernization 35 Uploading for Multiple Duplicate Tasks If you have multiples of the same task in one Studies project (e.g., two Perform Field Survey tasks assigned to two different Mapping Partners), you can upload data for all similar tasks without overwriting each others’ data. Submissions for duplicate tasks are stored separately. Each task is assigned a unique TaskSysID. TaskSysID is displayed on the data development task, Validate Content Submission, Independent QC/QA, and Store Content task screens. TaskSysID will be a subfolder within the task’s root folder (e.g., …\ \Terrain\ \Source) DFIRM-related tasks will not be assigned a TaskSysID; only one task should exist in the workflow.
FEMA’s Flood Map Modernization 36 Accessing Production Tools Single Sign-On Citrix Sign into your Citrix account while logged into the MIP The MIP remembers your Citrix login information, and automatically logs you into the Citrix environment every time you log into the MIP.
FEMA’s Flood Map Modernization 37 Uploading Data via DFIRM Tools DFIRM Metadata Builder Creates your metadata file based on the DFIRM metadata profile and saves it to the appropriate folder (e.g., Mapping\Draft_DFIRM_DB) on the Submittal drive (K:) Database Exporter Exports your DFIRM database deliverable submission to the appropriate folder (e.g., Mapping\Final_DFIRM_DB) on the Submittal drive (K:) Map Production Pro > Export to Images Exports your map panels to TIF and PNG formats and stores them in the \Mapping\Final_DFIRM_DB\RFIRM folder on the Submittal drive (K:) DFIT Pro > Export to Images Exports your INDEX map to TIF format and stores it in the \Mapping\Final_DFIRM_DB\RFIRM folder on the Submittal drive (K:)
FEMA’s Flood Map Modernization 38 MIPHelp@mapmodteam.com In all communications to MIPHelp, please include the following information: Region State County Community FEMA Case Number Current task in the MIP workflow Error message received and screen captures Details specific to your problem (e.g., cannot upload data, uploaded metadata but cannot advance through task).