VO Sandpit, November 2009 Airborne Research and Survey Facility Data CEDA summer work placement Nicola Siddall.

Slides:

Advertisements

Similar presentations

1 Metadata Tools for JISC Digitisation Projects of still images and text Ed Fay BOPCRIS, Hartley Library University of Southampton.

Advertisements

CC SQL Utilities.

The BADC-CSV Format Meeting user and metadata requirements Graham A Parton*, Sam J Pepler British Atmospheric Data Centre, Rutherford Appleton Laboratory,

The Caught and Coloured website: its EMu origins Alex Chubaty – Collection Information Systems Craig Churchill – IT Software Development Museum Victoria.

Obtain Data from Other Sources Data you need to include in an Access database often already exists in another file: in another Access database, in a table.

Introduction to metadata for IDAH fellows Jenn Riley Metadata Librarian Digital Library Program.

(A)ATSR archive at the NERC Earth Observation Data Centre Matt Pritchard, Jack Abolins, Brian Maddison & Chris Mutlow NEODC / CCLRC Rutherford Appleton.

NPS Introduction to GIS: Lecture 1

Use different sheets in one workbook to store raw data from different sources / about different aspects of the investigation. This way, you can see all.

1 SYSTEMS DESIGN Pertemuan 13 s.d 20 Matakuliah: A0554/Analisa dan Perancangan Sistem Informasi Akuntansi Tahun: 2006.

EMu and Fotoware: Integrating the EMu Collections Management Program with Image Management Software - Dr. Lance Wilkie, EMu Unit, Australian Museum.

1 Access Lesson 6 Integrating Access Microsoft Office 2010 Introductory Pasewark & Pasewark.

Software Development Unit 2 Databases What is a database? A collection of data organised in a manner that allows access, retrieval and use of that data.

An Introduction to Content Management. By the end of the session you will be able to... Explain what a content management system is Apply the principles.

Seiler Instrument January 7, 2013

METADATA Research Data Management. What is metadata? Metadata is additional information that is required to make sense of your files – it’s data about.

11 Games and Content Session 4.1. Session Overview  Show how games are made up of program code and content  Find out about the content management system.

Advanced File Processing

FireRMS SQL Audit, Archiving & Purging Presented by Laura Small FireRMS Quality Assurance.

CFR 250/590 Introduction to GIS, Autumn 1999 Data & Project Management © Phil Hurvitz, proj_mgmt.ppt 1 Overview Managing non-ARC/INFO data sets.

Gene Expression Omnibus (GEO)

AERONET Web Data Access and Relational Database David Giles Science Systems and Applications, Inc. NASA Goddard Space Flight Center.

Recordkeeping for Good Governance Toolkit Digital Recordkeeping Guidance Funafuti, Tuvalu – June 2013.

Miscellaneous Excel Combining Excel and Access. – Importing, exporting and linking Parsing and manipulating data. 1.

E-Commerce: Introduction to Web Development 1 Dr. Lawrence West, Management Dept., University of Central Florida Topics What is a Web.

ARSF Data Processing Consequences of the Airborne Processing Library Mark Warren Plymouth Marine Laboratory, Plymouth, UK RSPSoc 2012 – Greenwich, London.

CIS750 – Seminar in Advanced Topics in Computer Science Advanced topics in databases – Multimedia Databases V. Megalooikonomou Introduction.

The DigiTool to FDA Program Lydia Motyka Florida Center for Library Automation.

Chapter 6 Generating Form Letters, Mailing Labels, and a Directory

An introduction to MEDIN Data Guidelines. What MEDIN data guidelines are not… Protocols for collection methods Prescriptive of how you have to collect.

Database Basics   Describe the basic organization of a database.   Summarize advantage to using database software.   Define GIGO, and explain how.

School Census Summer 2008 for Secondary Schools Jim Haywood – Version 1.1.

Virtual Ice Charting System Archive Browser Interface Distribution IngestProduction Ice Analyst Application Database Click on the boxes for more information.

EMIS-R Data Collector Uncovered Teresa Williams NWOCA/SSDT OAEP

Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command to search for.

Chapter Five Advanced File Processing Guide To UNIX Using Linux Fourth Edition Chapter 5 Unix (34 slides)1 CTEC 110.

FILES AND ASSETS PANELS

IB FieldBook User Guide Graham McLaren May Introduction The Workbench opens with an assumed active 'breeding project' and an empty list of 'studies'

VO Sandpit, November 2009 CEDA Metadata Steve Donegan/Sam Pepler.

Sea Ice Mapping Systems Archive Browser Interface Distribution IngestProduction Ice Analyst Application Database Henrik Steen AndersonDMI Paul SeymourNIC.

Archive Access Tool Review of SSS Readiness for EVLA Shared Risk Observing, June 5, 2009 John Benson Scientist.

EndNote. What is EndNote? EndNote is referencing software that enables you to create a database of references from your readings.

Common Archive Observation Model (CAOM) What is it and why does JWST care?

+ Information Systems and Databases 2.2 Organisation.

VO Sandpit, November 2009 e-Infrastructure for Climate and Atmospheric Science Research Dr Matt Pritchard Centre for Environmental Data Archival (CEDA)

Sea Ice Mapping Systems Archive Browser Interface Distribution IngestProduction Ice Analyst Application Database Henrik Steen Anderson DMI Virtual System.

Copyright © Software Carpentry 2011 This work is licensed under the Creative Commons Attribution License See

The DSpace Course Module – An introduction to metadata in DSpace.

General Purpose Packages DATA TYPES. Data Types Computer store information in the form of data. Information has meaning. Eg 23 May 2005 Data has no meaning.

1 Overview Finding and importing data sets –Searching for data –Importing data_.

MySQL Importing and creating a database. CSV (Comma Separated Values) file CSV = Comma Separated Values – they are simple text files containing data which.

INTRODUCTION TO GIS  Used to describe computer facilities which are used to handle data referenced to the spatial domain.  Has the ability to inter-

December 2011 n 4ce Quick Start Guide Confirm DBX and Select Format (FTR) file. Choose GSI format including Pre/Post Coding and other system formats. If.

Introduction to KE EMu Unit objectives: Introduction to Windows Use the keyboard and mouse Use the desktop Open, move and resize a.

Introduction to KE EMu Unit objectives: Introduction to Windows Use the keyboard and mouse Use the desktop Open, move and resize a.

AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.

Publishing Combined Image & Spectral Data Packages Introduction to MEx M. Sierra, J.-C. Malapert, B. Rino VO ESO - Garching Virtual Observatory Info-Workshop.

CAA Database Overview Sinéad McCaffrey. Metadata ObservatoryExperiment Instrument Mission Dataset File.

Forensic Investigation Techniques Michael Jones. Overview Purpose People Processes Michael Jones2Digital Forensic Investigations.

UNIT 3 – MODULE 5: Data Input & Editing. INTRODUCTION Putting data into a computer (called data coding) is a fundamental process for virtually all GIS.

BRIEF INTRODUCTION: AND EPID 745, April 3, 2012 Xiaoguang Ma.

GNU EPrints 2 Overview Christopher Gutteridge 19 th October 2002 CERN. Geneva, Switzerland.

Making FAAM Flights Discoverable

Unit 1 ICT Skills for Business

TRAINING OF FOCAL POINTS ON THE CountrySTAT/FENIX SYSTEM

12 Product Configurator

5 Tips for Upgrading Reports to v 6.3

Virtual Ice Charting System

APE EAD3 introduction - DARIAH - Brussels

Presentation transcript:

VO Sandpit, November 2009 Airborne Research and Survey Facility Data CEDA summer work placement Nicola Siddall

VO Sandpit, November 2009 Introduction ARSF overview Sorting and summarising ARSF metadata held in legacy formats MOLES review Preparation of metadata for FAtCat Unix scripts - Retrieve old data from Atlas Write scripts to transfer to ingest directory Generate header files, format data, preview images and information files ready for archive Use deposit script to transfer to archive Move to FAtCat NERC ARSF Dornier aircraft

VO Sandpit, November 2009 ARSF Metadata Began by cross-checking operations reports against NEODC archive and atom editor:

VO Sandpit, November 2009 Metadata Objects for Linking Environmental Sciences ARSF MOLES v.2 structure at its most basic: Approximately 1400 valid activities and deployments Also edited entire set of NEODC data entities in-line with the BADC metadata quality report (understandable title, explained acronyms, suitable summary etc.). Data Entity Deployment Observation Station Data Production Tool Activity e.g. Flight 86/01: Brooms Barn area e.g. ARSF e.g. Data from x on y During Flight 86/01 over the Brooms Barn Area e.g. Piper Navajo Chieftain Aircraft e.g. Photographic camera Further MOLES info:

VO Sandpit, November 2009 Metadata Quality Report

VO Sandpit, November 2009 Deployment Example

VO Sandpit, November 2009 How Can This Help?

VO Sandpit, November 2009 Format of Data Computer compatible tapes held in Atlas tape store Unprocessed, no metadata, – raw A case of informed guesswork and ‘data archaeology’ CCT1986 format 1982 format

VO Sandpit, November 2009 Image Generation

VO Sandpit, November 2009 Image Generation Pre-1984 problem: Can only be displayed in 1 band, but still accurate Data format very unclear – displays distorted image split into different bands

VO Sandpit, November 2009 Script Development Separate the two paths If project exists in archive, create new directory in ingest directory with the code taken from the txt file and new ‘ATM’ directory Then go through each ‘sub’ tape in the tape directory Make /L10 and /raw directories in /ATM

VO Sandpit, November 2009 Script Development Create header file in /L10 directory Copy original tapes into /raw directory Create.bil files from raw tapes and copy these into /L10 Generate preview image in /L10

VO Sandpit, November 2009 Script Development Create.tar.gz tape archive in the /raw directory Create a readme file in each /L10 directory

VO Sandpit, November 2009 New Archive... /neodc/arsf/1986/86_01/ATM/...

VO Sandpit, November 2009 New Archive... /neodc/arsf/1986/86_01/ATM/L10.hdr.png.bil

VO Sandpit, November 2009 Into FAtCat Specific fields in csv format ready for the FAtCat relational databases Issues – Spatial coverage not actually specific to each tape Supplementary spatial coverage from elsewhere not in master spreadsheet Dates already on deployments don’t match master spreadsheet exactly echo $bilpath";"$hdrpath";"$pngpath";"$date";ATM;L10;"$size";"$location","$country";"$long";"$lat /neodc/arsf/1986/86_01/ATM/L10/A12180_1.bil;/neodc/arsf/1986/86_01/ATM/L10/A12180_1.hdr;/neodc/ar sf/1986/86_01/ATM/L10/A12180_1.png; ;ATM;L10; ;BROMS BARN,ENGLAND;0.6;52.27

VO Sandpit, November 2009 What’s Left Create deployments for newly created activities 1982 and 1983 image generation and data ingestion Scan all operations reports into CEDA repository, then link each deployment to the relevant report Rename a lot of archive directories... ? ?

VO Sandpit, November 2009 Questions?