Staying afloat in the sensor data deluge

Slides:



Advertisements
Similar presentations
1 Introduction to XML. XML eXtensible implies that users define tag content Markup implies it is a coded document Language implies it is a metalanguage.
Advertisements

Computational Physics Kepler Dr. Guy Tel-Zur. This presentations follows “The Getting Started with Kepler” guide. A tutorial style manual for scientists.
GLEON Data Management Luke Winslow PASEO 3/18/09.
Mgt 240 Lecture Website Construction: Software and Language Alternatives March 29, 2005.
Tools for Publishing Environmental Observations on the Internet Justin Berger, Undergraduate Researcher Jeff Horsburgh, Faculty Mentor David Tarboton,
Business Computing 550 Lesson 4. Fundamentals of Information Systems, Fifth Edition Chapter 4 Telecommunications, the Internet, Intranets, and Extranets.
1 Networks and the Internet A network is a structure linking computers together for the purpose of sharing resources such as printers and files Users typically.
Scientific Data Infrastructure in CAS Dr. Jianhui Scientific Data Center Computer Network Information Center Chinese Academy of Sciences.
Southern Sierra CZO Funding. The instrument cluster and the CZO are supported by NSFs Earth Sciences Division. KREW is a program of the U.S. Forest Service.
CPS120: Introduction to Computer Science The World Wide Web Nell Dale John Lewis.
Object and component “wiring” standards This presentation reviews the features of software component wiring and the emerging world of XML-based standards.
material assembled from the web pages at
Speciation by symbiosis Robert M. Brucker, Seth R. Bordenstein Trends in Ecology & Evolution Volume 27, Issue 8, Pages (August 2012) DOI: /j.tree
Indo-US Workshop, June23-25, 2003 Building Digital Libraries for Communities using Kepler Framework M. Zubair Old Dominion University.
1 An Overview of Telecommunications Telecommunications: the electronic transmission of signals for communications Telecommunications medium: anything that.
A framework to support collaborative Velo: Knowledge Management for Collaborative (Science | Biology) Projects A framework to support collaborative 1.
Web Design and Development for E-Business By Jensen J. Zhao Copyright 2003 Prentice Hall, Inc. Web Design and Development for E-Business Jensen J. Zhao.
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
Writing Scientific Papers Additional materials required for manuscript preparation and submission Prof Steve Leharne.
The Future of the iPlant Cyberinfrastructure: Coming Attractions.
1 Senn, Information Technology, 3 rd Edition © 2004 Pearson Prentice Hall James A. Senn’s Information Technology, 3 rd Edition Chapter 12 Creating Web-Enabled.
Visualization Workshop David Bock Visualization Research Programmer National Center for Supercomputing Applications - NCSA University of Illinois at Urbana-Champaign.
Ihr Logo Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang.
5 - 1 Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved.
How do emotion and motivation direct executive control? Luiz Pessoa Trends in Cognitive Sciences Volume 13, Issue 4, Pages (April 2009) DOI: /j.tics
MANAGING DATA RESOURCES ~ pertemuan 7 ~ Oleh: Ir. Abdul Hayat, MTI.
XML and SVG from PQL By Dave Doulton Computing Services University of Southampton.
XML Engr. Faisal ur Rehman CE-105T Spring Definition XML-EXTENSIBLE MARKUP LANGUAGE: provides a format for describing data. Facilitates the Precise.
CyberInfrastructure for Network Analysis Importance of, contributions by network analysis Transformation of NA Support needed for NA.
Breakout # 1 – Data Collecting and Making It Available Data definition “ Any information that [environmental] researchers need to accomplish their tasks”
XML Alyssa Roberts. What is XML? Extensible Markup Language Specification to creating custom mark-up languages Simplified version of SGML, originally.
Does the contraceptive pill alter mate choice in humans? Alexandra Alvergne, Virpi Lummaa Trends in Ecology & Evolution Volume 25, Issue 3, Pages
CSCE 315 – Programming Studio Spring Goal: Reuse and Sharing Many times we would like to reuse the same process or data for different purpose Want.
Module: Software Engineering of Web Applications Chapter 2: Technologies 1.
John Porter Sheng Shan Lu M. Gastil Gastil-Buhl With special thanks to Chau-Chin Lin and Chi-Wen Hsaio.
Satisfying Requirements BPF for DRA shall address: –DAQ Environment (Eclipse RCP): Gumtree ISEE workbench integration; –Design Composing and Configurability,
A WEB-ENABLED APPROACH FOR GENERATING DATA PROCESSORS University of Nevada Reno Department of Computer Science & Engineering Jigar Patel Sergiu M. Dascalu.
Project number: ENVRI and the Grid Wouter Los 20/02/20161.
Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.
Ocean Observatories Initiative OOI Cyberinfrastructure Life Cycle Objectives Review January 8-9, 2013 Scientific Workflows for OOI Ilkay Altintas Charles.
A WEB-ENABLED APPROACH FOR GENERATING DATA PROCESSORS University of Nevada Reno Department of Computer Science & Engineering Jigar Patel Sohei Okamoto.
V7 Foundation Series Vignette Education Services.
Data Management: Data Processing Types of Data Processing at USGS There are several ways to classify Data Processing activities at USGS, and here are some.
The Virtual Observatory and Ecological Informatics System (VOEIS): Using RESTful architecture and an extensible data model to provide a unique data management.
CyVerse Workshop Discovery Environment Overview. Welcome to the Discovery Environment A Simple Interface to Hundreds of Bioinformatics Apps, Powerful.
Managing Data Resources File Organization and databases for business information systems.
Information Retrieval in Practice
Principles of IT Basic Webpage Design Vocabulary.
CyVerse Discovery Environment
INTRODUCTION TO GEOGRAPHICAL INFORMATION SYSTEM
Computer Software Lecture 5.
Data Virtualization Tutorial: XSLT and Streaming Transformations
Visio.
Visio.
A Web-enabled Approach for generating data processors
Markup Languages Gilok Choi 9/17/2018
Tutorial 8 Objectives Continue presenting methods to import data into Access, export data from Access, link applications with data stored in Access, and.
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
MANAGING DATA RESOURCES
DIGITAL LIBRARY.
Objective Understand web-based digital media production methods, software, and hardware. Course Weight : 10%
An ecosystem of contributions
Big Data Overview.
Computational Physics Kepler
User interface design.
Middleware, Services, etc.
Ecoinformatics: supporting ecology as a data-intensive science
Microsoft Office Illustrated Fundamentals
Remedy Integration Strategy Leverage the power of the industry’s leading service management solution via open APIs February 2018.
Presentation transcript:

Staying afloat in the sensor data deluge John H. Porter, Paul C. Hanson, Chau-Chin Lin  Trends in Ecology & Evolution  Volume 27, Issue 2, Pages 121-129 (February 2012) DOI: 10.1016/j.tree.2011.11.009 Copyright © 2011 Elsevier Ltd Terms and Conditions

Figure 1 A generic view of sensor data processing. Sensor data typically pass through a series of steps or levels, although what is required to reach a given level can vary across projects. A special challenge for sensor data is finding ways to uniquely identify versions of the data for citation, when, in principle, each new line of data can define a new version of the data. Most publically-available sensor data is Level 1 data and not all projects create Level 2 data. Trends in Ecology & Evolution 2012 27, 121-129DOI: (10.1016/j.tree.2011.11.009) Copyright © 2011 Elsevier Ltd Terms and Conditions

Figure 2 Scientific workflow tools provide graphical interfaces for capturing and executing complex data manipulations and analyses. This Kepler workflow uses EML metadata to produce and run an R statistical language program that produces quality assurance reports and graphs. It incorporates a variety of tools including an eXtensible Markup Language (XML) stylesheet processor, a text editor, R statistical programs and text and graphical display tools. These are a small subset of the capabilities built into Kepler, which also include remote processing, database, mathematical and data conversion tools. Encapsulation of such diverse capabilities within a single graphical environment reduces the need for external documentation and facilitates sharing. Such workflows can be easily transferred between users, used to replicate analyses or further customized to add new capabilities or analyses. Trends in Ecology & Evolution 2012 27, 121-129DOI: (10.1016/j.tree.2011.11.009) Copyright © 2011 Elsevier Ltd Terms and Conditions

Figure I Sensor nodes and sensor networks come in different sizes and shapes. Sensor nodes include common components (a) but can vary in size from a ‘mote,’ which incorporates light sensors, processor and radio into a compact battery-powered unit (b), to a large installation such as a carbon flux tower (c), which incorporates temperature, wind, water level and CO2 sensors, data loggers and computers. Sensor nodes can be interconnected using star (d), mesh (e) and hierarchical (f) topologies. Sensor nodes are shown as circles and network links as dashed lines. Sensor nodes shown with a solid fill are used to transfer data out of the sensor network to researchers. Hierarchical topologies are frequently used for sensor networks where there are multiple study locations, each with its own sensor network. More powerful radios are used for the inter-site links whereas low powered radios can be used within a site. Trends in Ecology & Evolution 2012 27, 121-129DOI: (10.1016/j.tree.2011.11.009) Copyright © 2011 Elsevier Ltd Terms and Conditions

Figure I The ‘Cyberinfrastructure (CI) Ecosystem’ associated with GLEON. Sensor data from lake observatories stream to the data repository named Vega. Traditionally sampled data are collected in LakeBase. Data from both repositories can be exported to formats for use in common analysis software. Condor provides distributed computing to support complex analyses with long run times. Although data continually stream from sensing platforms to Vega, human intervention is required when platform changes are made. Because analysis models often are innovated to suite the science questions, data export to data analysis is manual, except for simple visualization, export to Web pages, common transformations, and synchronization of multiple variables. Trends in Ecology & Evolution 2012 27, 121-129DOI: (10.1016/j.tree.2011.11.009) Copyright © 2011 Elsevier Ltd Terms and Conditions