ERDDAP The Next Generation of Data Servers Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA Disclaimer: The opinions expressed here are my own.

In the old days, we had dock workers

Now, we have cranes, containers and container ships.

What do we have Now for data distribution? Labor intensive: Each dataset handled specially Minimal software reuse Limited ways to search for datasets of interest Different request APIs for each dataset: 1000's? With one response format: Often not the one the user wants No Consistency: Everything is a custom job and works differently.

How can we get to the next generation? By following standards By using reusable FOSS software

My offering for the next generation is ERDDAP which is already installed and used at more than 50 organizations around the world. ERDDAP is on NOAA's list of recommended data servers for use by groups within NOAA.

ERDDAP will give you 2 Things You Already Have An easy way for users to request data The response in some file format.

And ERDDAP will give you 20 Things You Probably Don't Have

No More Programming You don't have build a car in order to drive a car. Why should you have to write the code for a data server in order to distribute data?

ERDDAP is FOSS So it's free. And someone else maintains it. And if you ever need to modify or customize it, you can.

ERDDAP is Reusable Software ERDDAP is designed to be quickly and easily installed at other sites. One ERDDAP can serve multiple datasets. I have ~6000 in one ERDDAP, but ERDDAP is designed to handle a million datasets.

Installing ERDDAP and adding datasets is Fast and Easy Install ERDDAP and add your first dataset in hours. Add additional datasets in minutes.

ERDDAP can get data from Multiple Data Sources Local files:.csv,.tsv,.nc,.grib,.bufr,... Relational databases Remote services: DAP, SOS, ERDDAP,...

You can make a big dataset from a Collection of Files This is essential for large datasets and real-time datasets.

For every dataset, there are Web Pages for humans and Web Services for computers. And the web pages are optimized for being crawled by search engines (SEO).

Users can request data via a Simple, Powerful, RESTful Query API. They are (OPeN)DAP standard requests. They are a simplified subset of SQL. One URL specifies an entire request.

The user specifies the Response Format Metadata:.das,.dds Data files:.csv,.json,.nc,.mat,.dods,.xhtml,... with subsets of the dataset Image files:.png,.pdf with custom graphs and maps Web pages:.html,.graph,.subset

Every dataset has Metadata so users can find out the who, what, when, where, why, how of the dataset and understand the dataset.

Time data is presented in a consistent way. As ISO 8601 standard strings: T23:00:00Z Or, as UDUnits standard numbers: seconds since T00:00:00Z

Latitude and longitude are presented in a consistent way as decimal degrees.

Since there are multiple datasets, ERDDAP creates a Catalog of Datasets

Users can Search for Interesting Datasets Google-like Faceted/Category Advanced: including Lat, Lon, Time bounds OpenSearch 1.1 Standard

There are Subscriptions so users can be notified when a dataset changes.

ERDDAP automatically generates FGDC and ISO Metadata

One ERDDAP can re-serve datasets from another ERDDAP without actually transferring the data, so you can set up Federations of ERDDAPs

I didn't make this up. This is all based on Standards! OpenSearch 1.1 OPeN(DAP) data requests and responses WMS CF, ACDD, FGDC, ISO Metadata UDUnits and UCUM Units ISO 8601:2004(E) Time Format Numerous File Format Standards

The result is Consistency! Like containers on container ships: vastly more efficient Good for data providers: faster and easier to set up data services Good for users: a consistent way to find datasets and get data

It's your choice.

Please Give ERDDAP a try! Thank you!