1. www.eia.gov U.S. Energy Information Administration Independent Statistics & Analysis Transforming legacy data into state-of- the-art interactive visualizations September 6, 2013 | Washington, D.C. Mark Elbert Director, Office of Web Management

2. What to do when your data is in legacy table structures? Mark Elbert, Washington, D.C., September 6, 2013 2

3. Challenge: transform EIA’s data publishing 3 chose electricity data, with over 400,000 time series, to break the paper-centric publishing model different customers consume data in different ways: –researchers and energy analysts –classroom of students –Hill staffers –application developers push the boundaries of interactive charting and mapping on the web Mark Elbert, Washington, D.C., September 6, 2013

4. The Internet’s fallow period: ’03 - ’10 4 The internet turned into a powerful computing platform despite Microsoft –Telcos gave us multi megabit bandwidth –Google’s Chrome started the new browser war for speed –processors continued to improve open-source JavaScript libraries matured –jQuery / jQueryUI –jVectorMaps –Highcharts Mark Elbert, Washington, D.C., September 6, 2013

5. Web Data Ingestion and Application Architecture Mark Elbert, Washington, D.C., September 6, 2013 5 legacy tables Web DB Filtering & collating Indexing engine Data mapper Data writer (JSON)

6. The Electricity Data Browser Mark Elbert, Washington, D.C., September 6, 2013 6 aggregate data drill-downs to individual plant data user generated visualizations: –Line charts –Column charts –Animated maps links to analysis, data API Cross links to U.S. Energy Mapping System

7. Data ecosystem Mark Elbert, Washington, D.C., September 6, 2013 7 {"request":{"command":"series","series_id":"ELEC.PLANT.GEN.6 417-WAT- ALL.A"},"series":[{"series_id":"ELEC.PLANT.GEN.6417-WAT- ALL.A","data":[["2012","244685"],["2011","390119"],["2010","2510 53"],["2009","377491"],["2008","181726"],["2007","102447"],["200 6","292362"],["2005","263951"],["2004","471543"],["2003","44429 3"],["2002","422206"],["2001","285082"]],"name":"Net Generation : Center Hill : Conventional Hydroelectric : All Primemovers : Annual","units":"megawatthours","f":"A","unitsshort":"","descriptio n":"","copyright":"None","source":"EIA, U.S. Energy Information Administration","iso3166":"USA-TN","lat":"36.0972","lon":"- 85.8274","start":"2001","end":"2012","updated":"22-MAR-13 03.12.35 PM"}]}

8. Demo: www.eia.govwww.eia.gov 8 Mark Elbert, Washington, D.C., September 6, 2013

9. 9

10. 10

11. Mark Elbert, Washington, D.C., September 6, 2013 11

12. Mark Elbert, Washington, D.C., September 6, 2013 12

13. Mark Elbert, Washington, D.C., September 6, 2013 13

14. Mark Elbert, Washington, D.C., September 6, 2013 14

15. each data set is a complete n-Cube Mark Elbert, Washington, D.C., September 6, 2013 15 Ragged n-Cube Complete n-Cube

16. Mark Elbert, Washington, D.C., September 6, 2013 16 NameDescriptionLicenseVersionSizeFirst publishedDependencies jQueryhelper library; simplifies JavaScript coding open-sourcev1.7.192kBJan-06- jQuery UIuser interface controlopen-sourcev1.8.7202kBSep-07jQuery Highstockscharting librarycommercial ($80)v1.1.4117kBOct-11jQuery jVector Mapmap infographics libopen-sourcev0.123kBFeb-12jQuery USA map definition open-sourcev0.147kBFeb-12jVector Map, jQuery SlickGridfast, scalable grid componentopen-sourcev2.012kBMar-09jQuery, jQuery.event. drag JQuery Event Dragextends jQuery; used by Slickopen-sourcev2.05.1kB jQuery jQuery mousewheelsimplifies mousewheel programming open-sourcev3.0.63.5kBApr-07jQuery Loadmaskcreates wait screen effectsopen-sourcev0.44.1kBJun-09jQuery qTipshelp tipsopen-sourcev1.022kBApr-10jQuery Curlloads library as neededopen-sourcev0.6.114.2kBJan-11- Classyobjectoriented extension for JavaScript open-sourcev1.44.99kBApr-10jQuery Crossroadspattern matching frameworkopen-sourcev0.7.15kBApr-11- Hasherurl hash managementopen-sourcev1.12.8kBAug-11- Mustachelogic-less template systemopen-sourcev0.514kBOct-09- SimpleModalcreates modal panel (greys rest of screen) open-sourcev1.4.223kBOct-07jQuery JSON2provides JSON support for obsolete browsers open-sourcev1.05kBNov-10- PureMVCmodel-view-controller framework open-sourcev1.17kBNov-10- js-Signalsevent frameworkopen-sourcev0.7.43.1kBNov-10- Kizzycross-browser localStorage APIopen-sourcev1.04.6kBFeb-11- Open Source libraries = 2/3 code

17. Mark Elbert, Washington, D.C., September 6, 2013 17 Questions? Mark Elbert mark.elbert@eia.gov markelbert@gmail.com mark.elbert@eia.gov markelbert@gmail.com