1. TWC LOGD: A Portal for Linking Open Government Data Li Ding, Deborah L. McGuinness, Jim Hendler Tetherless World Constellation Rensselaer Polytechnic Institute Presented by Li Ding at Northwestern University Dec 1, 2010

2. 2 The TWC LOGD Portal Highlights Real World Data  US, UK, China,…  Health, energy, economy End User Applications  Community Portal  Fast, Low-cost Mashups Applied Semantic Web  Major partner of Data.gov  8.5 billion triples in LOD

3. Semantic Web Deployed at Data.gov http://www.data.gov/semantic

4. 4 Data.gov and World-Wide Open Government Data Activities January 1, 2009 “Openness will strengthen our democracy and promote efficiency and effectiveness in Government.” --- President Obama Putting Government Data online May 21, 2009 January 19, 2010 data.gov.uk online May 21, 2010 data.gov online data.gov relaunch with semantic web featured June30,2009 2009 2010 … Many countries US UK Australia New Zealand …

5. 5 First anniversary of Data.gov Semantic Web and RDF logo showed up on the frontpage of the US Data.gov website

6. 6 Semantic Web deployed at Data.gov: RDF data, SPARQL endpoint, semantic mashups

7. 7 RPI featured as a major partner of the US Data.gov project

8. 8 Government Adoption Process Data-gov Wiki @RPI online May 21, 2009 May, 2010 data.gov online SPARQL End Point & RDF data & Demos Replicated at Data.gov July,2009 2009 2010 … Demos Tutorials Videos SPARQL Endpoint 2009-2010 Oct, 2010 New Application published by a team at DOE Two-day Mashathon in Washington DC Aug, 2010 May 21, 2010 data.gov relaunch with semantic web featured TWC LOGD Drupal Site announced Oct, 2010

9. The Largest Real World LOD Dataset http://logd.tw.rpi.edu/twc-logd

10. Categories of Data.gov Datasets  Statistical data about various aspect of society  Over 3000 Datasets

11. Raw Government Data Now Metadata in PDF Data in Excel

12. Conversion: From Raw Tabular Data to RDF

13. Enhancement: Linking Open Government Data IDyearPHSY_STsite-idcost 199810.0 1999site12311.3 2000NY8.3 200120 site-idLatitudelongitude site12343.993-70.326 Year claims 2000382 PHSY_ST: state abbreviation ID: unique id cost: unit is million US dollars year: 1975-2008 Correlated dataset Complement dataset Metadata (field definition) Metadata (value definition) owl:sameAs DS123:NY

14. 14 The Largest Real World LOD Dataset  8.5+ billion triples from real world  7500+ LOD links  Accessible via Data Browser, e.g. Tabulator

15. Consuming Linked Open Government Data http://logd.tw.rpi.edu/demos

16. LOGD Application UI TWC LOGD data.gov.uk dbpedia W W W SPARQL Query SPARQL Results Format Data JSONXMLCSV Visualize DataQuery DataIntegrate Data LOGD Consumption Workflow

17. Exhibit Visualization API Data.gov CASTNET Ozone (CSV) epa.gov CASTNET Site (CSV) Convert raw dataset into linkable RDF Data MashupWeb Application Mashup Visualization Mashup query multiple RDF dataset via SPARQL end point surf to EPA applications 1 2 drill down for details 3 4 Created by Dominic DiFranzo, PhD student at RPI, http://www.data.gov/semantic/Castnet/html/exhibithttp://www.data.gov/semantic/Castnet/html/exhibit Mashing up LOGD Data

18. 18 Trends in Smoking Prevalence, Tobacco Policy Coverage and Tobacco Prices (1991-2007) Smoking Prevalence vs. Tax, Policy … Extensible and accountable Mashups with NCI Extensible Mashups via Linked Data  Diverse datasets from NIH  Potentially linking to “unemployment rate” Accountable Mashups via Provenance  Annotate datasets used in demos  Feedback users’ comment to gov contact (e.g. %) Created by Li Ding, Tim Lebo, RPI, http://logd.tw.rpi.edu/project/popscigridhttp://logd.tw.rpi.edu/project/popscigrid

19. Smoking Prevalence vs. Other Factors Integrating different sources for discovery Created by Sarah Magidson, U. Chicago. http://data-gov.tw.rpi.edu/demo/stable/tobacco-smoker/demo-state-10026-smoke-rate-statevarsapi.htmlhttp://data-gov.tw.rpi.edu/demo/stable/tobacco-smoker/demo-state-10026-smoke-rate-statevarsapi.html [Spatial Mashup] Data.gov (Population) + NIH (Tobacco Tax, Smoking rate) Gov data provides knowledge for poplation science study

20. 20 Linking GDP of the US and China Linking international government data meaningfully GDP of China (Billion Chinese Yuan ) GDP of the US (Billion Dollar) [Temporal Mashup] bea.gov + federalreserve.gov +stats.gov.cn 8.3 6.3 20002010 Created by Li Ding, RPI, http://logd.tw.rpi.edu/demo/linking_us_and_chinas_gdp_data/http://logd.tw.rpi.edu/demo/linking_us_and_chinas_gdp_data/

21. 21 XHTML+RDFa ARC2 http://data-gov.tw.rpi.edu/ Semantic Search on LOGD data rich snippet in results Web Search (HTML) Rich Snippet (RDFa)

22. Adding Social Factor to Mashups RDF Publish* Enhance* User Raw Data consume* feedback Import socially contributed data, e.g. DBpedia Let users contribute –links –feedbacks Other Social Web Apps Import/export

23. Wildland fire (NIFC) Budget on wildfire “DOI” and “USDA” (OMB) Category:Wildfires In The United States Created by Li Ding, RPI, http://data-gov.tw.rpi.edu/demo/stable/demo-1187-40x-wildfire-budget.htmlhttp://data-gov.tw.rpi.edu/demo/stable/demo-1187-40x-wildfire-budget.html [Temporal Mashup] Data.gov (statistics+ budget) + Wikipedia (famous fires) US Wildland Fire and Budget Linking to Wikipedia (socially contributed)

24. 24 White House Visitor Search Leveraging linked data (DBpedia & New York Times) “POTUS” dbpedia:Barack_Obama Created by Dominic DiFranzo, Evan Patton, RPI, http://data-gov.tw.rpi.edu/demo/stable/white-house-visitor/top100-visitees.phphttp://data-gov.tw.rpi.edu/demo/stable/white-house-visitor/top100-visitees.php  [Person Mashup] Data.gov (statistics) + DBpedia (personal profiles)+ NYTimes (news)  [Technologies] Semantic MediaWiki, Google Visualization, IPad Apps available in Apple Store The White House Semantic Wiki Wikipedia NYTimes

25. Created by Sarah Magidson, http://data-gov.tw.rpi.edu/demo/linked/demo-401-usps-news.htmlhttp://data-gov.tw.rpi.edu/demo/linked/demo-401-usps-news.html [Temporal Mashup] Data.gov (budget) + USPS + User Contributed News USPS Spending and News government data + User Feedbacks

26. Current Status of TWC LOGD http://data-gov.tw.rpi.eduhttp://data-gov.tw.rpi.edu => http://logd.tw.rpi.eduhttp://logd.tw.rpi.edu (Semantic MediaWiki) (Drupal + RDFa)

27. 27 Website Statistics 378,128 page hits 28,481 visits 16,041 visitors 4126 cities 34 countries Note: the above statistics are about http://data-gov.tw.rpi.edu. Dataset access not counted.http://data-gov.tw.rpi.edu

28. 28 Dataset Version Table Source Record Conversion Layer OGD (part1) Snapshot LOGD (raw) LOGD (e1) highLevels of structural data granularity low Data publishing stages OGD (part2) Snapshot … … … Data Abstraction and Versioning

29. 29 Provenance and Workflow Convert derive create derive revision Access Enhance Version SemDiff

30. 30 Linking Open Source Community Linking semantic web with web developers Social Semantic Web extensions/modules to popular CMS, e.g. Semantic Wiki, Drupal Process/consume integrated gov data in a number of different ways: social networks, natural language technologies, workflows, search…

31. Education: Linked Tutorials, Demos… project demo technology tutorial video dataset source person dcterms:contributor logd:uses_dataset logd:uses_technology logd:uses_datasource dcterms:relation dcterms:source http://logd.tw.rpi.edu/tutorials

32. 32 Summary of the TWC LOGD Portal Real World Data  8.5+ billion triples  400+ datasets  10+ sources  Many domains Semantic Web Technology  completely open source  Demos/tutorials/videos Community and Users  partner of US government  open source community  education in university http://logd.tw.rpi.edu Beyond just dogfood; Linking Open Government Data Now!

33. The Team and Sponsors Leaders –Jim Hendler –Deborah L. McGuinness –Li Ding Members –Dominic DiFranzo –Sarah Magidson –James Michaelis –Alvaro Graves –Jin Guang Zheng –Xian Li –Gregory Todd Williams –Tim Lebo –Zhenning Shangguan –Devin Gaffney –Peter Coons –Adam Bell –William Cooper –Brian Zaik –Johanna Flores 33 Government Sponsors DARPA NSF NASA IARPA NIH/NCI …