1. Using Data Science as Evidence in Public Policy With Big Data and Elections Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ November 1-2, 2012 http://semanticommunity.info/CNSTAT 1

2. Start by Asking Questions Which by State, Congressional District, and which by time? Which is the easiest to reformat? Which is the most interesting? Where have the candidates been? Which data is free? Etc. 2 Note: Drew Conway (@drewconway) speaking about the joys, challenges, and power of data science. "Data science, as a discipline, is fundamentally about human behavior.” http://semanticommunity.info/AOL_Government/2012_Recorded_Future_User_ConferenceDrew Conway http://semanticommunity.info/AOL_Government/2012_Recorded_Future_User_Conference

3. Then Look for the Evidence Brainstorm: – What Have I Done Before? 2012 Annual Statistical Abstract: – Chapter 7. Elections Google Searches: – Election and Voting Data Conferences: – National Academy Seminars Television: – Debates, etc. 3

4. Begin With the End In Mind (Stephen Covey) Story (publicity and money) Research Notes (document what I did and learned) Conditioned Data Sets (added value) Spotfire Dashboard (cool visualizations) Lecture to Students at George Mason University (help them learn what a data scientist/data journalist does) 4

5. My 5-Step Method So what I like to do to illustrate (data science) and explain (data journalism) in the following (like a recipe): – Put the Best Content into a Knowledge Base (e.g. MindTouch) The 2012 Annual Statistical Abstract, CNSTAT, etc. – Put the Knowledge Base into a Spreadsheet (Excel) Linked Data to Subparts of the Knowledge Base – Put the Spreadsheet into a Dashboard (Spotfire) Data Integration and Interoperability Interface – Put the Dashboard into a Semantic Model (Excel) Data Dictionaries and Models – Put the Semantic Model into Dynamic Case Management (Be Informed) Structured Process for Updating Data in the Dashboard 5

6. Knowledge Base 6 http://semanticommunity.info/CNSTAT

7. 2012 Annual Statistical Abstract: Chapter 7. Elections (Visualizations) 7 http://semanticommunity.info/FedStats.net#Section_7_ELECTIONS

8. 2012 Annual Statistical Abstract: Chapter 7. Elections (Metadata) 8 http://semanticommunity.info/FedStats.net#Section_7._Elections

9. FedStat.net: Commemorating over 135 years of making statistics available to citizens everywhere 9 http://semanticommunity.info/FedStats.net#Story

10. FedStats.gov Remains Rich Source Of Government Data For Citizens 10 http://gov.aol.com/2012/07/26/fedstats-gov-remains-rich-source-of-government-data-for-citizens/

11. 2012 Annual Statistical Abstract 11 http://www.census.gov/compendia/statab/

12. Data From CD-ROM to My Server 12 http://semanticommunity.net/StatAbs2012/

13. Spreadsheet 13 http://semanticommunity.info/@api/deki/files/19606/Elections2012.xls

14. Welcome to the Campaign 2012 Interactive Dashboard 14 http://campaign2012.c-span.org/electoral-college-map My Note: Not like the next slide!

15. CNN Electoral Map 15 http://www.cnn.com/ELECTION/2012/ecalculator

16. CNN Electoral Map in Excel 16 http://semanticommunity.info/@api/deki/files/19606/Elections2012.xls

17. CNN Electoral Map in Spotfire 17 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

18. Data Set Inventory and Results 18 http://semanticommunity.info/CNSTAT#Story

19. 2012 Annual Statistical Abstract Election Tables Metadata 19 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

20. Table 397. Participation in Elections for President and U.S. Representatives and Table 402. Vote Cast for President, by Major Political Party 20 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

21. Table 405. Electoral Vote Cast for President by Major Political Party--States 21 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

22. Table 408. Apportionment of Membership in House of Representatives, by State 22 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

23. Table 410. Vote Cast by Congressional Districts: 2010 23 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

24. Cover Page 24 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

25. Conclusions and Suggestions I had the pleasure of attending three very interesting and related professional statistical meetings recently that showed that statisticians really care about current issues. This made me appreciate that elections are a big data problem that is approached in three basic ways: Historical elections data, Collection and modeling of polling survey data before the election, and Use of social media. So I used inventoried the historical and polling survey data (I could get) to aid in selection and visualization in a dashboard and found I needed both Congressional and State boundary files as shown in a table. So imagine an election season in which we had less or no polls to influence voters so they could focus on the candidates and the issues and then we got an amazing example of big data processing just after the polls closed (by gentleman's agreement with Congress) which we could all participate in by seeing the precinct voting results posted to Twitter and processed by many apps that developers had developed to bring us interesting and useful results. I am eager to see that to happen in 2014 and 2016! I will be updating these results with the final 2012 elections data and providing another story. 25

26. Extra Slides Boundary Files: – US States Repositioned – US Counties Repositioned – US Congressional Districts 1 – US Congressional Districts 2 Sources: – Spotfire https://silverspotfire.tibco.com/us/library – US Census http://www.census.gov/cgi-bin/geo/shapefiles2010/main 26

27. US States Repositioned 27 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

28. US Counties Repositioned 28 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

29. US Congressional Districts 1 29 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire

30. US Congressional Districts 2 30 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?Elections-Spotfire