1. Data Science With Spotfire for Opening Government Data for Innovators and Entrepreneurs
Dr. Brand Niemann
Director and Senior Enterprise Architect – Data Scientist
Semantic Community
AOL Government Blogger
February 18, 2012

2. TIBCO Spotfire 4.0

3. TIBCO Spotfire 4.0
The Top 5 Reasons Why Spotfire Analytics is Better and Smarter:
Clarity of Visualization
Freedom of Spreadsheets
Relevance of Applications
Confidence of Statistics
Reach of Reports

4. TIBCOSilver Spotfire Features Matrix
Free

5. TIBCOSilver Spotfire Tutorials

6. My Silver Spotfire Data Science Library in the Cloud
I will show you examples
of how I built these later.
Federal Budget 2013
in a day!
Data Science Library in the Cloud

7. Federal Budget 2013 Dashboard
PC Desktop Spotfire

8. Federal Budget 2013 Dashboard
Silver Spotfire Web Player

9. The Value Proposition of Spotfire
More to Do with Less?
Take Control of Your Business Data
Visualize Your Data - Drag and Drop Your Spreadsheets
Customize Your Dashboards - Instantly Add New Visualization
Share Your Insights - Publish Your Dashboard
Get Trial of Silver Spotfire
Agile Analysis:
Fastest to Actionable Insight
Insight Into the Unknown
Self-Service Discovery
Universal Analytics Platform
Source:
Source:

10. The Value Proposition of Agile Analysis - Invert your "bath tub" with Spotfire Analytics
Spotfire offers dimension-free data exploration, data mashups, predictive and event driven, contextual collaboration and enterprise class technology.
Source: Jim Hawley, Spotfire Federal Government

11. Spotfire is Part of TIBCO
Video Presentation

12. The Value Proposition of Data Science
We are interested in learning about Taxonomy and Enterprise Vocabulary for fundamental architectural elements to enable interoperability and provide consistent understanding of shared architecture information across the enterprise.
Source: Walt Okon, Senior Architect Engineer, Enterprise Architecture & Standards, Department of Defense Chief Information Officer, October 4th, .
Aneesh Chopra: Government’s Big Data Opportunity. “The Federal Government needs Data Science and Data Scientists!”
Source: O’Reilly STRATA Conference New York, September 20, 2011.

13. What is Data Science?
Data science enables the creation of data products.
Data science is a holistic approach.
The first step of any data analysis project is “data conditioning,” or getting the data into a state where it is usable.
Statistics is the “grammar of data science.”
Edward Tufte’s Visual Display of Quantitative Information is a foundational text for anyone practicing data science. He calls himself a data scientist!
Data scientists are patient, inherently interdisciplinary, and can think outside the box.
Some References:
Data Science Graduate Class at RPI, Troy, NY
Data Science
AOL Government

14. Data Science Architecture
1. Create an inventory of documents and data sets.
2. Build that inventory in an Excel spreadsheet so it supports faceted search in a Spotfire dashboard.
3. Provide a sample knowledgebase of each of the four types of documents (Word, PDF, PowerPoint, and Excel).
4. Provide the multiple sample knowledgebases in a Spotfire dashboard so they can be seen, compared, merged, harmonized, sorted, searched, downloaded, and shared on mobile devices (e.g. iPad).
5. Scale the previous architectural pattern with more content volume and types if necessary.

15. Knowledgebase What is a knowledgebase? How is a knowledgebase built?
Knowledgebase = Model + Instances
Model = Vocabulary, Taxonomy, and Ontology/Rules
Instances = Linked Data Semantically Linked to the Model
How is a knowledgebase built?
Model = Vocabulary – Glossary in MindTouch
Taxonomy – Contents and Resources in MindTouch
Ontology/Rules in Be Informed 4
Instances = Linked Data Semantically Linked to Model – MindTouch, Excel and Spotfire

16. The Knowledgebase in MindTouch
MindTouch is often referred to as the “Swiss Army Knife” of collaboration tools! See MindTouch Web Site.
So I make MindTouch look like a “Knowledge Hub” (e.g., on top of SharePoint Portal like the Army Corps of Engineers Knowledge Hub) and feature key documents and data sets.
Relating one or more Spotfire dashboards to the key document and data sets points to the ability to track progress. It’s all about metrics!

17. MindTouch Social Knowledge Base – Social Help Center
MindTouch provides exceptional, purpose-built social help desks and knowledge bases for some of the world’s largest and most respected technology and media brands. Our solutions layer social and collaborative capabilities over existing systems and deliver strategic value to our customers. Product help is strategic for user assistance teams, product and marketing teams, community managers, and product evangelists as they look to build engaged communities around their brands to increase top and bottom line revenues.

18. Army Corps of Engineers Knowledge Hub
The Knowledge Hub is a dynamic online destination to feature products developed by the US Army Corps of Engineers as well as to engage end-users and others in innovative and intuitive interaction. Within the Knowledge Hub is a Navigation Community which provides a forum on which navigation personnel can discuss, share, learn, explore and search products, project and programs of concern them. One goal of the Hub is to be a web-based framework for enterprise decision support and tech transfer within the Corps of Engineers.
POC: Marty Kittrell,
Source:

19. MindTouch Knowledgebase
AOL Government Story
Spotfire Dashboard
Research Notes (Metadata)
Complete Budget Document
Attachments (see next slide)
Comments (see next slide)

20. MindTouch Knowledgebase

21. Data Science is Part of My System of Systems Architecture
Dynamic Case Management (e.g. Be Informed)
Data Science Library (e.g. Spotfire)
Data Science Products (e.g. Spotfire) S
Semantic Index of
Linked Data (e.g. Excel)

22. Agile Methods: Questions on Our Minds
What Should We Do with Enterprise Architecture?
Be like a building architect that provides a blueprint with building specifications and a scale (able) model.
How Should We Do That?
With Be Informed, an internationally operating, independent software vendor that has been recognized recently by Gartner and Forrester.
What is Be Structured?
It is complimentary to various well-known development, compliance and architecture frameworks, including ITIL, Cobit, Prince II, RUP, TOGAF, Zachman, SCRUM, Cogniam, DEMO, and Pronto. Note: See my tutorials.

23. Working Within A Broader Context
Begin with the End in Mind (see Next Slide):
Open Innovator's Toolkit
President Obama emphasizes a “bottom-up” philosophy that taps citizen expertise to make government smarter and more responsive to private sector demands. This philosophy of “open innovation” has already delivered tangible results in public and regulated sectors of the economy – areas like health IT, learning technologies, and smart grid – that are poised to deliver productivity growth and grow the jobs of the future. We have surfaced new or improved policy tools deployed by our government to achieve them. We’ve posted the Open Innovator’s Toolkit as a roster of 20 leading practices that an “open innovator” should consider when confronting any policy challenge – at any level of government. Our aspiration is to build upon this list, adding new tools and case studies to form an evidence base that will help to scale “open innovation” across the public sector.
Follow 5 Easy Steps:
1. Build an table of contents-like index of complex documents with well-defined web addresses in MindTouch.
2. Build that index in an Excel spreadsheet so it supports faceted search in a Spotfire dashboard.
3. Build a Spotfire knowledgebase with that Excel spreadsheet.
4. Build multiple knowledgebases in a Spotfire dashboard so they can be seen, compared, merged, harmonized, sorted, searched, downloaded, and shared on mobile devices (e.g. iPad).
5. Scale the previous architectural pattern with more content volume and types if necessary.

24. Open Government Initiative: Opening Data For Innovators and Entrepreneurs
Our aspiration is to build upon this list, adding new tools and case studies to form an evidence base that will help to scale “open innovation” across the public sector.

25. Step 1. Build an table of contents-like index of complex documents with well-defined web addresses in MindTouch.

26. 2. Build that index in an Excel spreadsheet so it supports faceted search in a Spotfire dashboard.
Note: This MindTouch table
copies directly to Excel in the
next slide.

27. 2. Build that index in an Excel spreadsheet so it supports faceted search in a Spotfire dashboard.

28. 3. Build a Spotfire knowledgebase with that Excel spreadsheet.
PC Desktop Spotfire

29. 3. Build a Spotfire knowledgebase with that Excel spreadsheet.
Building Steps:
1 - Drag and Drop Spreadsheet Onto Spotfire (see Scatter Plot automatically).
2 - Add New Table to Display Spreadsheet Data (make any adjustments/corrections and Refresh Data).
3 - Adjust Scatter Plot Axes, Color by, Shape by, Size by to produce desired display.
4 - Add New Test Area, Rename Page as Dashboard and Add MindTouch and Excel with Web links to sources of metadata and data.
5 - Insert Action Controls to Reset All Filters and Unmark Marked Rows.
6 - Save Spotfire file to hard drive with desired name and then save to Library.
7 - Test Web Player version and embed in MindTouch.

30. Building Step 7 - Test Web Player version and embed in MindTouch.
Spotfire Web Player

31. Follow 5 Easy Steps
Step 4. Build multiple knowledgebases in a Spotfire dashboard so they can be seen, compared, merged, harmonized, sorted, searched, downloaded, and shared on mobile devices (e.g. iPad).
Another example: How To Simplify Benefits Website For Veterans (AOL Government, MindTouch, Excel, Spotfire, and PowerPoint Tutorial).
Step 5. Scale the previous architectural pattern with more content volume and types if necessary.
My Silver Spotfire Library in the Cloud!

32. New Features in Spotfire 4.0

33. New Features in Spotfire 4.0
Information At A Glance:
Dynamic Values
Conditional Icons
Sparklines
Graphical Summary Table
Look and Feel:
All New Graphical Profile
Pop-Over Filter Panel
Pop-Over Legend
Individual Control Over Axis Label Visibility
More Control Over Legend Contents and Placement
Fixed Size Layout
Mix Filters and Controls on the Page
Nicer Looking Tables
Combine Different Slices of Data on the Same Page
Toolbars and Information

34. New Features in Spotfire 4.0 (Continued)
Navigation and Interaction:
Actions
Page History Navigation
Embed Interactive Controls
Building Dashboards:
Preserve Information When Switching Visualizations
Change All Fonts in One Place
Easier Access to Toggling Visualization Features
More Predefined Categorical Coloring Schemes
Manage Document Color Schemes
Better Defaults When Creating Visualizations
Toggle Auto Column Additions Off
Analysis Previews
Control Over Table Header Font

35. New Features in Spotfire 4.0
Collaboration:
Share with TIBBR
Add TIBBR Discussions to the Analysis
Embed Dashboards in Other Web Pages
Other Enhancements:
Export Footer
Stepped Linecharts
Automation Services 4.0

36. New Features in Spotfire 4.0
Information At A Glance:
Dynamic Values:
What – Dynamically display single values in text areas that responds to filtering and parameter changes.
Why – Look at most important numbers first before diving into more details.
My Note: See Next Slides.
Conditional Icons:
What – Dynamically calculated conditional icons that respond to filtering and parameter changes.
Why – Indicate change, comparisons to target and highlight important events.
Sparklines:
What – Dynamically calculated sparklines that respond to filtering and parameter changes.
Why – Show at a glance and when drilling in whether a metric is trending down, up or varies a lot.
Graphical Summary Table:
What – Dynamic values, conditional icons and sparklines in one compact table broken down by some category.
Why – Visually show everything you need on a single screen.

37. New Features in Spotfire 4
New Features in Spotfire 4.0: Dynamic Values, Conditional Icons, Sparklines, and Graphical Summary Table
PC Desktop Spotfire

38. New Features in Spotfire 4
New Features in Spotfire 4.0: Dynamic Values, Conditional Icons, Sparklines, and Graphical Summary Table
Filter for
Debt Service
PC Desktop Spotfire

39. New Features in Spotfire 4.0
Collaboration:
Share with TIBBR:
What – Right Click on any visualization or page and share the view in tibbr with a link back to the analysis
Why – Easy sharing of insights and findings.
My Note: Tibbr host name has to be set by Administrator.
Add TIBBR Discussions to the Analysis:
What – Integrated tibbr discussions right in the analysis filtered to a particular subject.
Why – Discuss insights and findings with colleagues directly in the analysis. Subscribe and get notified when someone posts a comment on an analysis you are interested in.
My Note: See Next Slide.
Embed Dashboards in Other Web Pages:
What – One click access to HTML fragments that displays a Spotfire page that can be pasted directly into portals and other web pages.
Why – Put a link to the analysis in your corporate blog or wiki. Integrate Spotfire analysis displays into SharePoint WebPart and other portals.
My Note: I was already doing this!

40. New Features in Spotfire 4
New Features in Spotfire 4.0 Collaboration: Add TIBBR Discussions to the Analysis

41. New Features in Spotfire 4.0
Other Enhancements:
Export Footer:
What – Include a footer when exporting or printing pages.
Why – Make it clear where the printout came from or indicate to the reader that the contents is confidential.
My Note: See Next Slide.
Stepped Linecharts:
What – Draw stepped linecharts that only show a change in value at the exact point where the value changed.
Why – Better representation of discrete data that avoids misleading the user by interpolating values in between data points.
Automation Services 4.0:
What – New task added to remap Information Services catalogs and schemas during an automated Library import.
What – Allow for the automation of migrating a Spotfire Information Model from a test to production environment in instances when the test and production instances of the data source are in different database catalogs or schemas.
My Note: See Slides That Follow.

42. New Features in Spotfire 4.0: Other Enhancements
Export Footer
Stepped Linecharts

43. New Features in Spotfire 4.0: Other Enhancements: Stepped Linecharts
PC Desktop Spotfire

44. New Features in Spotfire 4.0 Other Enhancements: Automation Services 4.0
TIBCO Spotfire Automation Services
Selecting a "Set data source credentials" task in the job builder will now allow you to go back and select a different certificate if the first one selected is invalid.

45. New Features in Spotfire 4.0 Other Enhancements: Automation Services 4.0

46. New Features in Spotfire 4.0 Other Enhancements: Automation Services 4.0
My Note: Customize Spotfire Documentation in MindTouch