NIST BIG DATA WG Reference Architecture Subgroup Meeting Agenda Co-chairs: Orit Levin (Microsoft) James Ketner (AT&T) Don Krapohl (Augmented Intelligence) August 22, 2013

Meeting Agenda Meeting summary from Aug 15 (posted as M0153) Deliverable #1: The White Paper (outline posted as M0151) Deliverable #2: The Common RA Review the proposed outline (posted as M0123) Review the proposed diagram (posted as M0126) Review the thoughts from the Roadmap subgroup on synchronization between the subgroups’ deliverables (posted as M0145) 8/15/2013NIST Big Data WG / Ref Arch Subgroup2

Reference Architecture Objectives Addresses a broad range of stakeholders (e.g., data owners, industries, academia, policy makers) Ref Arch subgroup gets the requirements from the Requirements subgroup and then defines the RA applicable to address these requirements Wide scope: Encompasses the whole data life cycle or in the ecosystem Can be applied to different use cases (including various verticals) Represents different system architectures (e.g., an enterprise data warehouse, distributed cloud-based system using multiple service providers) Focus Potentially with initial focus on the Big Data analytics and tools Assists in identifying security and privacy issues Agnostic to any specific technologies 8/15/2013NIST Big Data WG / Ref Arch Subgroup3

Deliverables Deliverable #1: White Paper describing different RA approaches The proposed Outline is in M0150 Call for review Call for volunteers to produce content for different sections Deliverable #2: Common RA Draft The proposed Outline is in M0123 Call for additional volunteers to produce content for different sections A number of participants volunteered already, more are welcome 8/15/2013NIST Big Data WG / Ref Arch Subgroup4

The Recent RA Diagram 8/15/2013NIST Big Data WG / Ref Arch Subgroup5

Deliverable #2: Dependencies on Other Subgroups 2 Big Data Reference Architecture: Use Cases Requirements This section will be prepared by the NIST BDWG Requirements SG and will contain high level requirements relevant to the design of the Reference Architecture. 6 Big Data Reference Architecture: Security and Privacy This section will be prepared by the NIST BDWG Security and Privacy SG and will contain high level security and privacy considerations relevant to the design of the Reference Architecture. 7 Big Data Taxonomy This section will be prepared by the NIST BDWG Def&Tax SG and will contain high level taxonomy relevant to the design of the Reference Architecture. 8/15/2013NIST Big Data WG / Ref Arch Subgroup6

RA Current View The current RA view is technological, meaning that It doesn’t show topology or deployment models It doesn’t show business stakeholders or business relationships By using different sets of taxonomies, the same baseline diagram can be modified to show topologies or business relationships These different views would need to be shown separately (e.g., technologies vs. deployment models vs. business stakeholders) A proposal: Discuss these aspects in the context of the RoadMap scope and direction and sync on the direction and responsibilities 8/15/2013NIST Big Data WG / Ref Arch Subgroup7

Backup Slides 8/15/2013NIST Big Data WG / Ref Arch Subgroup8

Starting Point From the Previous Joint Call Transformation includes Processing functions Analytic functions Visualization functions Data Infrastructure includes Data stores In-memory DBs Analytic DBs Sources Transformation Usage Data Infrastructure Security Management Cloud Computing Network 8/15/2013NIST Big Data WG / Ref Arch Subgroup9

Input for Discussion on the BD RA Next Level of Details The next slides are based on various input documents to NIST BD subgroups They are included here to provide insight on the evolution of the NIST BD RA work 8/15/2013NIST Big Data WG / Ref Arch Subgroup10

Proposal I: Highlighting the Different DB Approaches Security Management Cloud Computing Network Sources Transformation Usage Data Infrastructure Collection Export Curation Pre-analytics Visualization Data Manager Storage (disk, memory, etc.)File SystemSpecialized Abstractions Data Mining Analytics Buffer Manager Map Reduce Relational DB NoSQL DB Aggregation Integration Transfer Search Statistics RT Analytics VOLUME VARIETY VELOCITY Streaming Interactive An. Batch Analytics 8/15/2013NIST Big Data WG / Ref Arch Subgroup11

Proposal II: Based on Submitted Requirements 8/15/2013NIST Big Data WG / Ref Arch Subgroup12