Presentation on theme: "PLATFORM FOR BIG DATA, NOSQL AND RELATIONAL DATA. WHAT MAKES SENSE FOR ME? (+AZURE)"— Presentation transcript:
PLATFORM FOR BIG DATA, NOSQL AND RELATIONAL DATA. WHAT MAKES SENSE FOR ME? (+AZURE)
WHAT IS BIG DATA?
Batch ProcessingInteractive AnalysisStream Processing Query runtimeMinutes to hoursMilliseconds to minutesNever-ending Data volumeTBs to PBsGBs to PBsContinuous stream Programming modelMapReduceQueriesDAG UsersDevelopersAnalysts and developersDevelopers Originating projectGoogle MapReduceGoogle DremelTwitter Storm Open source projectHadoop / SparkDrill / Shark / Impala Hbase Storm / Apache S4 /Kafka
How do I optimize my fleet based on weather and traffic patterns? SOCIAL & WEB ANALYTICS LIVE DATA FEEDS ADVANCED ANALYTICS Whats the social sentiment for my brand or products How do I better predict future outcomes? A NEW SET OF QUESTIONS
COMMON BIG DATA CUSTOMER SCENARIOS GAIN COMPETITIVE ADVANTAGE BY MOVING FIRST AND FAST IN YOUR INDUSTRY Web app optimization Smart meter monitoring Equipment monitoring Advertising analysis Life sciences research Fraud detection Healthcare outcomes Weather forecasting Natural resource exploration Social network analysis Churn analysis Traffic flow optimization IT infrastructure optimization Legal discovery
persistent | distributed In Memory Efficient at Random Reads/Writes Distributed, large scale data store Utilizes Hadoop for persistence Both HBase and Hadoop are distributed
Orders_federation Federation Key The key used for data distribution int, bigint, guid, varbinary Atomic Unit Represent a single instance of a federation key. All rows in all federated tables with the same federation key value.
Federated Table Contains only atomic units for members key range Reference Table Non-Federated table
SalesDB Orders_federation Orders_Fed [5000, 10000) ALTER FEDERATION Orders_Fed SPLIT AT (tenant_id=7500) [5000, 7500) & [7500, 10000) Dynamic Partitioning SPLIT members to spread workloads over to more nodes DROP members to shrink back to fewer nodes
SalesDB Orders_federation Orders_Fed [5000, 7500) & [7500, 10000) USE FEDERATION Orders_Fed (tenant_id=7509) Built-in Data-Dependent Routing (DDR) Ensure apps can discover where the data is just-in-time No Shard Map caching Guaranteed member routing
EntityTableAccount contoso Name =… = … Name =… Add= customers Photo ID =… Date =… photos Photo ID =… Date =…
Table Details Insert Update Merge – Partial update Replace – Update entire entity Upsert Delete Query Entity Group Transactions Multiple CUD Operations in a single atomic transaction Create, Query, Delete Tables can have metadata Not an RDBMS! Table Entities
FIRSTLASTBIRTHDATE WadeWegner2/2/1981 NathanTotten3/15/1965 NickHarrisMay 1, 1976 FAV SPORT Canoeing
MANAGE ANY DATA, ANY SIZE ANYWHERE SQL Server Database & Parallel Data Warehouse Hadoop on Windows Hadoop on Azure StreamInsight Hadoop Connectors & ETL
Global Physical Infrastructure servers / network / datacenters computestoragenetworking virtual machinesweb sitescloud servicesSQL databasenoSQL databaseblob storageconnectvirtual networktraffic manager Frameworks Services Fabric Infrastructure N Central US, S Central US, N Europe, W Europe, E Asia, SE Asia + 24 Edge CDN Locations Automated Managed Resources Elastic Usage Based