We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byAlvin Mader
Modified about 1 year ago
1 1 Apache Hadoop and the Emergence of the Enterprise Data Hub Eli Collins, Chief Technologist ©2014 Cloudera, Inc. All rights reserved.
2 2 The Enterprise Data Warehouse ©2014 Cloudera, Inc. All rights reserved. Flat Files Operational Store Data Sources Staging Reporting Analysis Mining Operational Store Metadata Summary Facts & Dimensions EDW Archive Data marts
3 3 The Enterprise Data Hub ©2014 Cloudera, Inc. All rights reserved. images logs binary DB dumps 1.Inexpensive storage 2.Flexible storage 3.Co-located compute 4.Multiple compute engines MR, Pig/Hive, SQL, Spark, SAS, R, Search, Graph..
4 ©2014 Cloudera, Inc. All rights reserved.4 So it’s Like a Data Warehouse?
5 5©2014 Cloudera, Inc. All rights reserved. An Analogy
6 6©2014 Cloudera, Inc. All rights reserved. What changed? The need? Convenience? Cost?
7 Take and share good photos
8 Data Warehouse vs. Data Hub ©2014 Cloudera, Inc. All Rights Reserved. Enterprise Data Warehouse Enterprise Data Hub
©2014 Cloudera, Inc. All Rights Reserved. 9 An Operating System APP SCHEDULER FILE SYSTEM MGT SERVICES APP LIB APP 3rd PARTY APP
©2014 Cloudera, Inc. All Rights Reserved. 10 An Enterprise Data Hub BATCH PROCESSING ANALYTIC SQL SEARCH ENGINE MACHINE LEARNING STREAM PROCESSING 3 RD PARTY APPS WORKLOAD MANAGEMENT STORAGE FOR ANY TYPE OF DATA UNIFIED, ELASTIC, RESILIENT, SECURE DATA MANAGEMENT SYSTEM MANAGEMENT FilesystemOnline NoSQL
11 Data Warehousing with an EDH ©2014 Cloudera, Inc. All rights reserved. Flat Files Operational Store Data Sources EDH Reporting Analysis Mining Operational Store EDW 1. Stage, transform, archive 3. Exploratory, Discovery, Search, ML.. 2. Reporting, Mining, Analysis
12 ©2014 Cloudera, Inc. All rights reserved.12
13 ©2014 Cloudera, Inc. All rights reserved.
©2014 Cloudera, Inc. All Rights Reserved. 14 Data Warehousing in Cloudera’s EDH
15 ©2014 Cloudera, Inc. All rights reserved.15
16 ©2014 Cloudera, Inc. All rights reserved.
1 Apache Spark and Its Role in the Enterprise Data Hub Mike Olson, Chief Strategy Officer,
An Information Architecture for Hadoop Mark Samson – Systems Engineer, Cloudera.
PANEL SENIOR BIG DATA ARCHITECT BD-COE
Data Integration - The ETL Process Module 4: BIC#4 – Data Integration Capability Populating Data Warehouse (Data Mart) 1.
Evaluation of distributed open source solutions in CERN database use cases HEPiX, spring 2015 Kacper Surdy IT-DB-DBF M. Grzybek, D. L. Garcia, Z. Baranowski,
Securing Native Big Data Deployments Steven C. Markey, MSIS, PMP, CISSP, CIPP/US, CISM, CISA, STS-EV, CCSK, Cloud + Principal, nControl, LLC Adjunct Professor.
A Suite of Products that allow you to Predict Outcomes, Prescribe Actions and Automate Decisions.
نمايندگي استان يزد. نمايندگي استان يزد طراحی کسب و کار الکترونیکی ارائه کننده : محسن افسر قره باغ.
What is it and why it matters? Hadoop. What Is Hadoop? Hadoop is an open-source software framework for storing data and running applications on clusters.
Observation Pattern Theory Hypothesis What will happen? How can we make it happen? Predictive Analytics Prescriptive Analytics What happened? Why.
© 2009 VMware Inc. All rights reserved Big Data’s Virtualization Journey Andrew Yu Sr. Director, Big Data R&D VMware.
Architecting for the Internet of Things & Big Data Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data September 29, 2014.
LIMPOPO DEPARTMENT OF ECONOMIC DEVELOPMENT, ENVIRONMENT AND TOURISM The heartland of southern Africa – development is about people! 2015 ICT YOUTH CONFERENCE.
1 © Cloudera, Inc. All rights reserved. Alexander Bibighaus| Director of Engineering, Cloudera, Inc. The Future of Data Management with Hadoop and the.
1 Reviewing Data Warehouse Basics. Lessons 1.Reviewing Data Warehouse Basics 2.Defining the Business and Logical Models 3.Creating the Dimensional Model.
SQL on Hadoop. Todays agenda Introduction Hive – the first SQL approach Data ingestion and data formats Impala – MPP SQL.
Cloudera Image for hands-on Installation instruction – https://cern.ch/zbaranow/CVM.txt 2.
1 © Cloudera, Inc. All rights reserved. Engines, Algorithms, and Data Models Josh Wills | Senior Director of Data Science From Dimensional Modeling to.
C Copyright © 2007, Oracle. All rights reserved. Introduction to Data Warehousing Fundamentals.
Click to add text © 2012 IBM Corporation 1 Streams – DataStage Integration InfoSphere Streams Version 3.0 Mike Koranda Release Architect.
Apache Spark and the future of big data applications Eric Baldeschwieler.
WebSphere -DB2 Integration Web Browser Web Server (Apache) WebSphere –JSP/Servlet/EJB DB2 JDBC, SQL HTTP.
DataWarehousing and DataMining Prof. Sin-Min Lee.
Powered by Microsoft Azure, PointMatter Is a Flexible Solution to Move and Share Data between Business Groups and IT MICROSOFT AZURE ISV PROFILE: LOGICMATTER.
Data Warehousing: Defined and Its Applications Pete Johnson April 2002.
BigData Tools Seyyed mohammad Razavi. Outline Introduction Hbase Cassandra Spark Acumulo Blur MongoDB Hive Giraph Pig.
IST722 Data Warehousing Components of the Data Warehouse Michael A. Fudge, Jr.
© 2012 IBM Corporation Converting Big Data into Big Knowledge.
Hadoop tutorials. Todays agenda Hadoop Introduction and Architecture Hadoop Distributed File System MapReduce Spark 2.
MD240 - MIS Oct. 4, 2005 Databases & the Data Asset Harrah’s & Allstate Cases.
1 Cloud-Native Data Warehousing Bob Muglia. 2 Scenarios with affinity for cloud Gartner 2016 Predictions: By 2018, six billion connected things will be.
CS 157B: Database Management Systems II April 10 Class Meeting Department of Computer Science San Jose State University Spring 2013 Instructor: Ron Mak.
3 Hadoop? Cloud data warehousing? Machine learning? NoSQL?
Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Technology Education Copyright © 2006 by The McGraw-Hill Companies,
IoT Scenario - Connected Cars / Devices Cloud gateways Queue Service Get Data Get Reference Data Business Logic Store Raw Data Store Reporting Data.
“Innovation through Prediction” - Hybrid Cloud Big Data Platform John Andrew Oracle Enterprise Architect Learn. Predict. Influence.
1 Cloudera Impala and improvements in HDFS for real-time queries Todd Lipcon Software Engineer, Cloudera.
Copyright © 2005, SAS Institute Inc. All rights reserved. Making the Transition from MDDB-based OLAP Applications to a SAS ® 9 OLAP Solution Ivy Parker.
©2015 DesignMind. All Rights Reserved.. 2 About DesignMind.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Our experience with NoSQL and MapReduce technologies Fabio Souto.
Database Systems – Data Warehousing INTRODUCTION There exists an information gap amongst organizations. Organizations have plenty of data, but little information.
Slide 1 Data Warehousing in CIM 2000 YourNameHere Data Warehousing in Computer Integrated Manufacturing Steve Daino IEM 5303.
The Business Intelligence Side of Blue Mountain RAM Bill Lucas, IT Systems Architect and Senior Software Engineer.
Motivation Customer Trends Reporting Insights, predictions, actions Static data Dynamic intelligence Operational efficiency Competitive advantage.
Slide 1 © 2016, Lera Technologies. All Rights Reserved. SAP BO vs SPLUNK vs OBIEE By Lera Technologies.
Unit 5 Organizing Data and Information. Learning Outcomes Understand records in a database Understand components of a DBMS Identify database models and.
Faculty of Computer Science © 2006 CMPUT 605February 11, 2008 A Data Warehouse Architecture for Clinical Data Warehousing Tony R. Sahama and Peter R. Croll.
Enabling data management in a big data world Craig Soules Garth Goodson Tanya Shastri.
Senior Project Manager & Architect Love Your Data.
1 ACCTG 6910 Building Enterprise & Business Intelligence Systems (e.bis) The Data Warehouse Lifecycle Olivia R. Liu Sheng, Ph.D. Emma Eccles Jones Presidential.
© 2017 SlidePlayer.com Inc. All rights reserved.