Data Analytics 1 - THE HISTORY AND CONCEPTS OF DATA ANALYTICS

“The purpose of computing is insight, not numbers” Richard Hamming 1961

GIDEON & THE 300 SOLDIERS?

KEY EVENTS IN THE HISTORY OF DATA ANALYTICS 1890-Herman Hollerith invents the Hollerith Tabulating Machine which reduced crunching of census data from 10years to 3months!

KEY EVENTS IN THE HISTORY OF DATA ANALYTICS 1962 John Tukey writes a paper title “The Future Of Data Analysis”, where he brought into question the relationship between statistics and analysis. 1970-Edgar F. Codd presents his framework for relational databases. 1989-Howard Dresner at Gartner proposes the term “Business Intelligence.” 1990s-Data Mining is born following the success of the concept of data warehouses introduced by William H. Inman. 1991-Tim Bernes Lee sets out the specifications for a worldwide, interconnected web of data accessible to anyone across the world, now the internet. 2004-A whitepaper on MapReduce from Google inspires open source software projects like Apache Hadoop and Apache Cassandra to deal with huge volumes of data through distributed computing. 2008-Jeff Hammerbacher and DJ Patil, then at Facebook and LinkedIn coin the term “data scientist” to describe their work and it then becomes a buzzword. 2013-IBM shows statistics that 90% of the world’s data was created in the preceding 2 years!

DEFINITION OF KEY TERMS Business intelligence (BI) is an umbrella term that includes the applications, infrastructure and tools, and best practices that enable access to and analysis of information to improve and optimize decisions and performance.(Gartner) The purpose of Business Intelligence is to support better business decision making. Essentially, Business Intelligence systems are data- driven Decision Support Systems (DSS). Business Intelligence is sometimes used interchangeably with briefing books, report and query tools and executive information systems.

DEFINITION OF KEY TERMS Data Mining: is the computational process to discover patterns in large datasets stored in relational databases and data warehouses. It is an intersection of artificial intelligence, machine learning, statistics and database systems.

DATA ANALYTICS

DEFINITION OF DATA ANALYTICS Data Analytics also known as Predictive Analytics, is all about automating insights into a dataset through usage of queries and data aggregation procedures. It can represent various dependencies between input variables and discover hidden patterns in the dataset under analysis. Data Analytics is the science of examining raw data with the purpose of finding and drawing conclusions about the information in the data using methods from statistics and machine learning. Data Analytics goes beyond the concept of data mining by analysing semi-structured and unstructured data from different sources and in different formats e.g. text mining.

Advantages of Data Analytics Smart Decision Making Clearer insights into the inner workings of the business Improved Efficiency in business processes Better responds to market changes Better customer relationship management Better risk management Products and services tailored to customer need

BIG DATA

DEFINITION OF BIG DATA Big Data implies huge data volumes that cannot be processed effectively with traditional applications. Big Data processing begins with raw data that is not aggregated and it is often impossible to store such data in the memory of a single computer. In other words big data analytics is an extension of traditional data analytics which was mainly done on structured data e.g. stored in relational databases, to a more complex analysis on structured, semi-structured and unstructured data. Big data is characterized by the famous 3 ‘Vs’-{volume, velocity and variety} Big data deals with huge volumes of data, whose rate of growing is very fast and is from diverse sources and in different formats e.g. texts, tweets, web click streams, satellite images, sensors, web log data etc. Big data analytics is fuelled by improvements in bandwidth and connectivity, advancement in processing power and the falling price of hard disk.

SOURCES OF BIG DATA

THE SKILL SET FOR DATA ANALYTICS

CHALLENGES OF DATA ANALYTICS Strategic Alignment Organizational Culture Technology Skills Data