2015 9sight Consulting, All Rights Reserved Copyright © sight Consulting, All Rights Reserved Dr Barry Devlin Founder & Principal 9sight Consulting Big Data 2015 The bold and the beautiful… and the downright dangerous 25, 27 August 2015 Auckland and Wellington, New Zealand

Dr. Barry Devlin 2 Copyright © 2015, 9sight Consulting Founder and Principal 9sight Consulting, Dr. Barry Devlin is a founder of the data warehousing industry, defining its first architecture in A foremost authority on business intelligence (BI), big data and beyond, he is respected worldwide as a visionary and thought-leader in the evolving industry. Barry has authored two ground-breaking books: the classic "Data Warehouse--from Architecture to Implementation" and “Business unIntelligence--Insight and Innovation Beyond Analytics and Big Data” ( in Barry has over 30 years of experience in the IT industry, previously with IBM, as a consultant, manager and distinguished engineer. As founder and principal of 9sight in 2008, Barry provides strategic consulting and thought-leadership to buyers and vendors of BI and Big Data solutions. He is an associate editor of TDWI's Journal of Business Intelligence, and a regular keynote speaker, teacher and writer on all aspects of information creation and use. Barry operates worldwide from Cape Town, South Africa.

– An explosion of bold business opportunities 3 Copyright © 2015, 9sight Consulting Agenda 1.Big data – the bold 2.Big data – the beautiful 3.Big data – the downright dangerous 4.Three thoughts to take away

Big data boldly… “doth bestride the narrow world Like a Colossus, and we petty men Walk under his huge legs and peep about To find ourselves dishonourable graves” – Shakespeare, Julius Caesar  Answers marketing’s every desire  Perfects customer service  Eliminates cost from the supply chain  Streamlines manufacturing / delivery  Solves (and prevents?) crime  Footnote: modern engineering declares that the Colossus could never have spanned the harbour entrance 4 Copyright © 2015, 9sight Consulting The Colossus of Rhodes. 16th-century engraving by Martin Heemskerck.

Transport: peer-to-peer “ride sharing” goes data driven crazy  Company now valued at $50B – Uber Series E venture financing up to $2.8B (Feb 2015) – On top of $4B raised previously  Data driven core business operation – Data and comms key to business model  Deep analytics drive & optimize business – Cars available where needed – Real-time demand-based pricing  Highly disruptive to current transport model – Convenience, availability, cost… all data driven 5 Copyright © 2015, 9sight Consulting

6

Agriculture: sowing the data driven seeds  Monsanto acquires The Climate Corporation – October, 2013 for $930million  Monsanto has: – Weather measurements from 2.5 million locations – Forecasts from major climate models – 150 billion soil observations to generate 10 trillion weather simulation data points – Added to Monsanto’s own extensive databases  Big data and Internet of Things offers: – Counteract effects of climate change – Improved yields (proven 5% over 2 years) and cost savings for farmers – Lock in of farmers to Monsanto products – “I could easily see us in the next 5-10 years being an information technology company.” CTO, Robb Fraley 7 Copyright © 2015, 9sight Consulting

Finance (and other trusted third party business): disruption through data sharing and collaboration  TransferWise raises $58m in Series C funding round (Jan 2015) – London-based firm, possible $1B valuation  Pairs up currency buyers with sellers – Avoids the need to actually convert money – Saves on expensive foreign exchange fees – Real-time data driven business model  Potential to disrupt banking industry – Needs even money flow between countries – See also: CurrencyFair and others  BUT… Real disruption potential may come from “the Blockchain” – Basis of Bitcoin and other crypto-currencies… – Real impact potential is displacement of trusted third parties – Banking, trading, validating documents & licences, land rights, ownership of all kinds of intellectual property, etc. – See: 8 Copyright © 2015, 9sight Consulting

9 Agenda 1.Big data – the bold – An explosion of bold business opportunities 2.Big data – the beautiful – Big data demands a new and beautiful IT architecture 3.Big data – the downright dangerous 4.Three thoughts to take away

10 The data architecture since the mid-’80s  Two layers within the Data Warehouse… 1.Enterprise data warehouse – Reconciled data 2.Data marts – What the users need  … fed from and separate to operational systems – Data to run the business – Created by the processes of the business  All data created within the enterprise (or within partner ecosystem) Copyright © 2015, 9sight Consulting Data marts Enterprise data warehouse Metadata Data warehouse Operational systems “An architecture for a business and information system”, B. A. Devlin, P. T. Murphy, IBM Systems Journal, (1988)

“Big data” Big data demands a new architecture: the tri-domain information model is the basis  Process-mediated data – “Traditional” operational & informational data – Via data entry & cleansing processes  Machine-generated data – Output of machines and sensors – The Internet of Things  Human-sourced information – Subjectively interpreted record of personal experiences – From Tweets to Videos 11 Copyright © 2015, 9sight Consulting Human-sourced information Machine- generated data Process-mediated data Structure/Context Timeliness/ Consistency HistoricalReconciledStableLiveIn-flight Raw Atomic Derived Compound Textual Multiplex [“Data” is well-structured and/or modeled and “information” is more loosely structured.]

information pillars replace data layers  One architecture for all types of information – Mix/match technology as needed – Relational, NoSQL, Hadoop, etc.  Integration of sources and stores – Instantiation gathers inputs – Assimilation integrates stored info.  Data flows as fast as needed and reconciled when necessary – No unnecessary storage or transformations  Distinct data management / governance approaches as required 12 Copyright © 2015, 9sight Consulting Transactions Human- sourced (information) Machine- generated (data) Process- mediated (data) Context-setting (information) Assimilation Transactional (data) Events Measures Messages Instantiation

the modern meaning model (m 3 ) replaces metadata with context setting information (CSI)  Ackoff’s DIKW no longer viable  We need a new vision – Information precedes data – Data is simply information optimized for computers – “Metadata” is actually information that provides context – All information provides some context – People process information… to make their world meaningful 13 Copyright © 2015, 9sight Consulting Locus Structure Physical Loose Mental Strict Interpersonal Hard Information Soft Information Explicit Knowledge Tacit Knowledge Meaning The stories we tell ourselves Objective / universal Subjective / unique Sense- making Mentoring Understanding Insight Data Content Articulation Practice Documenting Learning Videoing Observing Modeling Interpreting From Physical World From Human World CSICSI

From BI to Business unIntelligence  People: Rational thought and far beyond – People make all decisions!  Process: Logic – predefined and emergent – Decision making is a process  Information: Data, knowledge and meaning – Data/information is only the foundation  People process information  Not business intelligence… Business unIntelligence  Amazon:  Or : 25% discount with code “BIInsights25” 14 Copyright © 2015, 9sight Consulting

15 Copyright © 2015, 9sight Consulting Agenda 1.Big data – the bold – An explosion of bold business opportunities 2.Big data – the beautiful – Big data demands a new and beautiful IT architecture 3.Big data – the downright dangerous – Big data dangers – governance, privacy, economics, society 4.Three thoughts to take away

Governance: Data can be difficult to understand and use with confidence and reliability  Context and the search for meaning – Modelling business needs for data during application development – Enterprise data warehouse / data marts – Information degraded with false or simplistic assumptions in spreadsheets – A long-standing problem, addressed in part by data warehouse automation  The long path from data to action – Data  information  knowledge  meaning  decisions  action – Rationality and intuition both needed – The new issue: do you want to be data driven or information informed? 16 Copyright © 2015, 9sight Consulting

Big data is even harder to decipher New management / governance approaches needed  External data is innately dirty – Reflects providers’ viewpoints – Often poorly documented – Continuously changing meanings – Missing, corrupted values and noise  Data no longer necessarily = facts – On whose authority? – Selective collection – Biased interpretation  Correlation ≠ causation – How many business people really understand statistics? 17 Copyright © 2015, 9sight Consulting

Privacy: Analytics of human-sourced information can drive serious breaches  Data brokers now gathering thousands of measurable attributes about consumers (people) to create marketing lists, e.g.: – Police officers at home addresses – Rape sufferers – Domestic violence shelters – Genetic disease, dementia and HIV/AIDs sufferers – People with addictive behavior  Scoring used to discriminate (target market) – (Pam Dixon, Executive Director, World Privacy Forum, before US Senate Committee, Dec 2013) – 66e4-42ad-94c5-fcd4f e4-42ad-94c5-fcd4f Copyright © 2015, 9sight Consulting

Marketing data gathered in your home allows…? 19 Copyright © 2015, 9sight Consulting -that-watches-you-back-verizon-applies-ambient-action- detecting-device-patent.shtml

Economics: Data, algorithms and automation drive down cost of production… and displace jobs  Travis Kalanick, CEO and founder of Uber, “would replace human Uber drivers with a fleet of self-driving cars in a second”  Autonomous trucks could displace 4-8 million jobs in the US within 10+ years – May 6, 2015, first self-driving truck on American road in Nevada – See  The ever reducing need for more advanced skills – Researchers construct statistical model able to predict the outcome of 70% of U.S. Supreme Court cases – Hong Kong-based venture capital firm Deep Knowledge Ventures adds AI program with equal voting rights to its board of directors – See 20 Copyright © 2015, 9sight Consulting

Big data is driving a revolution in robotics  Manufacturing robots come out of the cage – Working safely along side humans on production lines (Baxter) – Trained through direct human interaction  Robots are moving into the front office and home – June 2014, SoftBank Mobile (Japan) customers greeted by Pepper in selected stores, in more than 2,600 stores by year end 2015 – “Pepper is the first humanoid robot designed to live with humans”; converses with people, recognizes and reacts to emotions  Advances in machine learning stemmed from widespread availability of big data 21 Copyright © 2015, 9sight Consulting

Robotics is driving potentially extensive economic and social disruption  Care of the elderly – US National Science Foundation $1.2 million grant to teach robots to assist the elderly in picking an outfit and helping them dress – robots-to-dress-old-people / robots-to-dress-old-people / 22 Copyright © 2015, 9sight Consulting  Sex with a seemingly human touch – According to developer: “robotic, AI-driven heads—dubbed Realbotix—will be commercially available in two years, priced at around $10,000” – sex-robot-realdoll.html sex-robot-realdoll.html  Sensors, IoT data and high speed processing are key

In an age of exponentially improving technology, the classical economic models break down  In the old/current model – Work produces ever cheaper goods – Income, to purchase goods, depends on work – Leisure or meaning is earned through work  In the forthcoming model – Capital produces goods (via technology) – Work is increasingly unavailable, poorly paid – Lack of income to purchase goods – Leisure and meaning become internalized – See  The capitalist model contains the seeds of its own demise 23 Copyright © 2015, 9sight Consulting

24 Copyright © 2015, 9sight Consulting Agenda 1.Big data – the bold – An explosion of bold business opportunities 2.Big data – the beautiful – Big data demands a new and beautiful IT architecture 3.Big data – the downright dangerous – Big data dangers – governance, privacy, economics, society 4.Three thoughts to take away

Define and implement a big data strategy with governance at its core  The process of building a “big data ecosystem” is very different to that of building a data warehouse – Data warehouse: business needs  data structure and content – Big data: data structure and content  business opportunities  Context setting information (CSI) is central – Distilling context from incoming data – Managing and reuse through collaboration – Step beyond the rational mind – Information informed  Process automation is mandatory – Volumes and variety beyond manual limits 25 Copyright © 2015, 9sight Consulting

Take steps to ensure information privacy: Business 1.Does your business model depend on “free” services supported by targeted advertising? At what cost to users’ privacy? 2.Decide the ethics of data usage: – Having it doesn’t mean you should use it – What are the potential negative implications of having particular data about people? – Don’t collect it unless you understand them 3.Be fully transparent about intended use of data – Avoid data from dubious data broker sources 26 Copyright © 2015, 9sight Consulting

Take steps to ensure data privacy: Technology 1.Foundation is physical security, with policies and compliance 2.Combining data from multiple sources can expose added context and meaning – De-anonymisation can occur 3.Consider legal/financial implications – E.g. new EU privacy laws allow fines up to 5% of global revenue or €100m 4.Manage and use data within widely differing local privacy laws – E.g. partitioned EDW; anonymised / aggregated data crosses borders 5.Limit data usage to personal privacy preferences 27 Copyright © 2015, 9sight Consulting

Question broader economics & societal perspectives in strategic business / IT decisions  Is the “ever-increasing profit” motive still valid? – Lower costs or higher sales? – (N.B. In real life, everything is cyclical)  Does your cost-saving project: – Reduce employment / income? – Lower the job skills needed? – How to mitigate reduced purchasing power?  Does your marketing/sales project: – Just create a need? – Drive “hyper-competition”? – Beyond the shareholders, who really benefits?  Isn’t business essentially a social enterprise? – From profit to “public good” 28 Copyright © 2015, 9sight Consulting

Two novel economic/social solutions to explore 29 Copyright © 2015, 9sight Consulting Universal Basic Income An income unconditionally granted to all individuals, without means test or work requirement, driven by technological displacement and enabled by redistribution of wealth generated through capital and technology Universal Basic Income An income unconditionally granted to all individuals, without means test or work requirement, driven by technological displacement and enabled by redistribution of wealth generated through capital and technology Collaborative Commons Co-operative global sharing and inter-connection based on the Internet of Things, distributed energy production and 3D printing, all driving towards zero marginal cost production (Jeremy Rifkin), potentially leading to sustainable production-consumption of the finite resources of the planet 62b0160a7048 Collaborative Commons Co-operative global sharing and inter-connection based on the Internet of Things, distributed energy production and 3D printing, all driving towards zero marginal cost production (Jeremy Rifkin), potentially leading to sustainable production-consumption of the finite resources of the planet 62b0160a7048

Conclusions 1.Big data impacts more than marketing – Explosion of new business opportunities – Prepare for heavy disruption in all industries / functions 30 Copyright © 2015, 9sight Consulting 2.Big data disrupts more than business goals – Data governance and management are central – Protect privacy for today’s and future generations 3.Big data affects the economy and society – Analytics and automation are disrupting employment – Examine new economic and societal models

2015 9sight Consulting, All Rights Reserved Copyright © sight Consulting, All Rights Reserved Dr Barry Devlin Founder & Principal 9sight Consulting Thank You