1. Big Data and Official Statistics: Philippine Context Erniel B. Barrios

2. Outline Concepts and Definitions Coverage of Big Data Big Data and Official Statistics: Preliminary Framework Current Practices (Some Models) Possible Big Data in the Philippines Next Steps

3. Frequency of Documents Containing Big Data in ProQuest Research Library

4. Basis of Definitions Stakeholders may define Big Data differently Data storage and data analysis Intertwined technical and socio-technical issues Multiple, ambiguous and often contradictory definitions “Big” => significance, complexity, challenge Five V’s Volume (size) Velocity (rate of production) Variety (format, representations) IBM:V is Veracity (trust and uncertainty) SAS: Variability (complexity). Intel: generating a median of 300 TB

5. Basis of Definitions Size: volume of the dataset Complexity: structure, behavior and permutations of the dataset Technologies: tools and techniques which are used to process a sizable or complex dataset

6. Definitions Appropriate description, integration, and sustainability of very large datasets generated by high throughput experiments Large collection of small disparate, unstructured datasets, (taken together, can be analyzed to find unusual trends). Emergence of digital enterprise, ability for an organization to take full advantage of its digital assets, collectively large amount of data Oracle: Inclusion of additional data sources to augment current operations Microsoft: process of applying serious computing power (machine learning, AI) to seriously massive and often highly complex sets of information.

7. Definitions Big Data describes the storage and analysis of large and/or complex data sets using a series of techniques. High-volume, high-velocity, and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making. Describes large volumes of high velocity, complex and variable data that require advanced techniques and technologies to enable the capture, storage, distribution, management, and analysis of information. UNECE: Big Data-data that is difficult to collect, store of process within the conventional systems of statistical organizations. Either their volume, velocity, structure or variety

8. Online Survey of 154 Global Executives (April 2012)

9. Definition Big Data Not only in size (though volume can be part of it) Varying Sources, Several Variables (Indicators) Differing data collection methods (compilation) Frequency (possibly irregular) Issues Quality Architecture Security/Confidentiality Integrity Standardization Data extraction Data Mining

10. New Data Sources Consumer Usage Database Blogs Social Media Sensor Networks Image Data May vary in Size Structure format

11. Coverage of Big Data Basic research data Electronical health records Consumer Usage Database Proposals submitted Administrative data Censuses and Surveys

12. Types of Big Data (Classification) Social Network: Human-sourced information Social networks, Blogs, Personal Documents, Pictures, Videos, Internet Searches, Mobile Data, User-generated maps, E-mail Traditional Business Systems: Process-mediated data Public agencies (including medical records), produced by business (commercial transactions, banking/stocks records, E-commerce, Credit Cards) Internet: machine-generated Fixed sensors: home automation, weather/pollution sensor, traffic, scientific, security/surveillance Mobile sensors: mobile phone, cars, satellite images Computer systems: logs, web logs

13. Current Practices

14. Big Data and Official Statistics Location data for mobile phones used for instantaneous daytime population and tourism statistics proxy indicators for demand Social media messages Process into early indicators of consumer confidence Price information on the web, from loyalty cards Inflation level Google search Prevalence rate of Influenza Tweets Stock market prices

15. Big Data and Official Statistics: Preliminary Framework Official Statistics, SDG Other Big Data Surveys Census Administrative Reports Businesses Farms Households Individuals Collaboration (PPP) Methodology Human Resources

16. Possible Big Data in the Philippines From PSA/NGA, LGU Censuses Survey Administrative Reports Regulation, Licensing and Compliance Monitoring (e.g., MFO, Budgeting, Intervention (4Ps, RSBSA, etc.) Registers (BIR, COMELEC, UMID, GSIS/SSS, Philhealth, Pag-Ibig, etc.) Private/Commercial Telco Credit cards Loyalty Cards POS Images Sensor Social Media, Google, etc.

17. Next Steps What is available? Big data sources Data that can shared, frequency, timeliness Data security, confidentiality issues Big Data and Official Statistics: Is it feasible?, Is it worthy? What is needed for collaboration, data-sharing?

18. Thank you.