Software Systems for Survey and Census Statistics Indonesia Yudi Agusta Statistics Indonesia (Chief of IT Division Regional Statistics Office of Bali Province) Joint Meeting on Management of Statistical Information Systems (MSIS) – Daejeon, 26-28 April 2010

Outline BPS – Statistics Indonesia The Needed Software Systems The Case of The 2010 Indonesian Population Census (Sensus Penduduk 2010 – SP2010) Problem Background and Analysis Available and Applicable Technologies Expected Implementation Design of Software Systems Some Hurdles Further Advancement

BPS – Statistics Indonesia Vision: BPS is the Agent of Trustworthy Statistical Data for All Mission: To strengthen the constitutional and operational base of the institution for carrying out effective and efficient statistical activities. To deliver competent and professional statistic human resources, supported by the use of cutting edge information technology for the improvement of Indonesian statistics. To improve the implementation of the universal standard for classification, concept and definition, estimation, and statistics ethical code in every statistical activity. To improve the quality of statistical information services for all parties. To improve coordination, integration, and synchronization of statistical activities conducted by government and private sectors, within the frame of effective and efficient National Statistics Systems. Main Activities Output Setup of Census/Survey Methodology The Setup of Census Block, Sampling, Questionnaire Design, and Variable Definition Data Collection Census, Survey, Compilation of Administrative Records, Special Studies Data Processing Data Dissemination and Analysis Economic Growth Inflation Employment Poverty Farmer’s Terms of Trade Rice & Crops Production Export and Import

The Needed Software Systems Master File & Map Update & Browsing Systems Classification Standard Browsing Systems Concept and Definition Update and Browsing Systems Metadata Update and Browsing Systems Digital Map Update Systems Sampling, Weighting & Estimation Systems CATI Systems CAPI Systems OCR/OMR Data Capturing Systems E-Survey Systems Data Entry Systems Data Imputation Systems Automatic Editing Systems Data Exploring Systems Data Validation Systems Data Center Systems Data Mapping Systems Data Warehouse Systems Data Dissemina tion Systems Admin Communica tion Systems Managerial Scheduling Systems Reporting Systems Documents Handling Systems

The Needed Software Systems Master File & Map Update & Browsing Systems Classification Standard Browsing Systems Concept and Definition Update and Browsing Systems Metadata Update and Browsing Systems Digital Map Update Systems Sampling, Weighting & Estimation Systems CATI Systems CAPI Systems OCR/OMR Data Capturing Systems E-Survey Systems Data Entry Systems Data Imputation Systems Automatic Editing Systems Data Exploring Systems Data Validation Systems Data Center Systems Data Mapping Systems Data Warehouse Systems Data Dissemina tion Systems Admin Communica tion Systems Managerial Scheduling Systems Reporting Systems Documents Handling Systems

The Needed Software Systems Master File & Map Update & Browsing Systems Classification Standard Browsing Systems Concept and Definition Update and Browsing Systems Metadata Update and Browsing Systems Digital Map Update Systems Sampling, Weighting & Estimation Systems CATI Systems CAPI Systems OCR/OMR Data Capturing Systems E-Survey Systems Data Entry Systems 4 5 3 Data Imputation Systems Automatic Editing Systems Data Exploring Systems Data Validation Systems Data Center Systems Data Mapping Systems Data Warehouse Systems Data Dissemina tion Systems Admin Communica tion Systems Managerial Scheduling Systems Reporting Systems Documents Handling Systems 2 1

The Case of The 2010 Indonesian Population Census (Sensus Penduduk 2010 – SP2010)

Problem Background Extensive Changes/Consolidation In Terms of Concept and Definition, Scope, and Uses of Variables During The Conduct of Population Census Occurring Errors in Data Collection - the Enumerators and Respondents Point of Views Very Large Scope of Processes Needed to be Supervised The Need of Faster, Accurate, and Reliable Data Dissemination Process Problem Analysis – In Relation To The Possible Support of IT Communication Lag Question and Answer Suggestion and Confirmation Updated News & Files, Reporting Miscommunication Unclear Information Misinterpretation False in Filing Clarity of Process & Instruction Systems No Clear Procedures No Documentations Document Handling Problem Bad Documentation Lack of Accuracy Imprecise Implementation of Validation Rules Lack of Post Processing Analysis Uses of Census Data Lack of Preparation in Utilizing Census Data Lack of Automated Tools in Exploring Census Data

Available & Applicable Technologies Extraction Transformation Loading Online Analytical Processing Rule Base System Optical Mark Recognition Optical Character Recognition Intelligent Systems Data Mining Expected Implementation Implementation of Online Interaction System With Various Functions Online Enumeration Q&A Online Data Processing Q&A Online Validation Rule Suggestion and Confirmation/Approval Online Reporting System Implementation of Document Handling System Record Keeping of Census/Survey Documents Implementation of Post Processing Utility System Important Variables Analysis Double Counting Analysis Record Outlier Analysis Implementation of Data Exploring System Comparison with Previous Data Comparison Between Area Online Analytical Processing (OLAP) Web-Based Online System Information Systems

Design of Software Systems Online Interaction System Online Enumeration Q&A Online Data Processing Q&A Online Validation Rule Suggestion and Confirmation/Approval Online Enumeration Reporting System Online Data Processing Reporting System Online Patch and Other Supporting Files Upload and Download

Design of Software Systems Documents Handling System Record Keeping of Documents in Each Process (Receiving Batching, Editing, Cutting, Scanning, Correction, Completion, Validation, and Releasing)

Design of Software Systems: SP2010 Data Processing System Scanning System Recognition System Correction & Completion System Releasing System Validation System System Update Validation Rules Update Users Involved Scanning & Recognition Admin Correction & Completion Admin Correction and Completion Staffs Releasing & Validation Admin Validation Staffs & System Admin

Design of Software Systems: SP2010 Post Processing Utility System Double Counting Analysis Record Outliers Analysis Name and Address Analysis Age Analysis Name and Sex Analysis

Design of Software Systems Data Exploring System Area Analysis Time Series Analysis Online Analytical Processing (OLAP)

Some Hurdles Information Technology Blue Print In-House or Outsourcing There is still no general information technology blue print within the organisation Expected by the year of 2012, there will be one In-House or Outsourcing Common practice: in-house development Lack of standard, consistency, experience, and effectiveness Outsourcing? Lack of Funding

Further Advancement Managerial Support Systems Managerial Scheduling Systems Data Dissemination Supporting Systems Data Center Systems Data Warehouse Systems Metadata Systems A More General Systems for Handling Different Types of Surveys and Censuses General Design Universal Development Engine

Thank You Joint Meeting on Statistics Indonesia Joint Meeting on Management of Statistical Information Systems (MSIS) – Daejeon, 26-28 April 2010