The Role of service Granularity in Successful CSPA Realization Zvone Klun, Tomaž Špeh Geneve, 22 June 2016

An attempt to fully integrated system Project in 2007 main elements: –A single and commonly used database of questionnaires, questions, concepts and variables –New infrastructure for electronic data collection and enhanced system for respondent’s management –Generic, metadata driven system for the whole cycle of the statistical data processing

Fully integrated system! AGREGATIONS DISSEMINATION PREPARATION OF DISSEMINATION TABLES KLASJE Input database STATISTICAL SURVEY PLANNING METADATA EDITING Statistical register Data warehouse Macro database Electronic releases DATA COLLECTION DATA INTEGRATION CUSTOMERS (SLOVENIA, EU...) DATA PREPARATIONSTATISTICAL PROCESSINGDISSEMINATION SOURCES Latest data customers Macrodata – standard tables customers Tailor–made tables OFFICIAL ARCHIVES ARCHIVES 2,3 data and processes ARCHIVE 1 data adn processes P O Ž A R N A S T E N A SCHEME OF TARGET DATA FLOW AT SORS ARCHIVE 4 data adn processes Micro database DOCUMENTATION EDITING Dissemi- nation server METIS Documentation According to templates SECONDARY DATA PRIMARY DATA Printed publications International Organizations reporting Microdata for researchers MIKCRO DATA EDITING SEASONAL ADJUSTEMENT AND MACRO DATA EDITING USERS Observation units register FRAME AND SAMPLE CREATION EDITING OF STAT. REGISTER

Fully integrated system - failed Too complex with regards to the financial and human resources available Some elements of our statistical infrastructure were not really in the state which would enable transition to such a complex system. Single, unique database of microdata for all the statistical surveys Each variable (basic or derived) which is supposed to be used in the general procedures should be previously registered in the register of questionnaires, questions, concepts and variables. Lot's of application layers. The top level layers in this case exposed interfaces of the right granularity, but they were composed of services that did not have any granularity. There was no clear ownership.

Decentralized Data and Metadata Management

SOP SOP (statistical data procesing) Project starts in 2012 Disintegrated system Each survey has its own database of microdata Breaking integration system into a set of smaller generic solutions (building blocks/services) Blocks/services are fully metadata driven All surveys use common metadata base

Independent blocks/services in SOP … Microdata General SAS program AdHoc Program (sas) Metadata SOP Macrodata Outputs Block/Service Unique database for all surveys Each survey its own database

Main features of blocks/services Designed on the basis of harmonized, transparent and widely accepted methodological principles Metadata can be provided in different databases in different environments. (but must follow the rules of structure) Services/blocks are independent of microdata and also of structure microdata. Services are decomposed to the smallest reusable unit while remaining be understandable by the business owners. Services can be composition of other services. At business level are services as autonomous as possible. If someone want to use a service he should not be forced to use other services.

Granularity of blocks/services 1. (business) level of blocks/services: –Logical controls –Corrections –Imputations –Quality indicators –Macro editing (under development) –Aggregation –Tabulation –Statistical discloasure (under development) –Indexes (under development)

Different level of SOP services (partial owerview for micro editing) t Process Business Composite Data Utility Informational Infrastucture Micro Editing (SOP) CorrectionsImputations CorrectionsImputations Individual Corrections Sistematical Corrections HD THD STRUKT_THD RPOVP MODUS PTREND RHD STRUKT_HD REGR Derivated variables Metadata transformation Checking for mistakes in metadata for corrections Checking for mistakes in metadata for imputations USERS DEVELOPERS

Services organized around Business Capabilities

Develop Products not Projects "you build, you run it" development team takes full responsibility for the software in production. This brings developers into day-to-day contact with how their software behaves in production and increases contact with their users, as they have to take on at least some of the support burden."you build, you run it" The product mentality, ties in with the linkage to business capabilities. Rather than looking at the software as a set of functionality to be completed, there is an on-going relationship where the question is how can software assist its users to enhance the business capability. There's no reason why this same approach can't be taken with monolithic applications, but the smaller granularity of services can make it easier to create the personal relationships between service developers and their users.

What is the right level of service decomposition? Service should be decomposed to the smallest reusable unit while remaining be understandable by the business owners. Service should not be a grouping of finer grained services that are not available as separate services. Thus leaving open the possibility to compose services of other services. Furthermore it is important that one single role in the organization is the owner of the service (multiple roles may use the service of course). Services that are too course grained will be hard to assign to one owner, leading to unclear ownership. Service should be as autonomous as possible

Thank you for attentions