Antonio Bernardi - Fulvia Cerroni - Viviana De Giorgi (Istat) An application to the Tax Authority Source (Sector Studies) Session: Administrative data 10 July 2008 A methodological process for assessing variables coming from administrative sources

10 July Agenda A methodological process for assessing variables coming from administrative sources Part 1 - Scheme for assessing administrative sources for statistical use Part 2 - The process for assessing variables: the theory Part 3 - An application to the Tax Authority Source - Sector Studies (SS)

10 July Agenda A methodological process for assessing variables coming from administrative sources Part 1 - Scheme for assessing administrative sources for statistical use Part 2 - The process for assessing variables: the theory Part 3 - An application to the Tax Authority Source - Sector Studies (SS)

10 July Background and motivations A methodological process for assessing variables coming from administrative sources use of administrative archives in place of statistical surveys much more information on small medium enterprises reducing the statistical burden development of a general scheme for validating administrative data as statistical ones focus on the process of assessing quantitative variables with benchmark Sector Studies (SS) compared with the statistical survey on SMEs as a benchmark source

10 July Scheme for assessing administrative sources 1/2 A methodological process for assessing variables coming from administrative sources Part 1

10 July Scheme for assessing administrative sources 2/2 A methodological process for assessing variables coming from administrative sources Part 1 Preliminary judgement on an administrative archive Is it possible to identify a well defined universe?yes/no Reference population for coverageyes (specify) Mean coverage level(specify percentage) Coverage level (by existing disaggregation)between … and … (specify) Are there any benchmark variables?yes (specify)/no Can data be imported in a SAS format?yes/no Data delivery timeliness(specify) Does it need a formal request for data releasing?yes/no Variables’ classificationsspecify existing problems Judgement we can/can not go on processing the source

10 July A methodological process for assessing variables coming from administrative sources Part 1 - General scheme for assessing administrative sources for statistical use Part 2 - The process for assessing variables: the theory Part 3 - An application to the Tax Authority Source - Sector Studies (SS)

10 July Scheme for assessing quantitative variables having a benchmark A methodological process for assessing variables coming from administrative sources Part 2 QUANTITATIVE ASSESSMENT QUALITATIVE ASSESSMENT INPUT: DATA (ARCHIVES) OUTPUT: VARIABLE’S ASSESSMENT FOR STATISTICAL USE

10 July Qualitative and quantitative assessment of a variable 1/2 A methodological process for assessing variables coming from administrative sources 1.Outlier detection: irregular values/outliers Irregular values: legal and economic constraints are taken into account inexistence of a systematic scheme for them Outliers: 2 out of 3 criterions should be satisfied i.statistical/probabilistic (Bienaymé–Tchebicev) ii.computational/explorative (k-mean clustering method) iii.deterministic (relative differences within the threshold values of  5%,  2% or  1%) inexistence of a systematic scheme for them Part 2

10 July A methodological process for assessing variables coming from administrative sources 2. Standard validation: For both the source variable and its benchmark calculation of the main descriptive statistics (mean, std, median, asymmetry, kurtosis) and check whether the distance between the two variables decreases from the raw to the trimmed distribution through the kernel histogram check whether the series have the same graphical shape and the distribution of the deviations is symmetric, leptokurtic and with a zero mean. 3. Practical validation: It is useful for specific surveys and studies to check a level of concordance between the variable and its benchmark Frequency validation: concordance by class frequencies, simple index of dissimilarity, Cohen coefficient, relative weights of frequencies on the main diagonal, verification of correspondence by log-linear model adjusting test By group validation: per group concordance by checking the linearity of the groups’ means Micro-data validation: robust point to point correspondence through regression techniques Quantitative assessment of a variable 2/2 Part 2

10 July A methodological process for assessing variables coming from administrative sources Part 1 - General scheme for assessing administrative sources for statistical use Part 2 - The process for assessing variables: the theory Part 3 - An application to the Tax Authority Source - Sector Studies

10 July A methodological process for assessing variables coming from administrative sources Part 3 Assessing the source: The accounting table of Sector Studies Preliminary judgement on the accounting table of Sector Studies Is it possible to identify a well defined universe?yes Reference population for coverage Italian Business Register (ASIA) Mean coverage level79.4% Coverage level (by existing disaggregation)between 65% and 90% Are there any benchmark variables?yes (SME survey) Can data be imported in a SAS format?yes Data delivery timeliness15-months time lag Does it need a formal request for data releasing?yes Variables’ classifications some differences exist but they can be overcome Judgement the accounting table can be processed through the procedure for assessing variables

10 July A methodological process for assessing variables coming from administrative sources Part 3 Qualitative assessment First hypothesis: assess each cost variable of Sector Studies with its own SME survey benchmark Results: comparison of definitions is not effective for each variable. Even forcing the definition, the numerical evaluation is not effective: an appropriate combination of variables and its new benchmark should be taken into account Second hypothesis: assess total cost of Sector Studies with the total cost of SME survey Total cost of SS = Total cost of SME survey Assessing the variable: the total cost 1/5

10 July A methodological process for assessing variables coming from administrative sources Part 3 Quantitative assessment Outlier detection and standard validation Assessing the variable: the total cost 2/5

10 July A methodological process for assessing variables coming from administrative sources Part 3 Fig 1. Distribution of the deviations of SS from SME survey values Assessing the variable: the total cost 3/5

10 July A methodological process for assessing variables coming from administrative sources Part 3 Practical validation Frequency validation the independence between the two sources does not exist: the percentage of frequencies on the main diagonal (79.8%) plus the percentage found on its contiguous lines achieves 95.8% By group validation Assessing the variable: the total cost 4/5

10 July A methodological process for assessing variables coming from administrative sources Part 3 Micro-data validation Correlation coefficient (Pearson): Linear regression: TC (SS) = α + β×TC (SMEs) a ≈ 0 b ≈ 1 R 2 = Point to point correspondence (through the robust regression method) : 87,8% Conclusion Judgment on the total cost: the variable is reliable at an individual level Assessing the variable: the total cost 5/5

10 July Summary of the overall process A methodological process for assessing variables coming from administrative sources Part 3

10 July Thank you for your attention For further information: Antonio Bernardi: Fulvia Cerroni: Viviana De Giorgi: