1. Crime survey Neuchatel, 7-8 July 2011
CBS case study
Crime survey
Neuchatel, 7-8 July 2011

2. Crime and victimization survey Planned domains: police districts
Introduction
Crime and victimization survey
Planned domains: police districts
Sample size approx 750 / district
: NSM
2008 onwards: ISM
SAE of crime statistics

3. Data collection: sequential mixed-mode Different questionnaire
From NSM to ISM
Local oversampling
Data collection: sequential mixed-mode
Different questionnaire
Discontinuities expected
SAE of crime statistics

4. Quantifying discontinuities
Survey transition from NSM to ISM
Small scale NSM in parallel to new ISM (full scale: approx 18,000; small: 1/3rd)
Discontinuities at national level
Now: police district level discontinuities required
But: NSM sample too small => SAE
SAE of crime statistics

5. Example of discontinuity Bicycle thefts NSM and ISM
2009
Total: 541,000 (NSM) ; 897,000 (ISM)
SAE of crime statistics

6. Coeff of variation, bicycle theft, 2009
NSM: 0.41 ; ISM: 0.24
SAE of crime statistics

7. SAE to increase precision of NSM
Fay-Herriot model: linear mixed, area level
EBLUP and HB estimators
Bayesian estimation of model variance also in EBLUP (avoiding zero-estimates of model variance)
SAE of crime statistics

8. Bayesian estimation of model variance
SAE of crime statistics

9. Covariates from registers
Police: Reported offences: property crimes, violence, assaults, threats, illicit drugs, weapons, vandalism, traffic offences
Administration: age, ethnicity, urbanisation, house prices, welfare claimants
SAE of crime statistics

10. Covariates from ISM survey
Design based GREG estimates as auxiliary information (Ybarra & Lohr 2008)
Consequences for small area estimates
Model estimate weighted lower in BLUP due to error in covariate
Achieved through higher estimate of model variance in EBLUP (not Y & L adjustment)
Variance of GREGs approx. equal for all areas
(Other idea: multivariate FH model)
SAE of crime statistics

11. Simulating errors in covariates
Bicycle thefts: NSM survey ~ police-reported
No error
post. mean model var = 1.22
Adding error, mean 0, sd 2, iterate 1,000 x
post. mean model var = 1.32
To add detail, e.g. estimated beta
SAE of crime statistics

12. Dimension reduction: PCA
Rather than using a small subset of covariates, use small dimension of PC subspace
Not guaranteed to work as correlation with survey variables not used in PCA
Use as a separate set of potential covariates in model selection pc 1 2 3 4 5 6 … 12
var. expl.
.39
.55
.67
.75
.81
.87
.99
SAE of crime statistics

13. PC space of dim 2
SAE of crime statistics

14. Cross validation (CV) LOO: leave-one-out, predictive accuracy
Model selection
Conditional AIC (Vaida & Blanchart 2005) cAIC = - 2 cond_llh + 2 eff_d ( AIC = - 2 llh + 2 d )
Cross validation (CV) LOO: leave-one-out, predictive accuracy
Start from minimal model, and add terms, maximizing improvement wrt cAIC or CV, until no further improvement
Llh = log likelihood evaluated at ML estimates of fixed effects coeffs beta and model var
Cond_llh = conditional llh: conditional on random effects
D = model complexity = number of fixed effects
Eff_d = effective degrees of freedom, between no random effect and random effect as fixed (between 1 and the number of areas), is trace of hat matrix
SAE of crime statistics

15. Model selection results
For each NSM survey variable: 2 years, 2 criteria
CV-models are larger
cAIC are nested within CV models
Hence:
Use cAIC models
Models differ between years
Alternative: choose single model for both years
SAE of crime statistics

16. Selected models 2008 2009 violent crimes satisf. police victimization
ISM-bicycle-theft, REG-property, REG-weapons, ISM-property
pc21, pc10, pc4
satisf. police
ISM-satisf
age,ISM-satisf,urbanisation
victimization
ISM-property, REG-property
pc1, pc21,pc5,pc6
property crimes
ISM-victim, elderly
pc1, pc21,pc2,pc5,pc6
nuisance
ISM-nuisance, elderly
ISM-victim, REG-traffic, ISM-property
feeling unsafe
ISM-nuisance, house val, ISM-satisf
ISM-unsfae, ISM-satisf
degradation
pc1,pc4,pc10,pc22
ISM-degrad
bicycle theft
ISM-bicycle
ISM-bicycle, ISM-satisf
SAE of crime statistics

17. Selected models excl. ISM
2008
2009
violent crimes PC
satisf. police
victimization
REG-property, elderly
property crimes
REG-property, age
REG-property, REG-traffic, REG-weapons
nuisance
feeling unsafe
degradation
urban, house val, REG-vandalism
bicycle theft
SAE of crime statistics

18. SAE results (hybrid EBLUP), reduction in coeff. of variation
incl. ISM
excl. ISM
violent crimes
-40 %
satisf. police
-47 %
-46 %
victimization
-43 %
-41 %
property crimes
-44 %
-42 %
nuisance
-33 %
feeling unsafe
-25 %
degradation
-35 %
-16 %
bicycle theft
-39 %
SAE of crime statistics

19. Bicycle theft, cv, 2009 NSM: 0.41, EBLUP: 0.23, ISM:0.24
SAE of crime statistics

20. SAE results, weight of direct est. in BLUP
incl. ISM
excl. ISM
violent crimes
0.21
0.27
satisf. police
0.24
0.35
victimization
0.20
0.22
property crimes
0.19
nuisance
feeling unsafe
0.39
0.41
degradation
0.31
0.64
bicycle theft
0.32
SAE of crime statistics

21. EBLUP vs. Hierarchical Bayes
Diff. point est.
Diff. var est.
violent crimes
-0.1 %
-4.7 %
satisf. police
+0.0 %
-3.8 %
victimization
-0.0 %
property crimes
-4.5 %
nuisance
-4.6 %
feeling unsafe
-3.1 %
degradation
-4.0 %
bicycle theft
-0.2 %
-2.6 %
HB accounts for uncertainty in estimating the model variance
SAE of crime statistics

22. Considerable increase in precision with SAE
Conclusions
Considerable increase in precision with SAE
Gain in precision depends on variable
PCA is important for some variables
Using ISM outcomes important for some variables
MSE estimates HB higher (preferable)
SAE of crime statistics

23. Sort out errors in input data! And re-run everything.
To do (maybe)
Sort out errors in input data! And re-run everything.
Calibration to direct estimate of totals (is model diagnostic)
Study residuals
Elaborate on errors in covariates
Use past survey outcomes as covariates
More detailed comparison of HB-NSM estimates with ISM
SAE of crime statistics

24. Future work (post-ESSnet)
Multivariate modelling of NSM and ISM variables
Consider model averaging
Using more detailed areas, with smaller sample sizes: beneficial?
SAE of crime statistics