Challenges in Collecting Police-Reported Crime Data Colin Babyak Household Survey Methods Division ICES III - Montreal – June 20, 2007

Overview Structure of the Uniform Crime Reporting Survey (UCR) Structure of the Uniform Crime Reporting Survey (UCR) UCR vs. a “typical” business survey UCR vs. a “typical” business survey Data quality Data quality Recent developments Recent developments Future work Future work

Structure of the UCR 2 versions of the survey 2 versions of the survey Microdata (94%) Microdata (94%) Aggregate data (6%) Aggregate data (6%) ~1200 respondents (police services) ~1200 respondents (police services) Extraction of “administrative” data Extraction of “administrative” data 4 different vendors for extraction 4 different vendors for extraction Some respondents build their own system Some respondents build their own system

Structure of the UCR Receive information on: Receive information on: Incident Incident Accused Accused Victims Victims Monthly submissions Monthly submissions Monthly edit reports Monthly edit reports Monthly corrections Monthly corrections All statistics are annual All statistics are annual

UCR vs. a “typical” business survey Similarities: Population is skewed Population is skewed Most respondents are small in size Most respondents are small in size Frame is well-established, good quality Frame is well-established, good quality Regular, personal contact with the largest respondents Regular, personal contact with the largest respondents Respondent data relatively consistent over time Respondent data relatively consistent over time

UCR vs. a “typical” business survey Differences: UCR Census Census Extract admin. data Extract admin. data Respondents can be recontacted re: errors Respondents can be recontacted re: errors Data released at respondent level Data released at respondent level Multiple records per respondent Multiple records per respondent “Typical” survey Sample Questionnaire Respondents usually not recontacted Data released at aggregate level One record per respondent

UCR vs. a “typical” business survey Impact of differences: We cannot “treat” respondent errors without their consent We cannot “treat” respondent errors without their consent Non-respondents need to be consulted and “sign off” on their data Non-respondents need to be consulted and “sign off” on their data Very difficult to determine a response rate Very difficult to determine a response rate

UCR vs. a “typical” business survey Impact of differences (cont): Collecting new information is difficult: Collecting new information is difficult: Years to implement Years to implement Vendors do not update immediately Vendors do not update immediately Respondents do not update immediately Respondents do not update immediately In-house do not re-program immediately In-house do not re-program immediately Recent additions include: Recent additions include: Cybercrime, Hate Crime, Organized Crime, Geocoding, FPS Number Cybercrime, Hate Crime, Organized Crime, Geocoding, FPS Number

Data Quality Monthly edit reports Monthly edit reports 6-month review of aggregate data 6-month review of aggregate data Outlier detection of aggregate data Outlier detection of aggregate data Year end sign-off of data for major respondents Year end sign-off of data for major respondents Analyze distributions of key variables Analyze distributions of key variables

Recent Methodological Developments Analysis of new variables Analysis of new variables Spatial modeling Spatial modeling Correction rates Correction rates Record linkage projects Record linkage projects Time series imputation Time series imputation Key variable distribution analysis Key variable distribution analysis

Recent Developments New Variables Establishing baseline data for: Establishing baseline data for: Cybercrime Cybercrime Organized Crime Organized Crime Hate Crime Hate Crime First data release in Spring 2007: First data release in Spring 2007:

Recent Developments Spatial Modeling Goal is to determine explanatory variables for crime at neighbourhood level Goal is to determine explanatory variables for crime at neighbourhood level Observations are not independent Observations are not independent Using spatial models to “filter out” spatial effects Using spatial models to “filter out” spatial effects Has shown that traditional models are inefficient for neighbourhood crime data Has shown that traditional models are inefficient for neighbourhood crime data

Recent Developments Correction Rates Important data quality indicator Important data quality indicator Are respondents acting on the E&I reports? Are respondents acting on the E&I reports? Varies greatly across respondents Varies greatly across respondents Concrete information for follow-up Concrete information for follow-up

Recent Developments Record Linkage Creation of “quality codes” to reduce false positive matches Creation of “quality codes” to reduce false positive matches Time Series Using time series to impute for missing or poor quality data Using time series to impute for missing or poor quality data

Recent Developments Variable Distribution Analysis Analyze the distribution of certain key variables: Analyze the distribution of certain key variables: Relationship, Weapon, Location, etc. Relationship, Weapon, Location, etc. Useful data quality tool Useful data quality tool Score function to detect biggest anomalies Score function to detect biggest anomalies

Future Work Microdata imputation Microdata imputation Formalized time series imputation Formalized time series imputation Proactive and more timely data quality measures Proactive and more timely data quality measures Periodic audits of respondents Periodic audits of respondents Response / imputation rates Response / imputation rates