Bulletproofing the investigation JAWS 2014 Jennifer LaFleur, CIR (With artwork from young friends of journalism)

Investigative reporting is a powerful tool.

Snapshot of big famous story.

It can make this

Into this

It gives us the power to check assumptions.

Know the extremes

First, we must face a fact.

But don’t be afraid…

Bulletproof your data and documents Before ever reporting data or building an app Do integrity checks to find the flaws Add caveats where necessary Do your own analysis rather than relying on an agency’s analysis

External checks Understand what the data and documents mean. Know how many records you should have. Check counts and totals against reports. Are all possibilities included?

Internal checks Compare fields to check for red flags More teachers than students More money going to vendors than to contractors What things just don’t make sense

Integrity checks for every data set Check for missing data/documents

Integrity checks for every data set Check for missing data Check for duplicates

Duplicates may not always be obvious

Integrity checks for every data set Check for missing data, misplaced data or blank fields Check for duplicates Take margins of error into account if necessary

Median HH Income by Metro Area

Integrity checks for every data set Check for missing data, misplaced data or blank fields Check for duplicates Take margins of error into account if necessary (important if you’re using Census data). Check for outliers and extreme ups and downs

Truck accidents by year and agency

Beyond the basics Keep a notes file If you’re working with data or documents – keep a copy of your original Know the source Check against summary reports As you begin to write, footnote your documents and data

Texas test score data official results versus district Duncanville district reported 4 th grade writing Official report for Duncanville 4 th grade writing Courtesy Holly Hacker, The Dallas Morning News

Beyond the basics Keep a notes file If you’re working with data or documents – keep a copy of your original Know the source Check against summary reports Use the right tool

H/T Matt Waite

Disparities in water usage “Water use highest in poor areas of the city” Mapping and statistical analysis

Beyond the basics Check with experts Know the standards Find out what others have done Gut check – does it just seem wrong?

Beyond the basics Check with experts Know the standards Find out what others have done Gut check Go physically see a record or spot check against documents

If you’re matching or backgrounding people… A name is not enough. Lots of people have the same name Get dates of birth and other information to make sure you have the correct person.

Even people with seemingly unique names aren’t so unique

Be transparent: Bounce your findings off targets and tell users/readers/viewers what you did.

From the data source Get the questionnaire and methodology Beware of nonscientific methods: Web surveys, man on the street Know the sample size Account for margin of error and non-response Run statistical tests on the data if possible

Erroneous government databases can often be a story themselves

compared to

What appears to be an error may be the story.

Why data? Contrasts are in the data Your most powerful figures are in the data You can make connections you might not be able to make otherwise

Why data? Contrasts are in the data Your most powerful figures are in the data You can make connections you might not be able to make otherwise Test assumptions

When there is no data, you may have to make your own.

From the data source Get the questionnaire and methodology Beware of nonscientific methods: Web surveys, man on the street

@j_la28 A special thanks to: Tracy, Olivia, Iris, Emmeline, Thomas, Silas, Carson, Harper and Erin