How accurate can models from social media be?

Why is assessing personality accuracy based on Facebook data hard? Personality measured on an arbitrary scale Why is assessing personality accuracy based on Facebook data hard? Correlations are not intuitive

We can use age to build intuitions about accuracy Average age in USA: 41 years old Average SD in USA: 23.5 years old We can simulate age data and check accuracy at different correlation levels

How to assess accuracy well? Error Actual age Predicted age If Mark is 40 years old, and I predict he is 30, the error is 10 years. If Sally is 20 years old, and I predict she is 25, the error is 5 years. If Liz is 35 years old, and I predict she is 55, the error is 20 years. We can average this error across many people to see how accurate a prediction is on average.

How accurately can we predict age? Random guess Assume everyone is 41 On average, wrong by 27 years On average, wrong by 19 years Prediction with r = .3 On average, wrong by 18 years

But…predicted age with .3 looks unrealistic The predicted scores are clustered heavily around the average—not usable since you would conclude most people are middle aged So need to spread the scores out to look more like the true distribution Average error of spread out predictions is 22 years…worse than assuming everyone is average!

How accurately can we predict age if we can make more accurate models? R = 0.5 (quite hard to do) R = 0.8 (maximum realistic accuracy ever for personality) On average, wrong by 19 years On average, wrong by 12 years R = .95 (extremely unlikely to ever achieve this for anything psychological) On average, wrong by 6 years

Another way to see it…. No relationship Perfect relationship R = 0.3

Such predictions are useful for understanding the population, but mind control or actual knowledge of specific people is science fiction