1. Performance Prediction: Truths and Falsehoods
Dr. Jordan B Peterson
Professor of Psychology
University of Toronto

2. What do psychologists do?
Pursue scientific truth
Pursue careers

3. How to pursue scientific truth
MEASUREMENT
If the phenomenon cannot be measured
It does not exist

4. How to pursue scientific truth, continued
CONSTRUCT VALIDATION
Multimethod, multitrait
Analogous to sensory analysis
Five senses, not one
Plus intrasubjective measurement
Plus technological extension of sensory analysis

5. Use statistics destructively
Aim at demolishing your effect

6. How to pursue a career Invent a scale (or a construct)
Or rename a scale that already exists
Refuse to validate it
No convergence
No divergence
No criterion
No multimethod, multitrait analysis
Correlate it with some phenomenon

7. How to pursue a career, continued
Publish the paper
Establish a small primate dominance hierarchy
Based on the scale
Climb the hierarchy
Use statistics selectively
Very selectively
Enjoy the fruits of your success!
But seriously distort the knowledge base
And spend your time chasing red herrings

8. NATURE | NEWS FEATURE Replication studies: Bad copy Ed Yong
In the wake of high-profile controversies, psychologists are facing up to problems with replication.
Ed Yong
16 May 2012

9. Things that might not exist
Self-esteem
(neuroticism – extraversion)
25000 published papers
Whole political and educational agendas
Working memory and EF
IQ?
Emotional intelligence
Agreeableness

10. Things that might not exist
Grit
MacArthur Genius Grant (Angela Duckworth)
Martin Seligman (positive psychology)
Conscientiousness
Optimism/Pessimism
Extraversion/neuroticism
Promotion/prevention
Extraversion/conscientiousness

11. Things that might not exist
Empathy
Measures do not correlate well, but distill to agreeableness
Psychopathy
Agreeableness, negative, minus conscientiousness
Predatory parasite

12. Things that might not exist
Positive illusions
Extraversion, by Taylor’s own analysis
Practical intelligence
Sternberg is a crook
Multiple intelligences
Gardner is a crook

13. Things that might not exist
Any questionnaire measure independent of the Big Five
Any measure of cognitive function independent of IQ
Any behavioral measure of a trait, independent of IQ and the Big Five

14. Things that might exist

15. Individual Differences
Contextual and individual factors
Contextual
Cultural factors
Peer networks
Social support
Individual
Personality
Cognitive ability

16. Personality

17. Traits Measurement Items Employment Variables Economic Value

18. Traits

19. STABILITY Conscientiousness Emotional Stability Agreeableness
Industriousness
Orderliness
Conscientiousness
Volatility (r)
Withdrawal (r)
STABILITY
Emotional Stability
Politeness
Compassion
Agreeableness

20. PLASTICITY Openness Extraversion Creativity Intellect Assertiveness
Enthusiasm
Extraversion

21. Personality Extraversion Neuroticism Agreeableness Conscientiousness
Happiness, optimism, enthusiasm, gregariousness, assertiveness
Neuroticism
.2-.3 with major life outcomes
Particularly anxiety and depression, since that’s what it measures
Agreeableness
Compliance, empathy, maternality, kinship
criminality
Conscientiousness
.4 with major life outcomes
Openness
Creativity, intelligence
Disagreeable/neurotic
Personality disorders
Gender differences
75% classification accuracy

22. Emotion & Motivation Extraversion Neuroticism Agreeableness
Incentive reward (hope, curiosity, play, enthusiasm)/dopamine
Harvest attention/approach and exploit social situations
Neuroticism
Pain, frustration, disappointment, fear, anxiety/GABA, Serotonin
Avoid threat, uncertainty and punishment
Agreeableness
Empathy, sympathy, compliance, CARE (Oxytocin)
Form intimate relationships and share Vs
Pursue individual agenda/defend territory
People vs things
Conscientiousness
Orderliness: DISGUST
Industriousness: GUILT, SHAME
Maintain order
Stay uncontaminated and sparkly clean
Implement goals
Openness
Manipulate abstractions prior to implementation
General Cognitive Ability
Awe, curiosity
Non-social incentive reward/dopamine

23. Models are grounded in motivation
Capitalize on social groups
Maintain order

25. Measurement

26. Items

27. Measurement Trait Measures General cognitive ability Creativity
Big Five Aspect Scale
Unfakeable Big Five
General cognitive ability
Fluid
Crystallized
Dorsolateral prefrontal
Creativity
Creative Achievement Questionnaire
Divergent thinking and fluency

28. (BFAS) Conscientiousness
Industriousness
Waste my time (r)
Always know what I am doing
Orderliness
Am not bothered by disorder (r)
Follow a schedule

29. (BFAS) Emotional Stability
Volatility
Get angry easily
Keep my emotions under control
Withdrawal
Seldom feel blue
Am filled with doubts about things

30. (BFAS) Agreeableness Politeness Compassion Respect authority
Believe that I am better than others (r)
Compassion
Am not interested in other people’s problems (r)
Take no time for others (r)

31. (BFAS) Openness Creativity
Believe in the importance of art
Love to reflect on things
Intellect (but see general cognitive ability)
Am quick to understand things
Can handle a lot of information

32. (BFAS) Extraversion Assertiveness Enthusiasm Take charge
Lack the talent for influencing people (r)
Enthusiasm
Make friends easily
Am hard to get to know (r)

33. Unfakeable Big Five (+)

34. Unfakeable Big Five (-)

35. Cognitive Ability The complexity of the world
Motivated ends must be met
Modeling of possibility prior to implementation
Models generate solutions to three problems:
Where are you?
Where are you going?
How are you going to get there?
Allows testing and failure
Popper: “Our hypotheses die in our stead”

36. Conceptions of cognitive ability
INTELLIGENCES
Practical vs analytical
Social, emotional, moral
Multiple: linguistic, musical, logical-mathematical, spatial, body-kinesthetic, intrapersonal, interpersonal
INTELLIGENCE
Highest strata has the most explanatory power
“g” measures “success” across broad domains
r ~ .50

38. Effect Size Relative magnitude Binomial Predictor r = .30
r= %ile
r= %ile
r= %ile
r< %ile
Hemphill, J.F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78-8.
Binomial
Predictor r = .30
From 50/50
To 65/30
Predictor r = .50
To 75/25

39. General Cognitive Ability

40. Creative Achievement Questionnaire
Creativity
Creative Achievement Questionnaire
Visual Arts (painting, sculpture)
0. I have no training or talent in this area.
1. I have taken lessons in this area.
2. People comment on my talent in this area.
3. I won one or more prizes at juried art shows.
4. I showed my work in a gallery.
5. I sold a piece of my work.
6. My work was critiqued in local publications.
7. My work was critiqued in national publications.
Inventions
0. I do not have recognized talent in this area.
1. I find novel uses for household objects.
2. I sketched out an invention and worked on its design flaws.
3. I created original software for a computer.
4. I built a prototype of one of my designed inventions.
5. I sold an inventions to people I know.
6. I received a patent for one of my inventions.
7. I sold an invention to a manufacturing firm.

41. Employment Variables

42. Complexity
Innovation
Hi/Hi
Entrepreneurial
Lo/Hi
Managerial/Administrative
Lo/Lo
Rote Tasks
Hi/Lo

43. Who suits which position?
Cognitive ability
High levels of fluid and executive intelligence
Decision making and planning
Research
High levels of retrieval
Creativity and ideational production
Leadership and public speaking
High levels of crystallized intelligence
Information dispensing and storage
Writing and lecturing

44. Who suits which position?
Specific Performance
Openness (r = )
Associated with high levels of creativity/entrepreneurial ability
Extraversion (r = .20)
Managers
Sales People
Leadership
Agreeableness
Lower levels associated with creativity/managerial excellence
Higher levels for public relations, customer service (?)

45. Openness
Complexity and Intelligence
Innovation and Creativity

46. Conscientiousness
Duty and Industriousness
Disgust and Orderliness

47. Employment Variables II
Emotional Stability
High vs Low Stress
Agreeableness
People vs Things
Extraversion
Solitary vs Gregarious

48. Economic Value

49. Complexity and cognitive ability
95-86 percentile
IQ 130 – 116
85-73 percentile IQ
Attorney, Research Analyst
Editor, Advertising Manager
Chemist, Engineer, Executive
Manager, Trainee,
Systems Analyst, Auditor
Copywriter, Accountant
Manager/Supervisor
Sales Manager
Sales, Programmer
Analyst, Teacher, Adjuster
General Manager
Purchasing Agent
Registered Nurse
Sales Account Executive

50. Complexity and cognitive ability
Administrative Assistant
Store Manager, Bookkeeper
Credit Clerk. Drafter, Designer
Lab Tester/Tech, Assistant Manager
General Sales, Telephone Sales
Secretary, Accounting Clerk,
Medical Debt Collection
Computer Operator
Customer Service Rep
Technician, Automotive Salesman
Clerk, Typist
Dispatcher, General Office
Police Patrol Officer
Receptionist, Cashier
General Clerical
Inside Sales Clerk, Meter Reader
Printer, Teller, Data Entry
Electrical Helper
70-60 percentile IQ
55-50 percentile IQ

51. Complexity and cognitive ability
45-42 percentile
IQ 98-95
37-21 percentile
IQ 93-87
Machinist, Food Dept Manager
Quality Control Checker
Claims Clerk, Driver/Deliveryman
Security Guard, Unskilled Labor
Maintenance, Machine Operator
Arc Welder, Die Setter, Mechanic
Medical/Dental Assistant
Messenger, Factory Production
Assembler, Food Service Worker
Nurse’s Aide, Warehouseman
Custodian/Janitor
Material Handler
Packer

52. Economic Value I Predictive Power Intelligence
r = .6 to r = .2
Intelligence
Conscientiousness (managerial)
Openness Creativity (innovative)
Emotional Stability
Agreeableness
Extraversion

53. Economic Value II Binomial Effect Size Price’s Law
Economic Impact of Prediction

54. Pareto Distribution

55. Individual variation and productivity

56. Economic value and placement
Number of years
Current - Potential Predictor
Job Complexity
High complexity = high variability
Salary
Theoretically proportionate to output
Average output must be 2x salary

58. Process
Advertise
Test online
Screen
Interview best candidates

60. The Founder Institute is the world's largest entrepreneur training and startup launch program, helping aspiring founders across the globe build enduring technology companies.
Based in Silicon Valley and with chapters across 35 countries, the Founder Institute has helped launch over 1,112 companies in 5 years.
The company's mission is to "Globalize Silicon Valley" and build sustainable startup ecosystems that will create one million new jobs worldwide. 

61. First, only roughly 30% of applicants are accepted to the program.
Then, less than 40% of accepted Founders generally make it through the program to Graduation.
To date, over 20,000 people have applied, and the Founder Institute can predict with 85% accuracy somebody's potential to become a successful technology entrepreneur.