3 孔子“Real knowledge is to know the extent of one's ignorance.”
4 Charles Darwin“Ignorance more frequently begets confidence than does knowledge”
5 Expertise and Evaluation To be good at a task requires certain skills and knowledge. Often, to know whether one is good or bad at that task requires the same skills and knowledge one needs to be good at the task.
6 English as an ExampleTo write good, grammatical English, you need language skills and knowledge acquired over years of training. But to know if you’re writing good or bad English, you need the same skills! You can’t know that you’re wrong or how to correct yourself unless you have the skills + knowledge.
7 PhysicsOne finding showed that people who were new to physics were less good at deciding how hard a physics problem was, compared to experts.
8 Dunning-Kruger Effect The Dunning-Kruger effect is when people:Perform badly because they lack the skills and knowledge for the task.Think they are performing well because they lack the same skills and knowledge to evaluate themselves.
9 Study #1The experimenters (Dunning and Kruger) asked some professional comedians to rate how funny certain jokes were. The comedians did their ratings, and the experimenters found that they were highly correlated: what one professional said was funny, the other professionals said was funny.
10 Example: Bad JokeQuestion: What is big as a man, but weighs nothing? Answer: His shadow. Expert Rating: 1.3 on 1-11 scale.
11 Example: Good Joke“If a kid asks where rain comes from, I think a cute thing to tell him is ‘God is crying’… And if he asks why God is crying, another cute thing to tell him is probably because of something you did.” Expert Rating: 9.6 on 1-11 scale.
12 Then the researchers asked undergraduate psychology students both: To rate the jokes andTo decide how well their answers agreed with the experts.
18 Two ProblemsHumor is subjective. Even the experts disagreed (r = .76).It’s possible that the unfunny students weren’t overestimating their own abilities, but underestimating others’ abilities:They thought that though they were bad, others were really bad. (Instead of thinking that others were good, but they were really good.)
19 Logic!Buses 1, 2, and 3 make one trip each day, and they are the only ones that riders A, B, C, D, E, F, and G take to work.Neither E nor G takes bus 1 on a day when B does.G does not take bus 2 on a day when D does. When A and F take the same bus, it is always bus 3.C always takes bus 3.
20 Logic!Traveling together to work, B, C, and G could take which of the same buses on a given day? (A) 1 only (B) 2 only (C) 3 only (D) 2 and 3 only (E) 1, 2, and 3
21 Logic!Traveling together to work, B, C, and G could take which of the same buses on a given day? (A) 1 only (B) 2 only (C) 3 only (D) 2 and 3 only (E) 1, 2, and 3
23 No ProblemsLogic is not subjective. Here we can see that the worst students thought they scored better than average AND thought they were better than average.
24 Learning from OthersIn the final experiment, the researchers invited the worst students and the best students in each task to “grade” the other students’ tests. Prediction: the worst students cannot learn they are bad by looking at others’ work, because they cannot tell which answers are right or wrong.
25 Worst Students % Ability % Score Raw Score Before 66.8 60.5 12.9 After 63.265.413.7Difference-126.96.36.199Actual10.19.2
26 Best Students % Ability % Score Raw Score Before 71.6 69.5 16.9 After 77.279.716.6Difference5.610.2-0.3Actual88.716.4
27 Real Life Consequences People like Deepak Chopra use scientifically flawed theories to sell products to unsuspecting consumers. Chopra is not knowledgeable enough about quantum physics to see that he’s wrong, or to recognize the flaws others point out.
29 Sample SizeStudents in this class and previous classes have critiqued scientific studies by claiming “there are too few participants” when the participants are in the thousands. This is a case where students are wrong, they cannot see that they are wrong, and this prevents them from learning what’s right.
33 Important PointsMost studies are not large-scale RCTs or even better large-scale meta analyses of lots and lots of RCTs. So most studies are likely to be false. But that doesn’t mean that science can’t find the truth: we can always conduct more research and do meta analyses.
34 Important PointsSome things we know are true. Science has conclusively proven them. Smoking causes lung cancer. We did not always know that.
35 Important PointsPrior probability matters. Most (positive) studies on homeopathy are probably false, because it’s highly unlikely magic water cures people. But a new study showing smoking was linked to some other cancer? Probably not false.
36 Important PointsWe often have lots of reasons to suspect a treatment will work– cell studies or animal studies or pilot studies. Strong prior probability decreases the chances of false positives.
37 (I’ll admit, I’m doubtful about acupuncture and acne, because of its low prior probability.)
38 Important PointsAs we saw, science relies on replication. When we fail to replicate a finding, we get rid of it. So just because most scientific findings are false does not mean that most scientific beliefs are false. We have already gotten rid of the bad beliefs by trying to replicate the findings and failing!
39 ConclusionIf you want to know what’s true, ask a scientist. If you think this class taught you to ignore science, you are a victim of the Dunning-Kruger effect.
41 “Inverse” Base Rate Neglect Base rate neglect is when tests become unreliable because a condition is very rare. But the opposite can happen too. When a condition is very common, tests that show you don’t have that condition are unreliable.
42 Lawyer vs. SkepticA lawyer’s job is to prove that her client is right. Any argument that supports the view is good. Any doubt that can be cast on the opposing view is good. Truth is not the goal: winning is.
43 Lawyer vs. SkepticIt’s different for critical thinkers. We are supposed to accept what’s true and reject what’s false– even if it’s what we believe that’s false.
44 HyperskepticismHyperskepticism, as I define it, is being a lawyer when you need to be a critical thinker. It’s using all your intelligence and ability to show that you are right instead of using it to find out the truth.
45 Cognitive DissonanceSometimes how we behave and how we believe are different in a way that conflicts. I smoke, but I believe that smoking is terrible.
46 Cognitive Dissonance“Cognitive dissonance” is the bad feeling you get when what is happening conflicts with your beliefs about what should be happening. There are many ways people can resolve their cognitive dissonance. I could stop smoking. Or I could stop believing that smoking is bad.
47 Motivated ReasoningPeople whose beliefs or attitudes are a strong part of their personality/ identity may be tempted to irrationally change their beliefs instead of their behaviors. (Alt. medicine, climate change, religion, evolution…)
48 Barack ObamaIn 2008, America elected its first black president: Barack Obama. Many Americans think that blacks are lazy, stupid, and inferior. They have treated blacks poorly in the past and have voted in ways that harm blacks.
49 Barack ObamaThis is where cognitive dissonance comes in. People decided that Obama was not legitimate. He was born in Kenya! He was anti-American! He wanted to hurt, and not help, Americans.
50 Foreign Domestic Helpers Amnesty international found that “they worked on average 17 hours a day, half were denied their statutory weekly day off and two-thirds had been physically or verbally abused.” [SCMP]
51 Foreign Domestic Helpers Most of the comments on this article immediately call these women “liars” and “schemers,” question the sample size (!), and produce conspiracy theories.
52 Foreign Domestic Helpers Why are people so angry? Why are they so disbelieving? If you’re a bad person who treats helpers like non-humans, cognitive dissonance takes over.
54 From the SCMP“According to Hong Kong police figures, there were 551 rape cases and 7,000 sexual assaults from 2007 to However, a survey conducted this month by the Hong Kong Women's Coalition on Equal Opportunities has found the real figures much higher…”
55 From the SCMP“About one in four women said they had been victims of domestic violence, but fewer than one in 10 told police. Also, 49 per cent of the respondents said they had been sexually harassed, but less than 3 per cent of them reported it…”
56 From the SCMP“Also, 49 per cent of the respondents said they had been sexually harassed, but less than 3 per cent of them reported it. Out of 402 respondents, 23 per cent said they had experienced domestic violence, 14 per cent sexual abuse, and 49 per cent sexual harassment.” [21 January 2013]
57 Hyperskepticism and Rape Culture The powerful like to keep their power and the privileged like to keep their privilege. Men are powerful and privileged and lots of men quickly become hyperskeptics when rape and sexual harassment are brought up.