1. 2017 Empirical Philosophy Workshop – VUW Dan Weijers & Peter Unger
Watch the James Dean and Duration Neglect hypotheses crash and burn: Lessons for experimenters
2017 Empirical Philosophy Workshop – VUW
Dan Weijers & Peter Unger

2. The general approach
Thought experiments are often used to elicit views about the value of things
Thought experiments can mislead us in ways we are not aware of
2 & 3 are dubious and can be challenged by x-phi
1 isn’t dubious, but x-phi might reveal that people don’t accept it
An experience machine life has more happiness than a normal life
If happiness is all that really matters in life, then the vast majority of people would choose an experience machine life over a normal life
The vast majority of people would not choose an experience machine life over a normal life
Therefore, happiness is not all that really matters in life

3. Deconstructive replication
Test the original scenario
Ask why they chose that
Assess the justifications and relevant biases literature to see what modifications are required
Make 1 change at a time, test new scenarios on similar groups, compare results
There is this amazing experience machine designed by super-duper neuropsychologists…
Would you live the rest of your life in such a machine?
Yes No
Briefly justify your choice for 1.

4. The target One aspect (so far) of Diener et al.’s length of life cases
Diener, E., Wirtz, D., & Oishi, S. (2001). End effects of rated life quality: The James Dean effect. Psychological Science, 12(2),

5. Work in progress warning
Formal statistical analyses have not yet been run
Formal qualitative analyses have not yet been run

6. Hypotheses about the value of happy lives of various lengths
The Common Sense Hypothesis
Longer lives are more desirable than shorter lives
The James Dean Hypothesis
Shorter lives are more desirable than longer lives
The Duration Neglect Hypothesis
The length of a life doesn’t affect its desirability
Doesn’t have a preference

7. (Partial) Replication of Diener et al.
Jen [Jan] was a never-married woman without children. Her life was extremely happy, with enjoyable work, vacations, friends, and pleasant leisure. Jen died suddenly and painlessly in an automobile accident, when she was 30 [60] years old.
Taking Jen’s life as a whole,
how desirable was her life?
How much total happiness
would you say that Jen
experienced in her life?
Please briefly explain your answer to questions 1&2:
Most un-desirable 1 2 3 4
Neutral 5 6 7 8
Most desirable 9 O
Most un-happy 1 2 3 4
Neutral 5 6 7 8
Most happy 9 O

8. results
Diener et al. took the average of the desirability and total happiness responses for each respondent, and then compared the average of respondents’ judgments about the 30 and 60 year life.
We did the same
Diener et al.: “The age variable failed to have a significant effect,... This finding confirms the phenomenon of duration neglect…”
Our results are similar
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
Diener et al.
6.13
6.48
0.35
Weijers & Unger
6.26
6.15
-0.11

9. Implications for philosophy? If Diener et al. Were right, then…
Prudential value: what is good for us
Used in moral theory, political philosophy, applied ethics, and public policy
Should governments focus on making our lives happy at the cost of them being long?
No speed limit
Decriminalize drugs
Subsidize alcohol and drugs
Scrap cancer research in favor of subsidizing vacations
Quantitative hedonism: lives are valued by summing the net happiness of each moment
Prudential theories that value lives in a way different to the vast majority of people are bad theories
The vast majority of people do not value lives by summing the net happiness of each moment
Therefore, quantitative hedonism is a bad theory

10. Or we can take a closer look: Qualitative analysis
Method
There are 9 main categories of answer. Which main category an answer fits in depends on 3 things:
The question that motivated the thought experiment (related to what it is supposed to show)
The exact wording of the scenario and questions
The respondents’ answers to the questions
Then group by meaning within categories
Categories
Malicious response
Opposite justification
Imaginative resistance
Overactive imagination
Reasonable resistance
Useful response
No justification
Reasonable rejection
Demonstrates misunderstanding

11. Deconstructive replication: quant/Qualitative analysis
Categories
Malicious response
Opposite justification
Imaginative resistance
Overactive imagination
Reasonable resistance
Useful response
No justification
Reasonable rejection
Demonstrates misunderstanding
Uses
% of 6+7 answers can be used to assess a thought experiments fit for purpose
Compare “fit for purpose” scores of scenario wording changes to ensure they improve
% of1+2+9 (+3+4+7?) can be used to gauge the relative suitability of sample groups
Sub-groups of 3, 4, 5, & 8 used to criticize original & inform creation of new scenarios
Sub-groups of 6 used as support for theories
Could just use #s from 6 or 6+7 for quant analysis

12. Or we can take a closer look: Methodology
Potential methodological problems
Many respondents de-valued the lives because the women were “never-married… without children”
Evaluating a life without a comparator is difficult
“Total happiness” data might not be relevant to desirability
The scale and labels might cause problems (“most desirable” and “most undesirable” are ambiguous)
Potential solutions
Rewrite the scenario so it doesn’t mention whether they were married with children or not
Compare Jen’s (30) and Jan’s (60) lives directly against each other
Do not combine the “desirability” and “total happiness” responses
Run the study with a simpler response mechanism (whose life is more desirable, or are they equal?)

13. Or we can take a closer look: Harder test for JDH & DNH
Scenario setup tweaks
Having 3 options might help participants think more carefully about the potential differences between the lives
Increasing the age gap should make the difference between the lives more salient
Making the judgments about a life you will have (rather than the life of a stranger) should make the participant take any differences in value more seriously
Tweak details
E.g., Compare Jen’s (30), Jay (45), and Jan’s (60) lives directly against each other
E.g., Compare Jen’s (25) and Jan’s (75) lives directly against each other
E.g., “There are two lives…” “…and assuming you are forced to choose one to live yourself…”

14. And we can combine tweaks
Which enables the investigation of…
Whether some tweaks only seem to have an effect in the presence or absence of other tweaks
The average effect of each tweak across slightly different contexts
The total effect of all of the tweaks combined
So we can learn (approximately)…
About any interrelationships between tweaks
The relative importance and absolute effect of each tweak
Just how bad the original study was, and how people really value the desirability of happy lives of different lengths

15. Tweak 1: Clean “never married” etc.
Clean: “Imagine someone was extremely happy for every year of their life, until they died suddenly and painlessly in an automobile accident at age 30 [60].”
RESULTS
General increase
Results now favor commonsense
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
Weijers & Unger
Rate 1 (BS)
6.26
6.15
-0.10
Rate 1 (BS)-clean
6.73
7.32
0.58
Rate both (WS)
5.81
6.60
0.79
Rate both (WS)-clean
7.22
8.23
1.02

16. Tweak 2: Directly comparing the lives
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
Weijers & Unger
Rate 1 (BS)
6.26
6.15
-0.10
Rate both (WS)
5.81
6.60
0.79
Rate 1 (BS)-clean
6.73
7.32
0.58
Rate both (WS)-clean
7.22
8.23
1.02
Respondents were presented with both lives at the same time (i.e. within subjects design) with the same question wording as before
RESULTS
Increase for 60 year lives
Results favor commonsense even more

17. Tweak 3: Desirability results only
Instead of combining “desirability” with “total happiness”, we just took the desirability measure
RESULTS
Increase (for WS) evaluations of 60 over 30 year life
Study
30 combined
60 combined
Combined 60-30
30 desire only
60 desire only
Desire only 60-30
Diener et al.
Rate 1
6.13
6.48
0.35 ?
Weijers & Unger
6.26
6.15
-0.10
5.74
5.35
-0.40
Rate both (WS)
5.81
6.60
0.79
4.72
6.21
1.49
Rate 1-clean
6.73
7.32
0.58
6.12
6.66
0.54
Rate both (WS) clean
7.22
8.23
1.02
6.40
7.83
1.43

18. Tweak 4: Directly comparing the lives
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
W&U, Rate both (WS), clean, 2-option
7.22
8.23
1.01
W & U, Rate both (WS), clean, 3-option
6.43
7.71
1.28
W&U, Rate both (WS), clean, 2-option, bigger age gap (25-75)
6.25
8.04
1.79
W&U, Rate both (WS), clean, 3-option, bigger age gap (25-75)
5.02
7.54
2.52
W&U, Rate both (WS), dirty, 2-option
5.81
6.60
0.79
W&U, Rate both (WS), dirty, 3-option
5.74
6.62
0.88
Added a middle option (Jay: 45)
RESULTS
Mixed: this had a ‘commonsense’ effect on ‘total happiness’ judgments, but not ‘desirability’

19. Tweak 5: Bigger age gap Made Jen die at 25 and Jan 75 RESULTS
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
W&U, Rate both (WS), clean, 2-option
7.22
8.23
1.01
W&U, Rate both (WS), clean, 2-option, bigger age gap (25-75)
6.25
8.04
1.79
W&U, Rate both (WS), clean, 3-option
6.43
7.71
1.28
W&U, Rate both (WS), clean, 3-option, bigger age gap (25-75)
5.02
7.54
2.52
Made Jen die at 25 and Jan 75
RESULTS
Results favor commonsense even more

20. Tweak 6: Making it personal
“There are two lives…” “…and assuming you are forced to choose one to live yourself…”
RESULTS
Results favor commonsense even more, especially if just desirability is used
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
W&U, Rate both (WS), clean, 2-option
7.22
8.23
1.01
W&U, Rate both (WS), clean, 2-option, personal
5.64
7.62
1.98
W&U, Rate both (WS), clean, 3-option
6.43
7.71
1.28
W&U, Rate both (WS), clean, 3-option, personal
4.84
6.39
1.55

21. Combining Tweaks
Including all 6 tweaks, compare to our replication of D’s original
RESULTS
Huge difference: 4.04 on a 1-9 scale (over 50% movement!)
The totally tweaked version strongly supports commonsense
Study
Jen (30-year life)
Jan (60-year life)
Jan - Jen
W&U, Rate 1 (BS)
5.74
5.35
-0.40
W&U, clean, Rate both (WS), desi only, 3-option, bigger gap (25-75) personal
4.00
7.64
3.64

22. Differences tweaks made
Using desirability only
Simple average of comparisons (doesn’t take varying sample sizes into account)
Tweak
# of com-parisons
Average contribution
1) Cleaning up “no kids” etc language 3
+0.54
2) Within subjects design (rate 2), rather than BS 2
+1.39
3) Desirability scores only, rather than combo average with total happiness scores 10
+0.82
4) 3rd option added in middle 4
-0.27*
5) Bigger age difference (most instead of 30-60)
+1.20
6) Making it personal, rather than about a stranger
+0.79
*But did increase total happiness contribution

23. conclusion
Specific
Some students think they think that length of life doesn’t matter
Most students (probably) think that length of life is important, but far from the only intrinsically important thing in life
i.e., there is probably a partial duration neglect effect when we evaluate lives in ideal conditions
At least with these kinds of evaluations (“online” vs “narrative/3rd person/objective” may be different)
General
Experimental philosophy/ psychology is hard
It takes a long time
Multiple methods should be combined for more robust results
It’s no wonder many experiments’ results aren’t highly replicable!
Always ask the qualitative question
Some people believe very unusual things