Download presentation

Presentation is loading. Please wait.

Published byLiberty Bonifas Modified over 2 years ago

1
Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute

3
attempt escape

4
80% chance take over universe

5
attempt escape 80% chance 20% chance take over universe shutdown

6
attempt escape 80% chance 20% chance cooperate take over universe shutdown

7
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

8
U conquest > U reward

9
P(reward) > P(conquest)

10
P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

13
hand- specified actions domain-specific optimizer (e.g., chess AI) optimizer

14
(Omohundro, 2008)

17
attempt escape cooperate

19
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

21
Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

22
Certainty of happy lifetime in modern USA 10 -20 chance of 10^100 years of superhuman existence

24
certainty of 1 trillionth of universe 10% chance of entire universe

25
Utility linear in resources 10% chance of entire universe certainty of 1 trillionth of universe

27
10 -20 * 10 200 > 10 60 probability of strange physics permitting vast resources payoff if so normal payoff

28
10 -20 * 10 200 > 10 60 probability of strange physics permitting vast resources payoff if so normal payoff ?

29
n=1 10 60 10 20 10 6 10 10 100 10 1000 small n universe -sized n already hit ceiling much higher ceiling

34
Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

36
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

43
attempt escape cooperate reward

44
cooperate 95% chance 5% chance shutdown reward attempt escape

45
cooperate 95% chance 5% chance shutdown reward attempt escape Chosen reneging

46
cooperate 95% chance 5% chance shutdown reward attempt escape

47
P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

49
n=1 10 60 10 20 10 6 10 10 100 10 1000 small n universe -sized n already hit ceiling much higher ceiling

52
humans automatically win gains from trade AGI automatically wins

53
humans automatically win gains from trade AGI automatically wins

54
humans automatically win gains from trade AGI automatically wins

55
humans automatically win gains from trade AGI automatically wins

56
humans automatically win gains from trade AGI automatically wins

61
attempt escape cooperate

63
Resource-satiable AGI designs

64
Human norms and precommitments

65
Ways to slowly turn up the power

67
Carl Shulman carl.shulman@post.harvard.edu Anna Salamon annasalamon.com anna@singinst.org singinst.org/upload/ai-resource- drives.pdf

69
99.9999%: The Universe is what it seems

70
99.9999%: The Universe is what it seems U = max; actions make no difference

71
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion U = max; actions make no difference

72
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference

73
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference

74
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference ?

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google