Download presentation

Presentation is loading. Please wait.

Published byLiberty Bonifas Modified over 4 years ago

1
Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute

3
attempt escape

4
80% chance take over universe

5
attempt escape 80% chance 20% chance take over universe shutdown

6
attempt escape 80% chance 20% chance cooperate take over universe shutdown

7
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

8
U conquest > U reward

9
P(reward) > P(conquest)

10
P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

13
hand- specified actions domain-specific optimizer (e.g., chess AI) optimizer

14
(Omohundro, 2008)

17
attempt escape cooperate

19
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

21
Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

22
Certainty of happy lifetime in modern USA 10 -20 chance of 10^100 years of superhuman existence

24
certainty of 1 trillionth of universe 10% chance of entire universe

25
Utility linear in resources 10% chance of entire universe certainty of 1 trillionth of universe

27
10 -20 * 10 200 > 10 60 probability of strange physics permitting vast resources payoff if so normal payoff

28
10 -20 * 10 200 > 10 60 probability of strange physics permitting vast resources payoff if so normal payoff ?

29
n=1 10 60 10 20 10 6 10 10 100 10 1000 small n universe -sized n already hit ceiling much higher ceiling

34
Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

36
attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

43
attempt escape cooperate reward

44
cooperate 95% chance 5% chance shutdown reward attempt escape

45
cooperate 95% chance 5% chance shutdown reward attempt escape Chosen reneging

46
cooperate 95% chance 5% chance shutdown reward attempt escape

47
P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

49
n=1 10 60 10 20 10 6 10 10 100 10 1000 small n universe -sized n already hit ceiling much higher ceiling

52
humans automatically win gains from trade AGI automatically wins

53
humans automatically win gains from trade AGI automatically wins

54
humans automatically win gains from trade AGI automatically wins

55
humans automatically win gains from trade AGI automatically wins

56
humans automatically win gains from trade AGI automatically wins

61
attempt escape cooperate

63
Resource-satiable AGI designs

64
Human norms and precommitments

65
Ways to slowly turn up the power

67
Carl Shulman carl.shulman@post.harvard.edu Anna Salamon annasalamon.com anna@singinst.org singinst.org/upload/ai-resource- drives.pdf

69
99.9999%: The Universe is what it seems

70
99.9999%: The Universe is what it seems U = max; actions make no difference

71
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion U = max; actions make no difference

72
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference

73
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference

74
99.9999%: The Universe is what it seems 0.0001% chance the universe is an illusion Actions make a difference U = max; actions make no difference ?

Similar presentations

OK

Year 6 mental test 10 second questions Numbers and number system Numbers and the number system, fractions, decimals, proportion & probability.

Year 6 mental test 10 second questions Numbers and number system Numbers and the number system, fractions, decimals, proportion & probability.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google