Presentation is loading. Please wait.

Presentation is loading. Please wait.

Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute.

Similar presentations


Presentation on theme: "Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute."— Presentation transcript:

1 Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute

2

3 attempt escape

4 80% chance take over universe

5 attempt escape 80% chance 20% chance take over universe shutdown

6 attempt escape 80% chance 20% chance cooperate take over universe shutdown

7 attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

8 U conquest > U reward

9 P(reward) > P(conquest)

10 P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

11

12

13 hand- specified actions domain-specific optimizer (e.g., chess AI) optimizer

14 (Omohundro, 2008)

15

16

17 attempt escape cooperate

18

19 attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

20

21 Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

22 Certainty of happy lifetime in modern USA chance of 10^100 years of superhuman existence

23

24 certainty of 1 trillionth of universe 10% chance of entire universe

25 Utility linear in resources 10% chance of entire universe certainty of 1 trillionth of universe

26

27 * > probability of strange physics permitting vast resources payoff if so normal payoff

28 * > probability of strange physics permitting vast resources payoff if so normal payoff ?

29 n= small n universe -sized n already hit ceiling much higher ceiling

30

31

32

33

34 Certainty of happy lifetime in modern USA 10% chance of 10^100 years of superhuman existence Posner 2004

35

36 attempt escape 80% chance 20% chance cooperate take over universe shutdown reward

37

38

39

40

41

42

43 attempt escape cooperate reward

44 cooperate 95% chance 5% chance shutdown reward attempt escape

45 cooperate 95% chance 5% chance shutdown reward attempt escape Chosen reneging

46 cooperate 95% chance 5% chance shutdown reward attempt escape

47 P(conq.)U conq. + P(shutdown)U shutdown P(reward)U reward >

48

49 n= small n universe -sized n already hit ceiling much higher ceiling

50

51

52 humans automatically win gains from trade AGI automatically wins

53 humans automatically win gains from trade AGI automatically wins

54 humans automatically win gains from trade AGI automatically wins

55 humans automatically win gains from trade AGI automatically wins

56 humans automatically win gains from trade AGI automatically wins

57

58

59

60

61 attempt escape cooperate

62

63 Resource-satiable AGI designs

64 Human norms and precommitments

65 Ways to slowly turn up the power

66

67 Carl Shulman Anna Salamon annasalamon.com singinst.org/upload/ai-resource- drives.pdf

68

69 %: The Universe is what it seems

70 %: The Universe is what it seems U = max; actions make no difference

71 %: The Universe is what it seems % chance the universe is an illusion U = max; actions make no difference

72 %: The Universe is what it seems % chance the universe is an illusion Actions make a difference U = max; actions make no difference

73 %: The Universe is what it seems % chance the universe is an illusion Actions make a difference U = max; actions make no difference

74 %: The Universe is what it seems % chance the universe is an illusion Actions make a difference U = max; actions make no difference ?


Download ppt "Carl Shulman, Singularity Institute Anna Salamon, Singularity Institute."

Similar presentations


Ads by Google