1. Read Montague Baylor College of Medicine Houston, TX www.hnl.bcm.tmc.edu Reward Processing and Social Exchange

2. I love you Let’s do lunch B A A B A B

3. Natural visual statistics Specific algorithms evolved to solve efficiently the array of problems of early vision

4. Recharge or die The real world imposes another difficult requirement on all mobile organisms

5. Natural reward-harvesting statistics? generic woodland creature

6. Models of reinforcement learning can provide insight What computations should we expect? Reward harvesting is an economic problem involving both valuation and choice

7. Sustain goal Select goal Pursue goal Goal-directed choice Actual experience Counterfactual experience ‘what could have been’ guidance signals

8. Midbrain dopamine neurons burst pause R time naive R time After learning time After learning (catch trial) Pause, burst, and ‘no change’ responses represent reward prediction errors – ongoing emission of information ERROR SIGNAL = current reward +  next prediction - current prediction

9. Can we detect a reward prediction error signal in a human subject?

10. Neural correlates of these error signals have been observed in conditioning and decision tasks Passive conditioning tasks Instrumental conditioning tasks Sequential decision tasks Social and economic exchange tasks

11. Midbrain dopamine neurons burst and pause responses encode reward prediction errors ERROR SIGNAL = current reward +  next prediction - current prediction Can these systems be re-deployed for abstractly defined rewards (ideas)? “I want to solve Fermat’s last theorem”

12. What about the something more abstract like the expression and repayment of trust?

13. Trust Modeling Must involve risk (uncertainty) Harvesting returns from another agent

14. TRUST I love you Let’s do lunch B A A B A B

15. Simplifying and quantifying Trust (Berg et al., 1995; Weigelt and Camerer, 1988) Trust is the amount of money a sender sends to a receiver without external enforcement. Agent 2Agent 1 I R

16. X 3X 3 InvestorTrustee $20 A dynamic version of the Trust game (10 rounds)

17. Structure of a round

18. What is the behavioral signal that most strongly influences changes in trust (money sent) ? Agent 2Agent 1 I R Reciprocity = TIT-FOR-TAT

19. money sent to partner + Benevolent signal - Malevolent signal Neutral signal

20. Questions about brain response Responses that differentiate benevolent from neutral? Responses that differentiate malevolent from neutral? Responses that differentiate benevolent from malevolent? Surprise response

21. * time (sec) -8010 -0.2 0 0.2 Trustee Brain: ‘intention to increase trust’ shifts with reputation building Reputation develops Trustee will increase trust on next move Trustee will decrease trust on next move submitreveal * increases or decreases in future trust by trustee -0.2 0 0.2 time (sec) Signal is now anticipating the outcome

22. Temporal shift resembles value transfer in reward learning experiments burst pause R time naive R time After learning time After learning (catch trial)

23. -.9 -.8 -.7 -.6 -.5 -.4 -.3 -.2 -.1 0 -.8-.6-.4-.20.2.4.6.81 r =.00 TRUSTEE decreases repayment Trustee decreases  No information 0.1.2.3.4.5.6.7.8.9 1 -.8-.6-.4-.20.2.4.6.81 r =.27 Change in next investment TRUSTEE increases repayment Trustee decreases  positive correlation Why should intentions to increase repayment burn more energy? Change in next investment

24. 1.Social trust is about modeling others - always some underlying currency. 2.Re-deploy reward-harvesting machinery that we share with all vertebrates (abstractions gain reward status) 3.Use to probe pathologies (addicted state, autism spectrum disorder, borderline…) Trust?

25. Baylor College of Medicine Pearl Chiu Amin Kayali Brooks King-Casas Terry Lohrenz Sam McClure (Princeton) Read Montague Damon Tomlin Caltech Cedric Anen Colin Camerer Steve Quartz UCL Peter Dayan Nathaniel Daw Salk Institute Terry Sejnowski Princeton University Jon Cohen www.hnl.bcm.tmc.edu/trust Emory University Greg Berns University of Alabama Laura Klinger Mark Klinger Families of autistic subjects Funding Sources NIDA NIMH Dana Foundation Kane Family Foundation Angel Williamson Imaging Center Institute for Advanced Study, Princeton NJ Collaborators

26. rounds.1.2.3.4 345678910 fraction of highly accurate guesses by trustee Investor events Trustee events How do we know a reputation is forming? 8 s10 s4 s... invest period cue to invest investment revealed to both brains delay period guess period cue to guess Investment phase synchronized on submission time of both partners Trustee

27. What is Fair? i i i Split the profit Split ‘loaf’ Split the total 10 + i each i Investor Trustee 20 -i collective ownership? common goods?