1. Connectionist Models of Development Jeff Elman Department of Cognitive Science UCSD

2. Today’s class: – What biology can do – What learning can do The issue: Nature vs. Nurture

3. Two possible ways to control development...

7. More DNALess DNA

11. pyramidal cells mossy cells muscle cells sperm cells Purkinje cell

14. Genetic conservatism

17. from butterfly hostfrom alderfly host Trichogramma (wasp)

18. from butterfly hostfrom alderfly host Trichogramma (wasp)

19. Lesson 1: no “special purpose” genes Lesson 2: genetic conservatism Lesson 3: change through “tweaks & twiddles” Lesson 4: the importance of the environment

20. Modeling learning with “neural networks”

21. The first computers...

22. Walter Pitts

23.  e1e1 enen inin i1i1

24. “Every digital computing device contains certain relay like elements with discrete equilibria. Such an element has two or more distinct states in which it can exist indefinitely...It is worth mentioning that the neurons of the higher animals are definitely elements in the above sense... Following W. Pitts and W.S. McCulloch...we ignore the more complicated aspects of neuron functioning.” John von Neumann

25. The Colossus (one of ten ) Bletchley Park, 1944

26. Is the human brain a computer?

27. CPU and memory are separate the processor is very fast serial operation very reliable parts; but the whole computer may break easily computers are programmed the memory and CPU are the same the parts (neurons) are very slow parallel; enormous interconnectivity unreliable parts; but the whole brain is very reliable brains learn

28. Things digital computers do well Things human brains do well Numeric computation Symbolic computation Repetitive operations Follow instructions Find subtle patterns (intuition?) Find novel solutions by analogy Learn

29. ELIZA and the problem of natural language processing the problem with SCRIPTS

30. Eliza Eliza:Hello. I am Eliza. How can I help you? Me:You can’t help me at all. Eliza: You like to think I can’t help you, do you not? Me:I am sure of it. Eliza: How long have you been sure of it? Me:Ever since I was 10 years old. Eliza: Why do you tell me you were 10 years old just now? Me:You are not a real person Eliza: Does it please you to believe I am not a real person?

31. Scripts, Schemata, and Frames John went to McDonald's with Mary. They ordered a hamburger and fries. John gave the woman at the counter $3.49. They left. What is McDonalds? Why did John and Mary go there? Why did John give the woman money? Did they eat anything?

32. John went to McDonald's restaurant with Mary. They ordered a hamburger and fries. Suddenly, Mary's husband Lenny burst in with a shotgun. John hid under a table. Why did John and Mary go to McDonalds? Why did John hide under the table? Did John get to eat his Big Mac? What were they doing before going to McDonalds? Where will Mary’s husband spend the rest of his life?

33. fragile (vs. graceful degradation) –fixed repertoire –limited to well-defined domains how to get the knowledge in the system

36. What is a neural network?

38. Learning vs. Programming

39. 13 ? Input Output 2424 749 416

40. fever, runny nose, muscle ache->flu fever, runny nose-> ? Input Output no fever, runny nose->allergies no fever, skin rash-> staphylococcus infection

41. “ought”-> “ouch”-> “tough”-> “through”-> “though”-> “plouty”-> ? “plough”-> ? How do you pronounce “ou”? “aw” “au” “uh” “oo” “oh”

42. “The Voringian binx glorphed the Knappoboor.”

43. 1. Learning to read out loud 2. Discovering where the words are 3. Discovering categories

44. My grandmother lives near us. I like to visit my grandmother.

45. [first stages of learning]

46. “I like to go to my grandmother’s house. Well…because she gives us candy. Well... and we eat there sometimes. Sometimes we sleep overnight there. Sometime when I got to go to my cousin’s...”

47. A (surprisingly) hard problem: Where are the words?

48. Whereareth es il ens es b et w eew or d sWhereareth es il enc es b et w eenw or d s

49. “Many years ago, a boy and girl lived in a castle by the sea. They played with a dragon….”

50. manyyearsagoaboyandgirllivedina castlebytheseatheyplayedwithadr agon 00011 10111 00010 11011 11011 00100 10111 01111 01000 10111 11000 10010 10111...

51. INPUT: OUTPUT: time Task: Predict the next input m a a n n y y y y e e a a r r s s a a g...

52. Time

53. Word learning statistical learning Saffran, Aslin, Newport, 1996 8 mo. old infants Passive exposure to 2 minutes of artificial nonsense language Then present “words” vs. “non-words” Infants listened more to novel “non-words”

54. pabikulatidorepabikutalikulatidopabikulilitalatidotupabiku

55. 0000000000000000000000000000010 0000000000000000000000000010000 0000000000000000000001000000000 0000010000000000000000000000000 0000000000000000000100000000000 0000000000000000100000000000000 0001000000000000000000000000000 0000100000000000000000000000000 0100000000000000000000000000000 0000000000000000000100000000000 0000000000001000000000000000000 0000000000100000000000000000000 0010000000000000000000000000000 0000000010000000000000000000000 0000000000000000000100000000000 0000000000000000001000000000000 0000000000000000000100000000000 1000000000000000000000000000000 0000000000000000000000000010000 0000000000000000000001000000000 0000010000000000000000000000000 0000000000000000000100000000000 0000000000000000100000000000000 0001000000000000000000000000000 0000100000000000000000000000000 0100000000000000000000000000000 0000000000000000000100000000000 0000000000001000000000000000000 0000000000100000000000000000000 0010000000000000000000000000000 0000000010000000000000000000000 0000000000000000000100000000000 0000000000000000001000000000000 0000000000000000000100000000000 1000000000000000000000000000000 time (woman) (smash) (plate) (cat) (move) (man) (break) (car) (boy) (move) (girl) (eat) (bread) (dog) (move) (mouse) (move) (book) (smash) (plate) (cat) (move) (man) (break) (car) (boy) (move) (girl) (eat) (bread) (dog) (move) (mouse) (move) (book) Input Output (prediction)

56. CURRENT WORD PREDICTED NEXT WORD

58. NOUNS VERBS DO absent DO optional DO obligatory small big edible breakable ANIMATES INANIMATES HUMANS ANIMALS

59. man woman child girl dog cat bird paper book touch be drink grab see think eat

60. The vocabulary burst

61. milk bottle bed chair kitty doggie cookie candy horse bird mommy daddy Jane go run see man drink Do you want to eat a MACAROON?

62. “Critical mass effect”: A critical number of words must be learned before categories, concepts, and relationships will become apparent. Once that number is learned…things take off.

63. So where does language come from? Nature? Nurture?

64. A new machine built out of old parts

65. bonobomacaquehumansong birdtermite Control over respiration Control over articulators Sequencing Memory Sociability Auditory processing Imitation Predictive learning Language is possible