1. Encoding Sensory Inputs in Numeric / Binary Form n CS/PY 399 Lecture Presentation # 18 n February 19, 2001 n Mount Union College

2. The Problem n So far, we have seen networks that can learn to produce numeric output patterns in response to a sequence of numeric inputs n What do organisms do? –produce thoughts and behaviors in response to sensory inputs n We need a way to represent real-world data in numeric form

3. Example: Sequence of Letters n What if we wanted to train a neural network to respond to a letter of the alphabet with the letter that follows the input letter? n e.g., C => D, U => V n Simple task, yet our TLearn networks need to have numeric data to process n What prior art can we use?

4. Computer Hardware example n Computer Memory is a collection of a large number of electronic circuits n Each circuit has two states: on or off –number each state: 1 or 0 n A single circuit can represent a 2-valued datum –yes/no, true/false n what to do for data items that need more than 2 possible values?

5. Binary #s  Group of Circuits n Consider two circuits as a single group –00 01 10 11 n 2 choices for first bit x 2 choices for second bit = 4 possible values –yes, no, maybe, huh? n still not enough for letters of the alphabet, digits, etc.

6. Binary #s  Group of Circuits n Consider three circuits as a single group –000 001 010 011 –100 101 110 111 n Twice as many possible patterns as with 2 bits –2 x 4 = 8 = 2 3 n 4 bits: 2 4 = 16 patterns; 5 bits: 2 5 = 32 patterns; n n bits: 2 n patterns

7. Example from Lab # 6 n We trained a network with 3 inputs and 2 outputs –Example from Lab # 6Example from Lab # 6 n Answer: output is the number of zeros in the input pattern, expressed as a binary number! n But how could you know that? We also need to decode numeric output into a form that humans can use….

8. Question for Understanding n How many binary input and output signals would be needed to train a network to discriminate thusly: given an American League baseball team, ranking the Indians as great, the Yankees, White Sox, Red Sox and Orioles as lousy, and all others as mediocre? n 15 teams, 3 possible rankings

9. Equivalence of Binary and Decimal Encodings n It turns out that we can use a single decimal value for one input, instead of a series of bits for the same value –the two encodings are equivalent n Example: Give each team a unique number n Output: 1 = great, 2 = lousy, 3 = mediocre n 1 input and 1 output, simpler network

10. Where does encoding/decoding happen? n In our computational model, sense organs convert analog data into digital signals that can be operated on by a neural network n To model this, we have two options: n Build hardware components that perform this conversion for us [:o (] n Convert data into digital form, then present this to the network [:o)]

11. Where does encoding/decoding happen? n Output of our model is a bunch of numeric signals. n We need to convert this into understandable information n Two approaches: n Build hardware to do the conversion (robot lab!!!) n Convert data ourselves (either by writing a program, or doing it manually)

12. Example: Diagnosing Illness n Input parameters: –temperature: numeric value –pulse: numeric value –palpating abdomen: hard, squishy, normal –headache? yes or no –how many fingers? too few, too many, right number n Output diagnoses: –ebola, migrane, hypochondria, normal health

13. Example: Diagnosing Illness n Input parameters: –numeric value –1 = hard, -1 = squishy, 0 = normal –1 = headache, 0 = no headache –-1 = too few fingers, 1 = too many, 0 = right number n Output diagnoses: –some numeric code for each disease

14. Central Problem in Compuational Neural Networks n Without a correct and proper encoding scheme, the network won’t produce meaningful answers n A large, important part of your term project will be selecting a data representation for the problem you choose –more on that on Wednesday

15. Encoding Sensory Inputs in Numeric / Binary Form n CS/PY 399 Lecture Presentation # 18 n February 19, 2001 n Mount Union College