1. Applications of Neural Networks in Biology and Agriculture
Jianming Yu
Department of Agronomy and Plant Genetics

2. Introduction to Neural Networks
Applications of Neural Networks in Biology and Agriculture

3. Boy
Girl

4. How Can We Recognize a Given Schematic Face?
It is a Boy? Or a Girl?

5. That Is What A
Neural Network
Is All About!

6. INFORMATION
INFORMATION
UNDERSTANDING
INFORMATION
INFORMATION
INFORMATION
UNDERSTANDING
INFORMATION
INFORMATION
INFORMATION
UNDERSTANDING
INFORMATION

7. Girls Boys +1,+1,+1,-1,-1 +1,-1,+1,+1,+1 -1,+1,-1,+1,+1 -1,-1,+1,-1,-1
-1,+1,-1,+1,-1
+1,-1,+1,-1,-1

8. Introduction of Neural Network
Structure of a Neuron/Node
Analogy of Neural Networks
Definition
Architecture
Learning Process
Constructing a Neural Network

9. Dendrites: Provides input to the neural
origin: in efforts to produce a computer model of the information processing that takes place in the nervous system, such as brain.
Dendrites: Provides input to the neural
Axon: Send neuron out put to other neuron
Neurons: combine signals from many dendrites.
If signal is strong enough, the neuron fires and a signal sent along the neuron’s axons
Synapses: connect axons of one neuron to the dendrites of others.
Reinforces or Weakens the strength of the signal passed on.
Learning involves modifying the synapses.

10. Biological vs. Artificial
Human Brain Neural Network
Neurons Nodes
Dendrites Inputs/Sensors
Axons Outputs
Synapses Weights
Information procession
Learning by examples
Generalize beyond examples

11. What is a Neural Networks?
Hidden Layer
weights
Weights express the relative strength or importance of an input (in mathematical terms) in determining the value of an output
Finds weighted sum of all input elements to each processing element
Output Layer
Input Layer

12. Architecture of Neural Networks
Perceptron
Feed-forward
Feedback
Feedforward:
Nodes do not connect to nodes in the current or previous layers
Transformations & combinatorial effects produce non-linear results.
Feedback/Recurrent:
Nodes may connect to nodes in the current or previous layers
Provide a temporal component to the network as past outputs feedback as current inputs.
In principle, neural networks are universal approximators and can compute any computable function. In practice, neural networks are especially useful for classification and function approximation/mapping problems that have plenty of training data available and can tolerate some imprecision but that resist the easy application of hard and fast rules.
A Neural Network method is often quoted as a Data-Driven method. The weights are adjusted on the basis of data. In other words, neural networks learn from training examples and can generalize beyond the training data. Therefore, neural networks are often applied to domains where one has little or incomplete understanding of the problem to be solved. but where training data is readily available. As a data modeling method, neural networks can combine the multiple features or rules, and fit complex nonlinear models. Furthermore, unlike other nonlinear statistical modeling techniques, these models do not have to be specified in advance; thus one can avoid making assumptions that may not be correct or relevant for the biological domain. As a useful adjunt to other statistical and mathematical models, neural networks will continue to play important roles in life science, where complex biological knowledge cannot be easily modeled, and will help us understand and answer fundamental biological questions.
What if the task involves classifying data dissimilar to any of the training cases? To detect novel cases for which the classification may not be reliable, one can apply a classification method based on probability density estimation, such as a probabilistic neural network.

13. Learning Process Supervised learning Back-propagation algorithm
Least Mean Square Convergence
If output too small, + / - unit weights
If output too large, - / + unit weights
Supervised learning (e.g. back propagation)
incorporates an external teacher, so that each output unit is told what its desired response to input signals ought to be.
network presented with both input data AND desired output.
global minimization. Least mean square convergence.
Minimization of error between the desired and computed unit values
Unsupervised learning.
uses no external teacher and is based upon only local information. it is also referred as self organization, in the sense that it self-orgnises data presented to the network and detects their emergent collective properties.
network presented with only the input data, and NOT the the desired output.
Reinforcement Learning
Network is told simply whether output is good or bad but is not given desired output.
Weights initialized with very small random values. (Why?)
Adjust weights by minimizing D = Z - YT : reducing difference between actual and desired outputs
If output too small ­ +ve weights, ¯ -ve weights
If output too large ¯ +ve weights, ­ -ve weights
Stopping rule: 0
(what is approximately zero?)
Why “Back Propagation”?
Errors between the desired output and the network output are propagated back through the network to adjust the weights.
Supervised learning:
Back Propagation or “Delta Rule Learning”.
Hebbian Learning: input weights strengthened if both the input is high and the desired output is high.
Unsupervised Learning:
Competitive learning: nodes compete with each other, the one with strongest response to a given input modifies itself to become more like that input.

14. Knowledge/ Weight Matrix
Network Output
Known Output
-1.00
W1 = 0.3
-0.25
W2 = 0.2
-0.50
-0.45
0.50
W3 = 0.5
W4 = - 0.5
-1.00
Knowledge/ Weight Matrix
-1.00
W1 = 0.2
0.2
0.4
-0.5
-0.25
W2 = 0.2
-0.45
-0.45
0.50
W3 = 0.4
W4 = - 0.5
-1.00

15. Learning Process
Supervised learning
Unsupervised learning

16. Constructing a Supervised Neural Network
Determine architecture
Set learning parameters & initialize weights.
Code the data
Train the network
Evaluate performance

17. Applications of Neural Networks
General Information
Search for a gene
Gene expression network
Kernel number prediction

18. Applications of Neural Network
Pattern classification
Clustering
Forecasting and prediction
Nonlinear system modeling
Speech synthesis and recognition / Function approximation / Image compression / Combinational optimization

19. Business
Forecast the maximum return configuration of a stock portfolio
Credit risk analysis
Forecasting airline passenger booking

20. You have diabetes. Please …
Medicine
Diagnosing the cardiovascular system
Electronic noses
Instant Physician
Dr. Computer:
You have diabetes. Please …

21. Example 1 Coding Region Recognition and Gene Identification
ATGCATATCGCACTATAGCCGCCCCGACATAGCCGCAAAT

22. Sensors Training Data Validating Data Objective
Example 1
Sensors
Codon usage, base composition, periodicity, splice site, coding 6-tuples, etc.
Training Data
Known genes
Validating Data
Objective
Find genes in unannotated sequence

23. Example 1
Uberbacher, E. C. et al., Locating protein-coding regions in human DNA sequences by a mulitple sensor-neural network approach. PNAS. 88,
Discrete exon score
Sensors
The use of neural networks to recognize features in DNA sequence has been pioneered by Lapedes and colleagues at Los Alamos National Laboratory. The problem associated with their work was that they did not deal with overlap edges of coding and non-coding regions. And the system yielded a high false positive. A low false positive rate, in statistics they call the power of a certain statistic, is important, for locating unknown genes in anonymous DNA sequences, since the analysis of a predicted coding region can be very time-consuming.
It configuration the ‘coding recognition module’ identifies 90% of coding exons of length 100 bases or greater with less than one false positive coding exon indicated per five coding exons.

24. Human, Mouse, Arabidopsis, Drosophilae, E.coli
Example 1
Human, Mouse, Arabidopsis, Drosophilae, E.coli
GRAIL / GRAIL EXP

25. Example 1
“With modest effort, an investigator can greatly enrich the value of the sequence under study by including descriptions of the genes, proteins, and regulatory regions that are present. Such analysis will provide a starting point to this most exciting phase of genome research”
--Uberbacher, 1996
Grail
--It recognizes simple repeats, polyas, promoters, exon candidates, genes, and complex repetitive elements.
Grail EXP
--a Grail-like exon finder
--the gene message alignment program and the gene assembly program)

26. Example 2 Gene Expression Networks
Off On On

27. Sensors Training Data Validating Data Objectives
Example 2
Sensors
Temperature, Day length
Training Data
Experiment
Validating Data
Molecular Maker, Microarray, Experiment
Objectives
Simulate the gene expression network
Test the developmental gene hierarchy

28. Example 2
Welch, S. M., et al Modeling the Genetic Control of Flowering in Arabidopsis thalina. J. of Agro.
Arabidopsis thalina wildtype
Temp
Flower
The idea was to test whether we can model the inflorescence transition control in Arabidopsis thaliana using a neural network.
Day length

29. Temp Flower Day length Example 2
The idea was to test whether we can model the inflorescence transition control in Arabidopsis thaliana using a neural network.

30. Example 2
light/dark
16 oC
20 oC
24 oC
8/16 hr
Run 4
Run 6
Run 3
16/ 8 hr
Run 2
Run 5
Run 1

31. The fits of the other five genotypes are equally good
The fits of the other five genotypes are equally good. The three shown were selected for display because of the unusual switch in the order of FVE and CO. This is difficult to reconcile with the standard thermal integration methods used in many plant models.
To assess whether our good fits were only due to the approximation power of neural networks, we measured the goodness-of-fit of 300 random networks to the 24°C data. The random networks all had the same number of nodes, the same connectivity into each node, and the same patterns of coefficients.

32. GxE less readily mimicked by existing models
Example 2
Neural networks can be employed to model the genetic control of plant process
Nodes can be linked in one-one correspondence with networks constructed by genomic techniques
Complex phenotypic behavior can be related to internal network characteristics
GxE less readily mimicked by existing models
To assess whether our good fits were only due to the approximation power of neural networks, we measured the goodness-of-fit of 300 random networks to the 24°C data. The random networks all had the same number of nodes, the same connectivity into each node, and the same patterns of coefficients. The results are shown in Fig. 7. The best random network fit is about 10 times worse than that of the gene-based network indicating a sensitivity to structure.

33. Example 3 Kernel Number Prediction

34. Sensors Training Data Validating Data Objective Experiment
Example 3
Sensors
Biomass: Total biomass produced during the critical period of ear elongation
Population: Plant Population
Training Data
Experiment
Validating Data
Objective
Predict the Kernel Number

35. Biomass Kernel Number Population
Example 3
Dong, Zhanshan, et al A Neural Network Model for Kernel Number of Corn -- Training and Representing in STELLA.
Corn Field
Biomass
Kernel Number
STELLA is a program that can create dynamic simulation model
Population

36. Example 3
STELLA is a program that can create dynamic simulation model

37. Corn Cultivar : Medium maturity Automatic irrigation Data for training
Example 3
Corn Cultivar : Medium maturity
Automatic irrigation
Data for training
10 years in Manhattan, KS ( )
5 years in Parsons, KS ( )
Data for validation
2 years in Manhattan, KS ( )

38. Validation Data Example 3 * Actual KN  Predicted KN Kernel number
800
600
Kernel number
400
120
200
100 12 80 10
Biomass 8 6 60 4
Plant Population

39. Predicted KN vs. Actual KN
Example 3
Predicted KN vs. Actual KN
700
650
KNPred= *KNActual
600
550
500
Predicted KN
450
400
350
* Validation data
o Training data
300
250
200
200
300
400
500
600
700
Actual KN

40. STELLA can represent the neural network easily and efficiently
Example 3
Training a neural network that can effectively simulate kernel number of corn needs a wild range of data set
Neural network can simulate kernel number of corn by using total biomass produced during the critical period of ear elongation and plant population
STELLA can represent the neural network easily and efficiently

41. Strengths Knowledge not needed (?) Can handle complex problems
Fairly fast run time looks at all the information at once
Can deal with noisy and incomplete data (?)
Adaptability over time, continuous learning.
Data can be numeric, qualitative, T/F, pass/fail, up/down, yes/no, etc.
But must be numerically coded.
Weights express the relative strength or importance of an input (in mathematical terms) in determining the value of an output
Adjustment of weights is a neural network’s learning mechanism
Because of combinatorial effects weights are not readily interpretable

42. Weaknesses Cannot separate correlation and causality
No explanation or justification facilities
Weights don’t have obvious interpretations (?)
No confidence intervals (?)
Requires lots of data and training time
Data can be numeric, qualitative, T/F, pass/fail, up/down, yes/no, etc.
But must be numerically coded.
Weights express the relative strength or importance of an input (in mathematical terms) in determining the value of an output
Adjustment of weights is a neural network’s learning mechanism
Because of combinatorial effects weights are not readily interpretable

43. If you want to know more about Neural Network,
Neural Networks, by Herve Abdi, et al., 1999.
Neural Networks and Genome Informatics, by C. H. Wu, and McLarty, J. W., 2000

44. Acknowledgement Dr. Rex Bernardo Dr. Nevin Young
Dr. JoAnna Lamb, Jimmy Byun, Bill Peters, Marcelo Pacheco, Bill Wingbermuehle, Luis Moreno-Alvarado, Ebandro Uscanga-Mortera
APS and Agronomy Dept.
Dr. S. M. Welch, Zhanshan Dong, and Yuwen Zhang. (K-State)

45. Research & Interest of Neural Network?
“Neural networks do not perform miracles. But if used sensibly they can produce some amazing results.”
-- C. Stergiou and D. Siganos
Time
1940
1960
1969
1970
1980
1990
2000