NEURAL NETWORK By : Farideddin Behzad Supervisor : Dr. Saffar Avval May 2006 Amirkabir University of Technology

2 Agenda Definition Definition Application fields Application fields History History Application Application Biological inspiration Biological inspiration Mathematical model Mathematical model Basic definition Basic definition Learning Learning Neuron types and some issues Neuron types and some issues Example of application in energy & engineering Example of application in energy & engineering

3 Definition Haykin(1999) massive parallel-distributed processor massive parallel-distributed processor natural propensity for storing experiential knowledge natural propensity for storing experiential knowledge available for use. available for use. Acquiring knowledge by the network from its environment through a learning process Acquiring knowledge by the network from its environment through a learning process Using interneuron connection strengths, (a.k.a. synaptic weights), to store the acquired knowledge Using interneuron connection strengths, (a.k.a. synaptic weights), to store the acquired knowledge

4 Application fields Data analysis Data analysis Pattern recognition Pattern recognition Control application Control application

5 History 1943, Warren McCulloch & Walter Pitts, works of neurons 1960, Bernard Widrow & Marcian Hoff, developed ADALINE and MADLINE From late 1960s to 1981, decreasing of researches From late 1960s to 1981, decreasing of researches Early 1980s, renewed interest in neural network Early 1980s, renewed interest in neural network 1986, Daivid Rummelhart & James McLand, error back- propagation algorithm 1986, Daivid Rummelhart & James McLand, error back- propagation algorithm

6 Applications Aerospace industry Aerospace industry Automotive industry Automotive industry Banking Banking Military industry Military industry Economics Economics Manufacturing Manufacturing Medical applications Medical applications Oil & petroleum industry Oil & petroleum industry And many more … And many more …

7 Biological inspiration Brain structure Brain structure Cell Cell Cell body Cell body Axon Axon Denderites Denderites Dendrites Soma (cell body) Axon

8 Mathematical model Inputs Output w2w2 w1w1 w3w3 wnwn w n-1. x 1 x 2 x 3 … x n-1 x n y Node Artificial neural cell

9 Mathematical model input Cell body Mathematic model of artificial neural cell output

10 Basic definition Architecture: formal mathematical description of a Neural Network. (feed-forward & feed-back) Architecture: formal mathematical description of a Neural Network. (feed-forward & feed-back) Layer or Slab: A subset of neurons in a neural network. (Input, Hidden, Output) Layer or Slab: A subset of neurons in a neural network. (Input, Hidden, Output) Episodical vs continuous networks Episodical vs continuous networks Neuron weight Neuron weight Activation function

11 Activation function Linear Non-Linear Step Sigmoid Linear Gaussian

12 Learning Coincidence learning Coincidence learning Performance learning Performance learning Competitive learning Competitive learning Filter learning Filter learning Spatiotemporal learning Spatiotemporal learning learning Supervised learning Unsupervised learning

13 Neuron types Hebb Hebb Perceptron Perceptron Adaline Adaline Kohonen Kohonen

14 Some issues Training dataset Training dataset Test dataset Test dataset Network size Network size

15 Example of application in energy Soleimani. M, Thomas. B, Per Fahlen, “Estimation operative temperature of building using artificial neural network”, Journal of Energy and Building 38,2006 Soleimani. M, Thomas. B, Per Fahlen, “Estimation operative temperature of building using artificial neural network”, Journal of Energy and Building 38,2006 Luis M. Romeo, Raquel Gareta, “ neural network for evaluating boiler behaviour”, Applied Thermal Engineering 26, 2006 Luis M. Romeo, Raquel Gareta, “ neural network for evaluating boiler behaviour”, Applied Thermal Engineering 26, 2006 Seyedan B., Ching C.Y., “Sensitivity analysis of freestream turbulence parameter on stagnation region heat transfer using a neural network”, International Journal of Heat and Fluid Flow, 2006 Seyedan B., Ching C.Y., “Sensitivity analysis of freestream turbulence parameter on stagnation region heat transfer using a neural network”, International Journal of Heat and Fluid Flow, 2006 Perez-roa P., Vesovic V., “Air-pollution modelling in an urban area: Correlation turbulent diffusion coefficients by means of an artifical neral network approach”, Atmospheric Environment 40, 2006 Perez-roa P., Vesovic V., “Air-pollution modelling in an urban area: Correlation turbulent diffusion coefficients by means of an artifical neral network approach”, Atmospheric Environment 40, 2006

16 References 1. منهاج محمد باقر، ” مباني شبكه هاي عصبي “ ، انتشارات دانشگاه صنعتي اميركبير، پاييز Hecht-Nielsen R., “ Neurocomputing “, Addison-Wesley publishing company, MATLAB help documentation