Presentation is loading. Please wait.

Presentation is loading. Please wait.

Recurrent Neural Networks

Published byAndrea Hicks Modified over 5 years ago

Similar presentations

Presentation on theme: "Recurrent Neural Networks"β€” Presentation transcript:

1 Recurrent Neural Networks

2 Today Recurrent Neural Network Cell
Recurrent Neural Networks (unenrolled) LSTMs, Bi-LSTMs, Stacked Bi-LSTMs

3 Recurrent Neural Network Cell
𝑅𝑁𝑁 β„Ž 0 β„Ž 1 π‘₯ 1

4 Recurrent Neural Network Cell
β„Ž 1 =tanh⁑( π‘Š β„Žβ„Ž β„Ž 0 + π‘Š β„Žπ‘₯ π‘₯ 1 ) 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 π‘₯ 1

5 Recurrent Neural Network Cell
𝑦 1 β„Ž 1 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 β„Ž 1 =tanh⁑( π‘Š β„Žβ„Ž β„Ž 0 + π‘Š β„Žπ‘₯ π‘₯ 1 ) π‘₯ 1 𝑦 1 =softmax⁑( π‘Š β„Žπ‘¦ β„Ž 1 )

6 Recurrent Neural Network Cell
𝑦 1 β„Ž 1 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 π‘₯ 1

7 Recurrent Neural Network Cell
𝑦 1 =[0.1, 0.05, 0.05, 0.1, 0.7] β„Ž 1 =[ ] 𝑅𝑁𝑁 β„Ž 0 =[ ] π‘₯ 1 = [ ] a b c d e c

8 Generating Samples from the Recurrent Neural Network Cell

9 LSTM Cell (Long Short-Term Memory)
β„Ž 0 β„Ž 1 𝐿𝑆𝑇𝑀 𝑐 0 𝑐 1 π‘₯ 1

10

11 Recurrent Neural Network Cell
𝑦 1 β„Ž 1 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 π‘₯ 1

12 Recurrent Neural Network Cell
β„Ž 1 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 π‘₯ 1

13 (Unrolled) Recurrent Neural Network
<<space>> 𝑦 1 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 2 π‘₯ 3 c a t

14 (Unrolled) Recurrent Neural Network
cat likes eating 𝑦 1 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 1 π‘₯ 1 the cat likes

15 (Unrolled) Recurrent Neural Network
positive / negative sentiment rating 𝑦 1 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 1 π‘₯ 1 the cat likes

16 (Unrolled) Recurrent Neural Network
π‘₯ 1 𝑅𝑁𝑁 β„Ž 0 β„Ž 1 β„Ž 2 β„Ž 3 c a t <<space>> 𝑦 1

17

18

19

20 Bidirectional Recurrent Neural Network
gato quiere comer 𝑦 1 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝐡𝑅𝑁𝑁 β„Ž 1 B𝑅𝑁𝑁 β„Ž 2 𝐡𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 1 π‘₯ 1 the cat wants

21 Stacked Recurrent Neural Network
𝑦 1 𝑦 1 𝑦 1 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 1 π‘₯ 1 c a t

22 Bidirectional Stacked Recurrent Neural Network
𝑦 1 𝑦 1 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 β„Ž 1 β„Ž 2 β„Ž 3 β„Ž 0 𝑅𝑁𝑁 β„Ž 1 𝑅𝑁𝑁 β„Ž 2 𝑅𝑁𝑁 β„Ž 3 π‘₯ 1 π‘₯ 1 𝑦 1 π‘₯ 1 c a t

23

24 Questions?

Download ppt "Recurrent Neural Networks"

Similar presentations

Introduction to Neural Networks 2. Overview  The McCulloch-Pitts neuron  Pattern space  Limitations  Learning.

Introduction to Neural Networks 2. Overview  The McCulloch-Pitts neuron  Pattern space  Limitations  Learning.
Introduction to Recurrent neural networks (RNN), Long short-term memory (LSTM) Wenjie Pei In this coffee talk, I would like to present you some basic.

Introduction to Recurrent neural networks (RNN), Long short-term memory (LSTM) Wenjie Pei In this coffee talk, I would like to present you some basic.
Deep Learning Neural Network with Memory (1)

Deep Learning Neural Network with Memory (1)
Convolutional LSTM Networks for Subcellular Localization of Proteins

Convolutional LSTM Networks for Subcellular Localization of Proteins
Predicting the dropouts rate of online course using LSTM method

Predicting the dropouts rate of online course using LSTM method
NOTE: To change the image on this slide, select the picture and delete it. Then click the Pictures icon in the placeholder to insert your own image. SHOW.

NOTE: To change the image on this slide, select the picture and delete it. Then click the Pictures icon in the placeholder to insert your own image. SHOW.
Learning to Answer Questions from Image Using Convolutional Neural Network Lin Ma, Zhengdong Lu, and Hang Li Huawei Noah’s Ark Lab, Hong Kong http://www.ee.cuhk.edu.hk/~lma/

Learning to Answer Questions from Image Using Convolutional Neural Network Lin Ma, Zhengdong Lu, and Hang Li Huawei Noah’s Ark Lab, Hong Kong
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.
Audio-Based Multimedia Event Detection Using Deep Recurrent Neural Networks Yun Wang, Leonardo Neves, Florian Metze 3/23/2016.

Audio-Based Multimedia Event Detection Using Deep Recurrent Neural Networks Yun Wang, Leonardo Neves, Florian Metze 3/23/2016.
Lecture 6 Smaller Network: RNN

Lecture 6 Smaller Network: RNN
Deep Learning RUSSIR 2017 – Day 3

Deep Learning RUSSIR 2017 – Day 3
Hierarchical Question-Image Co-Attention for Visual Question Answering

Hierarchical Question-Image Co-Attention for Visual Question Answering
End-To-End Memory Networks

End-To-End Memory Networks
CS 388: Natural Language Processing: LSTM Recurrent Neural Networks

CS 388: Natural Language Processing: LSTM Recurrent Neural Networks
CS 4501: Introduction to Computer Vision Computer Vision + Natural Language Connelly Barnes Some slides from Fei-Fei Li / Andrej Karpathy / Justin Johnson.

CS 4501: Introduction to Computer Vision Computer Vision + Natural Language Connelly Barnes Some slides from Fei-Fei Li / Andrej Karpathy / Justin Johnson.
Recurrent Neural Networks

Recurrent Neural Networks
Jie Wu1, Dongyan Huang2, Lei Xie1 and Haizhou Li2,3

Jie Wu1, Dongyan Huang2, Lei Xie1 and Haizhou Li2,3
Recursive Neural Networks

Recursive Neural Networks
Recurrent Neural Networks for Natural Language Processing

Recurrent Neural Networks for Natural Language Processing
Show and Tell: A Neural Image Caption Generator (CVPR 2015)

Show and Tell: A Neural Image Caption Generator (CVPR 2015)

Similar presentations

Ads by Google