Download presentation

Presentation is loading. Please wait.

1
Branch Prediction with Neural- Networks: Hidden Layers and Recurrent Connections Andrew Smith CSE Dept. June 10, 2004

2
Outline What is a Perceptron? –Learning? What is a Feed-Forward Network? –Learning? What is a Recurrent Network –Learning? How to do it on hardware??? Results – Adding hidden units Results – Modeling latency of slow networks. Results – Varying the hardware budget

3
The Perceptron Linear (affine) combination of inputs DECISION

4
Perceptron Learning inputs x j, outputs y i and targets t i are {-1, +1} Cycle through training set if X i = (x1, x2, …, xd) is misclassified, do w j w j + a * t i * x j end if

5
Feed-Forward Network A network of perceptrons…

6
Feed-forward Network Learning Use A gradient-descent algorithm. Network output is: Error is: Derivatives of error are:

7
Feed-Forward Networks, BACKPROP But no error defined for hidden units??? Solution, assign responsibility for output units error to each hidden unit, then descend gradient This is called “back-propagation”

8
Recurrent Networks Now it has state…

9
Learning weights for a RNN Unroll it and use back-propagation? No! Too Slow, and wrong…

10
Use Real-Time Recurrent Learning Keep list, at each time T: –For each Unit u –For each Weight w –Keep partial derivative du/dw Update with recurrence relation:

11
But on hardware??? Idea, represent real numbers in [-4, +4] with integers in [-4096..4096] Adding, is ok… –1024 i + 1024 j = (i+j)1024 Multiplying requires a divide (shift): –(1024 i) * (1024 j) = (i*j)1024^2 Compute activation function by looking up in a discretized table.

12
Results, different numbers of hidden units

13
Results, Different latencies

14
Results, different HW budget (crafty)

15
Results, Different HW budges (BZIP-PROGRAM)

16
Conclusions DON’T use a RNN! Maybe use a NNet with a few hidden units, but don’t over do it Future work: explore trade-off between –Number, size (hidden units), inputs

Similar presentations

© 2021 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google