Presentation is loading. Please wait.

Presentation is loading. Please wait.

A machine learning perspective on neural networks and learning tools

Published byReyna Neale Modified over 10 years ago

Similar presentations

Presentation on theme: "A machine learning perspective on neural networks and learning tools"— Presentation transcript:

1 A machine learning perspective on neural networks and learning tools
Tom Schaul

2 Overview PyBrain: training artificial neural networks for classification, (sequence) prediction and control Neural networks Modular structure Available architectures Training Supervised learning Optimization Reinforcement learning (RL) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

3 Disclaimer Only version 0.3, you may encounter But growing
inconsistencies bugs undocumented “features” But growing 10+ contributors 100+ followers (github, mailing list) 1000+ downloads 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

4 (Our) Neural Networks No spikes Continuous activations
Discrete time steps 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

5 Network Structure: Modules
input error output error Derivatives derivatives output Parameters parameters input 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

6 Network Structure: Connections
Module input output input error output error FullConnection Module input output input error output error Module input output input error output error 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

7 Network Structure: Graphs, Recurrency, Nesting
Module 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

8 Network Components: Modules
Module types layers of neurons additive or multiplicative sigmoidal squashing functions stochastic outputs gate units memory cells (e.g. LSTM cells) … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

9 Network Components: Connections
Fully connected or sparse Time-recurrent Weight-sharing may contain parameters … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

10 Network Architectures
Feed-forward networks, including Deep Belief Nets Restricted Boltzmann Machines (RBM) Recurrent networks, including Reservoirs (Echo State networks) Bidirectional networks Long Short-Term Memory (LSTM) architectures Multi-Dimensional Recurrent Networks (MDRNN) Custom-designed topologies 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

11 Overview Neural networks Training Modular structure
Available architectures Training Supervised learning Optimization Reinforcement learning (RL) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

12 Training: Supervised Learning
compare to target Module input output Parameters parameters input error output error Derivatives derivatives Backpropagation gradient update on parameters 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

13 Training: Black-box Optimization
fitness function based on e.g. MSE, accuracy, rewards multiple fitness values: multi-objective optimization BlackBoxOptimizer Parameters parameters Black box update parameters fitness 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

14 Optimization Algorithms
(Stochastic) Hill-climbing Particle Swarm Optimization (PSO) (Natural) Evolution Strategies (ES) Covariance Matrix Adaptation (CMA) Genetic Algorithms (GA) Co-evolution Multi-Objective Optimization (NSGA-II) … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

15 Training: Reinforcement Learning
Experiment Environment state action Task Environment Agent action observation reward 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

16 RL: Agents, Learners, Exploration
action reward observation Explorer Module DataSet Learner LearningAgent 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

17 RL: Learning Algorithms and Exploration
Value-based RL Q-Learning, SARSA Fitted-Q Iteration Policy Gradient RL REINFORCE Natural Actor-Critic Exploration methods Epsilon-Greedy Boltzmann State-Dependent Exploration 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

18 RL: Environments and Tasks
2D Mazes (MDP / POMDP) Pole balancing 3D environments (ODE, FlexCube) Board games (e.g. Atari-Go, Pente) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

19 Also in PyBrain Unsupervised learning and preprocessing
Support Vector Machines (through LIBSVM) Tools Plotting / Visualization netCDF support XML read/write support arac: fast C version 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

20 References Source download, documentation Mailing list (200+ members)
Mailing list (200+ members) groups.google.com/group/pybrain Feature requests github.com/pybrain/pybrain/issues Citation T. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010. 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

21 Acknowledgements Justin Bayer Martin Felder Thomas Rückstiess Frank Sehnke Daan Wierstra and many more who contributed code, suggestions, bug fixes … … and you for your attention! 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Download ppt "A machine learning perspective on neural networks and learning tools"

Similar presentations

Artificial Neural Networks

Artificial Neural Networks
Greedy Layer-Wise Training of Deep Networks

Greedy Layer-Wise Training of Deep Networks
Reinforcement learning

Reinforcement learning
Memristor in Learning Neural Networks

Memristor in Learning Neural Networks
A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.

A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.
Learning in Neural and Belief Networks - Feed Forward Neural Network 2001 년 3 월 28 일 20013329 안순길.

Learning in Neural and Belief Networks - Feed Forward Neural Network 2001 년 3 월 28 일 안순길.
Deep Learning Bing-Chen Tsai 1/21.

Deep Learning Bing-Chen Tsai 1/21.
Ch. Eick: More on Machine Learning & Neural Networks Different Forms of Learning: –Learning agent receives feedback with respect to its actions (e.g. using.

Ch. Eick: More on Machine Learning & Neural Networks Different Forms of Learning: –Learning agent receives feedback with respect to its actions (e.g. using.
Deep Learning.

Deep Learning.
Machine Learning Neural Networks

Machine Learning Neural Networks
Lecture 14 – Neural Networks

Lecture 14 – Neural Networks
Supervised and Unsupervised learning and application to Neuroscience Cours CA6b-4.

Supervised and Unsupervised learning and application to Neuroscience Cours CA6b-4.
1 Part I Artificial Neural Networks Sofia Nikitaki.

1 Part I Artificial Neural Networks Sofia Nikitaki.
Neural Networks. R & G Chapter 8 8.1 Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.

Neural Networks. R & G Chapter Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.
Neural Networks Chapter 8. 8.1 Feed-Forward Neural Networks.

Neural Networks Chapter Feed-Forward Neural Networks.
CHAPTER 11 Back-Propagation Ming-Feng Yeh.

CHAPTER 11 Back-Propagation Ming-Feng Yeh.
Neural Optimization of Evolutionary Algorithm Strategy Parameters Hiral Patel.

Neural Optimization of Evolutionary Algorithm Strategy Parameters Hiral Patel.
Soft Computing Colloquium 2 Selection of neural network, Hybrid neural networks.

Soft Computing Colloquium 2 Selection of neural network, Hybrid neural networks.
Presentation on Neural Networks.. Basics Of Neural Networks Neural networks refers to a connectionist model that simulates the biophysical information.

Presentation on Neural Networks.. Basics Of Neural Networks Neural networks refers to a connectionist model that simulates the biophysical information.
Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.

Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.

Similar presentations

Ads by Google