1. Publication Venues Main Neural Network Conferences –NIPS (Neural Information Processing Systems) –IJCNN (Intl Joint Conf on Neural Networks) Main Neural Network Journals –Neural Networks –Neural Computation –IEEE Transactions on Neural Networks

2. Publication Venues Main Machine Learning Conferences –ICML (Intl Conf on Machine Learning) –COLT (Computational Learning Theory) Main Machine Learning Journals –ML (Machine Learning) –JMLR (J. Machine Learning Research) –JAIR (J. Artificial Intelligence Research)

3. Underfit and Overfit

4. Need for Bias 2 2 n Boolean function of n inputs x1x2x3ClassPossible Consistent Function Hypotheses 00011111111111111111 00111111111111111111 01011111111111111111 01111111111111111111 1000000000011111111 1010000111100001111 1100011001100110011 111?0101010101010101

5. No Free Lunch Any inductive bias chosen will have equal accuracy compared to any other bias over all possible functions (assuming all functions are equally likely). If correct on some cases, must be incorrect on equally many cases. Is this a problem? –Random vs. Regular –Anti-Bias (even though regular) –The “Interesting” Problems – subset of learnable?

6. Automatic Discover of Inductive Bias Defining the set of Interesting/Learnable problems – No Free Lunch concepts Defining the set of available inductive biases oProposing novel learning algorithms oAnalysis, comparison, and extension of current learning algorithms oDefining/discovering a set of biases which covers I (Interesting problems) oParameter Free learning algorithms – Automatic selection of learning parameters Automatically fitting a bias to a problem – Overfit, underfit, noise issues, etc. Automatic Feature Selection

7. ADIB (Cont.) Dynamic Inductive Biases oPre-selection of an appropriate bias based on the application data set oAutomatically selecting a bias during learning oBias which adjusts dynamically in time during learning oBias which adjusts dynamically in space during learning (different parts of the problem space are better learned with different biases, including differing parameters in one bias). oCombinations of the above Combination of Biases oLinear and non-linear combinations of biases oDynamic combinations of biases oEnsemble variants