An SVM Based Voting Algorithm with Application to Parse Reranking Paper by Libin Shen and Aravind K. Joshi Presented by Amit Wolfenfeld.

Slides:

Advertisements

Similar presentations

Statistical Machine Learning- The Basic Approach and Current Research Challenges Shai Ben-David CS497 February, 2007.

Advertisements

Introduction to Support Vector Machines (SVM)

Generative Models Thus far we have essentially considered techniques that perform classification indirectly by modeling the training data, optimizing.

CSC321: Introduction to Neural Networks and Machine Learning Lecture 24: Non-linear Support Vector Machines Geoffrey Hinton.

Lecture 9 Support Vector Machines

ECG Signal processing (2)

VC theory, Support vectors and Hedged prediction technology.

SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.

Classification / Regression Support Vector Machines

Support Vector Machines and Kernels Adapted from slides by Tim Oates Cognition, Robotics, and Learning (CORAL) Lab University of Maryland Baltimore County.

SVM—Support Vector Machines

Search Engines Information Retrieval in Practice All slides ©Addison Wesley, 2008.

Machine learning continued Image source:

The Disputed Federalist Papers : SVM Feature Selection via Concave Minimization Glenn Fung and Olvi L. Mangasarian CSNA 2002 June 13-16, 2002 Madison,

Support Vector Machines (and Kernel Methods in general)

Support Vector Machines (SVMs) Chapter 5 (Duda et al.)

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

Measuring Model Complexity (Textbook, Sections ) CS 410/510 Thurs. April 27, 2007 Given two hypotheses (models) that correctly classify the training.

Reliability and Information Gain Ida Sprinkhuizen-Kuyper Evgueni Smirnov Georgi Nalbantov (UM/EUR)

Computational Learning Theory

Support Vector Machines Based on Burges (1998), Scholkopf (1998), Cristianini and Shawe-Taylor (2000), and Hastie et al. (2001) David Madigan.

Sketched Derivation of error bound using VC-dimension (1) Bound our usual PAC expression by the probability that an algorithm has 0 error on the training.

Support Vector Machines

SVM Support Vectors Machines

What is Learning All about ?  Get knowledge of by study, experience, or being taught  Become aware by information or from observation  Commit to memory.

Support Vector Machines

Support Vector Machines Piyush Kumar. Perceptrons revisited Class 1 : (+1) Class 2 : (-1) Is this unique?

Linear hyperplanes as classifiers Usman Roshan. Hyperplane separators.

Exploring a Hybrid of Support Vector Machines (SVMs) and a Heuristic Based System in Classifying Web Pages Santa Clara, California, USA Ahmad Rahman, Yuliya.

1 SUPPORT VECTOR MACHINES İsmail GÜNEŞ. 2 What is SVM? A new generation learning system. A new generation learning system. Based on recent advances in.

Classification and Ranking Approaches to Discriminative Language Modeling for ASR Erinç Dikici, Murat Semerci, Murat Saraçlar, Ethem Alpaydın 報告者：郝柏翰 2013/01/28.

Universit at Dortmund, LS VIII

Date: 2014/02/25 Author: Aliaksei Severyn, Massimo Nicosia, Aleessandro Moschitti Source: CIKM’13 Advisor: Jia-ling Koh Speaker: Chen-Yu Huang Building.

1 Boosting-based parse re-ranking with subtree features Taku Kudo Jun Suzuki Hideki Isozaki NTT Communication Science Labs.

Stochastic Subgradient Approach for Solving Linear Support Vector Machines Jan Rupnik Jozef Stefan Institute.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

1 Chapter 6. Classification and Prediction Overview Classification algorithms and methods Decision tree induction Bayesian classification Lazy learning.

CS 478 – Tools for Machine Learning and Data Mining SVM.

Kernel Methods: Support Vector Machines Maximum Margin Classifiers and Support Vector Machines.

Ohad Hageby IDC Support Vector Machines & Kernel Machines IP Seminar 2008 IDC Herzliya.

Linear hyperplanes as classifiers Usman Roshan. Hyperplane separators.

SVM – Support Vector Machines Presented By: Bella Specktor.

Classification (slides adapted from Rob Schapire) Eran Segal Weizmann Institute.

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

Dec 21, 2006For ICDM Panel on 10 Best Algorithms Support Vector Machines: A Survey Qiang Yang, for ICDM 2006 Panel Partially.

Support Vector Machines

Support Vector Machines Tao Department of computer science University of Illinois.

CZ5225: Modeling and Simulation in Biology Lecture 7, Microarray Class Classification by Machine learning Methods Prof. Chen Yu Zong Tel:

Text Categorization With Support Vector Machines: Learning With Many Relevant Features By Thornsten Joachims Presented By Meghneel Gore.

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

Support Vector Machines and Kernel Methods for Co-Reference Resolution 2007 Summer Workshop on Human Language Technology Center for Language and Speech.

Support Vector Machines (SVM): A Tool for Machine Learning Yixin Chen Ph.D Candidate, CSE 1/10/2002.

Chapter 6. Classification and Prediction Classification by decision tree induction Bayesian classification Rule-based classification Classification by.

Feature Selction for SVMs J. Weston et al., NIPS 2000 오장민 (2000/01/04) Second reference : Mark A. Holl, Correlation-based Feature Selection for Machine.

Support-Vector Networks C Cortes and V Vapnik (Tue) Computational Models of Intelligence Joon Shik Kim.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Kernel Methods: Support Vector Machines Maximum Margin Classifiers and Support Vector Machines.

SVMs in a Nutshell.

FUZZ-IEEE Kernel Machines and Additive Fuzzy Systems: Classification and Function Approximation Yixin Chen and James Z. Wang The Pennsylvania State.

Support Vector Machine (SVM) Presented by Robert Chen.

SUPPORT VECTOR MACHINES Presented by: Naman Fatehpuria Sumana Venkatesh.

Finding Clusters within a Class to Improve Classification Accuracy Literature Survey Yong Jae Lee 3/6/08.

Introduction to Machine Learning Prof. Nir Ailon Lecture 5: Support Vector Machines (SVM)

Support Vector Machines Reading: Textbook, Chapter 5 Ben-Hur and Weston, A User’s Guide to Support Vector Machines (linked from class web page)

Support Vector Machines (SVMs) Chapter 5 (Duda et al.) CS479/679 Pattern Recognition Dr. George Bebis.

1 CS 391L: Machine Learning: Computational Learning Theory Raymond J. Mooney University of Texas at Austin.

CS 9633 Machine Learning Support Vector Machines

An Introduction to Support Vector Machines

Pawan Lingras and Cory Butz

Support Vector Machines and Kernels

Presentation transcript:

An SVM Based Voting Algorithm with Application to Parse Reranking Paper by Libin Shen and Aravind K. Joshi Presented by Amit Wolfenfeld

Outline Introduction of Parse Reranking SVM An SVM Based Voting Algorithm Theoretical Justification Experiments on Parse Reranking Conclusions

Introduction – Parse Reranking Motivation (Collins) votererankf-scoreLog- likelihood parsesrank 392%-120.0P21 490%-121.5P32 x196%-122.0P13 293%-122.5P44

Support Vector Machines The SVM is a large margin classifier that searches for the hyperplane that maximizes the margin between the positive samples and the negative samples

Support Vector Machines Measures of the capacity of a learning machine: VC Dimension, Fat Shattering Dimension The capacity of a learning machine is related to the margin on the training data. - As the margin goes up, VC-dimension may go down and thus the upper bound of the test error goes down. (Vapnik 79)

Support Vector Machines SVMs’ theoretical accuracy is much lower than their actual performance. The margin based upper bounds of the test error are too loose. This is why – SVM based voting algorithm.

SVM Based Voting Previous work (Dijkstra 02) - Use SVM for parse reranking directly. - Positive samples: parse with highest f-score for each sentence. First try -Tree kernel: compute dot-product on the space of all the subtrees (Collins 02) -Linear kernel: rich features (Collins 00)

SVM based Voting Algorithm

Preference Kernels

SVM based Voting

Theoretical Issues Justifying the Preference Kernel Justifying Pairwise Samples Margin Based Bound for the SVM Based Voting Algorithm

Justifying the Preference Kernel

Justifying the Pairwise Samples

Margin Based Bound for SVM Based voting

Experiments – WSJ Treebank N-best parsing results (Collins 02) SVM-light (Joachims 98) Two Kernels (K) used in the preference kernel: - Linear Kernel - Tree Kernel Tree Kernel- very slow

Experiments – Linear Kernel

Results

Conclusions Using an SVM approach : - achieving state-of-the-art results - SVM with linear kernel is superior to tree kernel in speed and accuracy.

T noukhaY !