Forward Semi-Supervised Feature Selection Jiangtao Ren, Zhengyuan Qiu, Wei Fan, Hong Cheng, and Philip S. Yu.

Slides:

Advertisements

Similar presentations

Current Trends in Machine Learning and Data Mining

Advertisements

Actively Transfer Domain Knowledge Xiaoxiao Shi Wei Fan Jiangtao Ren Sun Yat-sen University IBM T. J. Watson Research Center Transfer when you can, otherwise.

Systematic Data Selection to Mine Concept Drifting Data Streams Wei Fan IBM T.J.Watson.

An Improved Categorization of Classifiers Sensitivity on Sample Selection Bias Wei Fan Ian Davidson Bianca Zadrozny Philip S. Yu.

A General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions Jing Gao Wei Fan Jiawei Han Philip S. Yu University of Illinois.

Decision Tree Evolution using Limited number of Labeled Data Items from Drifting Data Streams Wei Fan 1, Yi-an Huang 2, and Philip S. Yu 1 1 IBM T.J.Watson.

Type Independent Correction of Sample Selection Bias via Structural Discovery and Re-balancing Jiangtao Ren Xiaoxiao Shi Wei Fan Philip S. Yu.

Type Independent Correction of Sample Selection Bias via Structural Discovery and Re-balancing Jiangtao Ren 1 Xiaoxiao Shi 1 Wei Fan 2 Philip S. Yu 2 1.

Adding Unlabeled Samples to Categories by Learned Attributes Jonghyun Choi Mohammad Rastegari Ali Farhadi Larry S. Davis PPT Modified By Elliot Crowley.

Mustafa Cayci INFS 795 An Evaluation on Feature Selection for Text Clustering.

Co Training Presented by: Shankar B S DMML Lab

Data Mining Feature Selection. Data reduction: Obtain a reduced representation of the data set that is much smaller in volume but yet produces the same.

Unsupervised Learning

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions Presentor:

Ensemble Methods An ensemble method constructs a set of base classifiers from the training data Ensemble or Classifier Combination Predict class label.

Universal Learning over Related Distributions and Adaptive Graph Transduction Erheng Zhong †, Wei Fan ‡, Jing Peng*, Olivier Verscheure ‡, and Jiangtao.

Author : Zhen Hai, Kuiyu Chang, Gao Cong Source : CIKM’12 Speaker : Wei Chang Advisor : Prof. Jia-Ling Koh ONE SEED TO FIND THEM ALL: MINING OPINION FEATURES.

Learning on Probabilistic Labels Peng Peng, Raymond Chi-wing Wong, Philip S. Yu CSE, HKUST 1.

Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.

Cross Domain Distribution Adaptation via Kernel Mapping Erheng Zhong † Wei Fan ‡ Jing Peng* Kun Zhang # Jiangtao Ren † Deepak Turaga ‡ Olivier Verscheure.

Graph-based Iterative Hybrid Feature Selection Erheng Zhong † Sihong Xie † Wei Fan ‡ Jiangtao Ren † Jing Peng # Kun Zhang $ † Sun Yat-sen University ‡

Semi-Supervised Learning Using Randomized Mincuts Avrim Blum, John Lafferty, Raja Reddy, Mugizi Rwebangira.

Cross Validation Framework to Choose Amongst Models and Datasets for Transfer Learning Erheng Zhong ¶, Wei Fan ‡, Qiang Yang ¶, Olivier Verscheure ‡, Jiangtao.

1 A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions Zhihong Zeng, Maja Pantic, Glenn I. Roisman, Thomas S. Huang Reported.

Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.

Active Learning for Networked Data Based on Non-progressive Diffusion Model Zhilin Yang, Jie Tang, Bin Xu, Chunxiao Xing Dept. of Computer Science and.

Dongyeop Kang1, Youngja Park2, Suresh Chari2

Active Learning for Class Imbalance Problem

Predictive Modeling with Heterogeneous Sources Xiaoxiao Shi 1 Qi Liu 2 Wei Fan 3 Qiang Yang 4 Philip S. Yu 1 1 University of Illinois at Chicago 2 Tongji.

Fuzzy Entropy based feature selection for classification of hyperspectral data Mahesh Pal Department of Civil Engineering National Institute of Technology.

Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu

A Weakly-Supervised Approach to Argumentative Zoning of Scientific Documents Yufan Guo Anna Korhonen Thierry Poibeau 1 Review By: Pranjal Singh Paper.

Transfer Learning Task. Problem Identification Dataset : A Year: 2000 Features: 48 Training Model ‘M’ Testing 98.6% Training Model ‘M’ Testing 97% Dataset.

Data Mining and Machine Learning Lab Unsupervised Feature Selection for Linked Social Media Data Jiliang Tang and Huan Liu Computer Science and Engineering.

Partially Supervised Classification of Text Documents by Bing Liu, Philip Yu, and Xiaoli Li Presented by: Rick Knowles 7 April 2005.

Xiangnan Kong,Philip S. Yu Department of Computer Science University of Illinois at Chicago KDD 2010.

Stefan Mutter, Mark Hall, Eibe Frank University of Freiburg, Germany University of Waikato, New Zealand The 17th Australian Joint Conference on Artificial.

Pattern Recognition April 19, 2007 Suggested Reading: Horn Chapter 14.

Data Reduction via Instance Selection Chapter 1. Background KDD  Nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable.

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

RSVM: Reduced Support Vector Machines Y.-J. Lee & O. L. Mangasarian First SIAM International Conference on Data Mining Chicago, April 6, 2001 University.

An Efficient Greedy Method for Unsupervised Feature Selection

Image Classification for Automatic Annotation

COP5992 – DATA MINING TERM PROJECT RANDOM SUBSPACE METHOD + CO-TRAINING by SELIM KALAYCI.

Dimensionality Reduction in Unsupervised Learning of Conditional Gaussian Networks Authors: Pegna, J.M., Lozano, J.A., Larragnaga, P., and Inza, I. In.

Semi-supervised Mesh Segmentation and Labeling

Machine Learning ICS 178 Instructor: Max Welling Supervised Learning.

Competition II: Springleaf Sha Li (Team leader) Xiaoyan Chong, Minglu Ma, Yue Wang CAMCOS Fall 2015 San Jose State University.

Optimal Reverse Prediction: Linli Xu, Martha White and Dale Schuurmans ICML 2009, Best Overall Paper Honorable Mention A Unified Perspective on Supervised,

1 Learning Bias & Clustering Louis Oliphant CS based on slides by Burr H. Settles.

Web-Mining Agents: Transfer Learning TrAdaBoost R. Möller Institute of Information Systems University of Lübeck.

On Reducing Classifier Granularity in Mining Concept-Drifting Data Streams Peng Wang, H. Wang, X. Wu, W. Wang, and B. Shi Proc. of the Fifth IEEE International.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Chapter 3 Basic Data Mining Techniques Jason C. H. Chen, Ph.D. Professor of MIS School of Business.

Label Embedding Trees for Large Multi-class Tasks Samy Bengio Jason Weston David Grangier Presented by Zhengming Xing.

Experience Report: System Log Analysis for Anomaly Detection

Cluster Analysis II 10/03/2012.

Linli Xu Martha White Dale Schuurmans University of Alberta

Semi-supervised Machine Learning Gergana Lazarova

Cross Domain Distribution Adaptation via Kernel Mapping

COMBINED UNSUPERVISED AND SEMI-SUPERVISED LEARNING FOR DATA CLASSIFICATION Fabricio Aparecido Breve, Daniel Carlos Guimarães Pedronette State University.

iSRD Spam Review Detection with Imbalanced Data Distributions

Supervised vs. unsupervised Learning

Classification and Prediction

Concave Minimization for Support Vector Machine Classifiers

Semi-Supervised Time Series Classification

Three steps are separately conducted

Data Preprocessing Copyright, 1996 © Dale Carnegie & Associates, Inc.

Learning Dual Retrieval Module for Semi-supervised Relation Extraction

Low-Rank Sparse Feature Selection for Patient Similarity Learning

Presentation transcript:

Forward Semi-Supervised Feature Selection Jiangtao Ren, Zhengyuan Qiu, Wei Fan, Hong Cheng, and Philip S. Yu

Feature Selection Challenges of high dimension dataChallenges of high dimension data Dimensional curse Noise Objective of feature selectionObjective of feature selection Improving the performance of the predictorsImproving the performance of the predictors Providing more cost-effective predictorsProviding more cost-effective predictors Better understanding of the underlying process that generated the dataBetter understanding of the underlying process that generated the data

Supervised / unsupervised learning Supervised learningSupervised learning Used labeled data onlyUsed labeled data only Unsupervised learningUnsupervised learning Used unlabeled data onlyUsed unlabeled data only

Challenges of traditional feature selection methods A lot of supervised learning methodsA lot of supervised learning methods Lack of labeled dataLack of labeled data The class labels are obtained manuallyThe class labels are obtained manually The class labels are expensive to obtainedThe class labels are expensive to obtained Data biasData bias Challenges:Challenges: The training dataset cannot reflect the distribution of the real data in some time.The training dataset cannot reflect the distribution of the real data in some time. The model constructed on training set may be not suitable for the unseen dataThe model constructed on training set may be not suitable for the unseen data

Abundance of the unlabeled data Easy to obtainEasy to obtain Don t need the manually-labeled informationDon t need the manually-labeled information Can reflect the distribution of the real dataCan reflect the distribution of the real data

Then … How to used unlabeled data effectively?

Forward Semi-Supervised Feature Selection Basic ideaBasic idea Random selection from unlabeled data with predicted labelsRandom selection from unlabeled data with predicted labels Form new training setForm new training set Feature selection on new training setFeature selection on new training set Perform several iterationsPerform several iterations Add the most frequent one to the result feature subsetAdd the most frequent one to the result feature subset

Forward Semi-Supervised Feature Selection Iterations Select the best features feature subset SFFS Random selection New training set Select the most frequent one Unlabeled data with predicted labels Iterations Form the new Feature subset Train the Classifier and Prediction

Forward semi-supervised feature selection

Experiment DatasetsDatasets UCIUCI ClassifiersClassifiers NaiveBayes, NNge, and k-NNNaiveBayes, NNge, and k-NN Comparison FULL, SFFS and SLS Z. Zhao and H. Liu. ``Semi-supervised Feature Selection via Spectral Analysis", SIAM International Conference on Data Mining (SDM-07), April 26-28, 2007, Minneapolis, Minnesoda SLSSDM-07

Empirical Results

Conclusion The proposed algorithm works in an iterative procedure;The proposed algorithm works in an iterative procedure; Unlabeled examples receive labels from the classifier constructed on currently selected feature subset;Unlabeled examples receive labels from the classifier constructed on currently selected feature subset; Form joint dataset with labeled and randomly selected unlabeled data with predicted labels;Form joint dataset with labeled and randomly selected unlabeled data with predicted labels; Experiment results show that the proposed approach, can obtained higher accuracy than other supervised and semi-supervised feature selection algorithms in sometime.Experiment results show that the proposed approach, can obtained higher accuracy than other supervised and semi-supervised feature selection algorithms in sometime.