Presentation is loading. Please wait.

Presentation is loading. Please wait.

scikit-learn Machine Learning in Python Vandana Bachani

Similar presentations

Presentation on theme: "scikit-learn Machine Learning in Python Vandana Bachani"— Presentation transcript:

1 scikit-learn Machine Learning in Python Vandana Bachani
Spring 2012

2 Outline What is scikit-learn? How can it be useful to the lab?
There are other packages too! Features Usage Conclusion

3 What is scikit-learn? scikit-learn is a Python module integrating classic machine learning algorithms in the tightly-knit world of scientific Python packages (numpy, scipy, matplotlib) A comprehensive package for all machine learning needs. Faster Accuracy? If you have the right data, it is pretty loyal. Ref:

4 Ref:

5 How can it be useful to the lab?
Our daily jobs: Regression/Prediction Text Classification Text Feature Extraction Text Feature Selection Using Chi-Square and other metrics Cross-Validation K-Fold Clustering (K-Means, etc.) Maybe in future: Image Classification All in one package!

6 There are other packages too!
NLTK Orange scikit-learn Machine Learning + Text Processing + … Machine Learning + visualizations Machine Learning + Machine Learning Mature (Book exists!) Naïve and sophisticated New, Still developing Documentation – Not so great. Good. Sufficient code examples. Documentation – Very good, but incomplete Lacks in functionality (w.r.t ML), old school Lacks lot of functionality (unsupervised learning) Almost complete w.r.t. machine learning + additional utilities Good Metrics Support Complicated to use Easy to use Easy and intuitive to use Rest API No API support

7 Features Linear Models Regression (Predicting Continuous Values)
Example: Prices of houses (Boston house dataset) Linear, Ridge, Lasso (for sparse coefficients, useful in field of compressed sensing), LARS (very-high dimensional data), Bayesian Classification Logistic Regression, Stochastic Gradient Descent

8 Features Support Vector Machines Classification Regression
SVC (one-vs-one), LinearSVC (one-vs-rest) Regression SVR Density Detection & Outlier Detection (unsupervised learning)

9 Features Unsupervised Learning Clustering Manifold Learning
K-Means, Mean Shift, Spectral Clustering Ward (hierarchical, constructs tree) Manifold Learning Dimensionality Reduction (for visualization, etc) Novelty and Outlier Detection Uses SVM

10 Features Miscellaneous Nearest neighbors Decision Trees
Unsupervised, Classification Decision Trees Classification, Regression Gaussian Processes Regression Metrics metrics.roc_curve(y_true, y_score) metrics.precision_recall_fscore_support(...) joblib and pickle

11 Features Cross-Validation Datasets Feature Extraction
cross_validation.KFold(n, k[, indices]) Datasets Feature Extraction Text feature_extraction.text.WordNGramAnalyzer([...]) feature_extraction.text.CharNGramAnalyzer([...]) Image feature_extraction.image.extract_patches_2d(...) Feature Selection feature_selection.chi2(X, y) feature_selection.SelectKBest(score_func[, k])

12 Usage Linear Regression Classification
>>> from sklearn import linear_model >>> clf = linear_model.LinearRegression() >>> ([[0, 0], [1, 1], [2, 2]], [0, 1, 2]) LinearRegression(copy_X=True, fit_intercept=True, normalize=False) Classification >>> from sklearn.linear_model import SGDClassifier >>> X = [[0., 0.], [1., 1.]] >>> y = [0, 1] >>> clf = SGDClassifier(loss="hinge", penalty="l2") >>>, y) SGDClassifier(alpha=0.0001, class_weight=None, eta0=0.0, fit_intercept=True, learning_rate='optimal', loss='hinge', n_iter=5, n_jobs=1, penalty='l2', power_t=0.5, rho=0.85, seed=0, shuffle=False, verbose=0)

13 Usage SVC & Cross-Validation >>> from sklearn import datasets
>>> from sklearn import svm >>> from sklearn import cross_validation >>> iris = datasets.load_iris() >>> clf = svm.SVC(kernel='linear') >>> scores = cross_validation.cross_val_score(clf,,, cv=5) >>> scores array([ , , , , ])

14 Sample Code penalty = "l2" #LinearSVC can be tried with L1, L2 penalties print "LinearSVC" linearSVC = LinearSVC(loss='l2', penalty=penalty, C=1000, dual=False, tol=1e-3) classify(linearSVC, X_train, y_train, X_test, y_test) #SGDClassifier print "SGDClassifier" sgdClf = SGDClassifier(alpha=.0001, n_iter=50, penalty=penalty) classify(sgdClf, X_train, y_train, X_test, y_test) print "NaiveBayes - Multinomial" bernoulliNBClf = BernoulliNB(alpha=.01) classify(bernoulliNBClf, X_train, y_train, X_test, y_test) def classify(clf, X_train, y_train, X_test, y_test):, y_train) train_time = time() - t0 print "train time: %0.3fs" % train_time pred = clf.predict(X_test) test_time = time() - t0 print "test time: %0.3fs" % test_time print "classification report:" print metrics.classification_report(y_test, pred, target_names=categories) data_train, data_test =, y_train, y_test =, print "Extracting features from the training dataset" #can use a specific analyzer to be passed to vectorizer #by default WordNGramAnalyzer is used vectorizer = Vectorizer() X_train = vectorizer.fit_transform(data_train) print "done in %fs" % (time() - t0) print "n_samples: %d, n_features: %d" % X_train.shape print "Extracting features from the test dataset" X_test = vectorizer.transform(data_test) print "n_samples: %d, n_features: %d" % X_test.shape

15 Sample Results SGDClassifier train time: 1.505s test time: 0.023s classification report: precision recall f1-score support TECHNOLOGY IDIOMS POLITICAL MUSIC GAMES SPORTS MOVIES CELEBRITY avg / total

16 Conclusion If you are a python person - Good documentation wins!
Seems like a good library NLTK + scikit-learn should make an excellent pair for our lab Good documentation wins!

17 Thanks

Download ppt "scikit-learn Machine Learning in Python Vandana Bachani"

Similar presentations

Ads by Google