Support Vector Machine
Support Vector Machine
Luyang Liu
152003815
I. Introduction
Support vector machines (SVMs) are a basic machine learning method for supervised learning models with associated learning algorithms that analyze data and recognize patterns, used for classification and regression analysis. Introduced by Vladimir Vapnik and his colleagues, SVMs are a relatively new learning method used for binary classification. The basic idea is to find a hyperplane which separates the d-dimensional data perfectly into its two classes.
Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on.
SVM can be considered as a revision of logistic regression. Logistic regression is also used to predict a binary response from a binary predictor, used for predicting the outcome of a categorical dependent variable (i.e., a class label) based on one or more predictor variables (features). While logistic regression considers more on overall distributions of the classes, SVM seems consider more on the boundaries of the classes are more informative than the overall distributions.
However, since example data is often not linearly separable, SVM's introduce the notion of a “kernel induced feature space” which casts the data into a higher dimensional space where the data is separable. Typically, casting into such a space would cause problems computationally, and with overfitting. The key insight used in SVM's is that the higher-dimensional space doesn't need to be dealt with directly (as it turns out, only the
Luyang Liu
152003815
I. Introduction
Support vector machines (SVMs) are a basic machine learning method for supervised learning models with associated learning algorithms that analyze data and recognize patterns, used for classification and regression analysis. Introduced by Vladimir Vapnik and his colleagues, SVMs are a relatively new learning method used for binary classification. The basic idea is to find a hyperplane which separates the d-dimensional data perfectly into its two classes.
Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on.
SVM can be considered as a revision of logistic regression. Logistic regression is also used to predict a binary response from a binary predictor, used for predicting the outcome of a categorical dependent variable (i.e., a class label) based on one or more predictor variables (features). While logistic regression considers more on overall distributions of the classes, SVM seems consider more on the boundaries of the classes are more informative than the overall distributions.
However, since example data is often not linearly separable, SVM's introduce the notion of a “kernel induced feature space” which casts the data into a higher dimensional space where the data is separable. Typically, casting into such a space would cause problems computationally, and with overfitting. The key insight used in SVM's is that the higher-dimensional space doesn't need to be dealt with directly (as it turns out, only the