Support Vector Machine

Concept

Support Vector Machine is in essence "Maximum Margin Classifier".

It is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data the algorithm outputs an optimal hyperplane which categorizes new examples. For example, in two dimensional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side.

Kernels in SVM

The objective of using kernels is to map feature vectors from smaller dimensional space to a higher one.

Amazing explanations from Quora.

Hyperparameters

Kernel

Come back later.

Regularization

In Python.sklearn the regularization parameter is often referred as C parameter.

For large values of C, the optimization will choose a smaller margin hyperplane if the plane does a better job of getting all the training points classified correctly. A small C will cause the optimizer to search for a larger margin separating hyperplane, even if it misclassfies more points.

Gamma

The gamma parameter defines how far the influence of a single training example reaches, with low values meaning "far" and high values meaning "close".

Margin

A margin is a separation of line to the closest class points.