Support Vector Machines (SVM) are a powerful set of supervised machine learning algorithms used for classification and regression tasks. In MATLAB, SVM is available through the Statistics and Machine Learning Toolbox, offering robust tools for training models, optimizing hyperparameters, and visualizing results. This article explores the technical aspects of support vector machines, their mathematical foundation, implementation using MATLAB, and key points summarized in a table format.

Understanding Support Vector Machines

Support Vector Machines are based on finding the hyperplane that best divides a dataset into two classes. The SVM algorithm can classify data that is linearly separable and extend this capability to non-linear data through the use of kernel functions.

Mathematical Foundation

The goal of SVM is to find the optimal hyperplane that maximizes the margin between two classes. The margin is defined as the distance between the hyperplane and the nearest data point of each class.

The equation of a hyperplane in an $n$-dimensional space can be written as: $$w \cdot x - b = 0$$ where $w$ is the weight vector, $x$ is the input vector, and $b$ is the bias term.

Optimization Problem

The optimal hyperplane is obtained by solving the following optimization problem: $$\min\_{w, b} \frac{1}{2}||w||^2$$

Subject to the constraints: y\_i (w \cdot x\_i - b) \geq 1, ; i = 1, 2, \ldots, n\

Kernels

In cases where data is not linearly separable, kernel functions are employed to transform the input data into a higher-dimensional space where a linear separation is possible. Commonly used kernels include:

• Linear Kernel: $K(x_i, x_j) = x_i \cdot x_j$ • Polynomial Kernel: $K(x_i, x_j) = (x_i \cdot x_j + c)^d$ • Radial Basis Function (RBF) Kernel: $K(x_i, x_j) = \exp(-\gamma ||x_i - x_j||^2)$ • Sigmoid Kernel: $K(x_i, x_j) = \tanh(\alpha x_i \cdot x_j + c)$

Implementing SVM in MATLAB

MATLAB offers comprehensive functionalities for SVM implementation, provided by the `fitcsvm` function for training and `predict` function for making predictions. Here's an example of how to implement a two-class SVM classifier: