Concepts 🚀 Project 🧮 Labs

SVM visualized — hyperplane + margins + support vectors (iris)

SEE the SVM hyperplane, the margin, and the support vectors. Compare linear vs RBF kernel. See how C controls the margin width.

script.py v3.1

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler

# ── Load iris, use 2 features for visualization ──
iris = load_iris()
X = iris.data[:, [2, 3]]  # petal length, petal width
y = iris.target
# Use only 2 classes (versicolor vs virginica) for binary SVM visualization
mask = y != 0  # remove setosa (too easy)
X = X[mask]
y = y[mask] - 1  # relabel to 0, 1

X_scaled = StandardScaler().fit_transform(X)

print(f'Dataset: iris versicolor (0) vs virginica (1) — {len(y)} samples')
print(f'These two species OVERLAP — perfect for demonstrating SVM margins')

# ════════════════════════════════════════════════════════════════════
#  Compare 4 SVM configurations side by side
# ════════════════════════════════════════════════════════════════════
configs = [
    ('Linear, C=1', SVC(kernel='linear', C=1.0, random_state=42)),
    ('Linear, C=100', SVC(kernel='linear', C=100, random_state=42)),
    ('RBF, C=1, γ=scale', SVC(kernel='rbf', C=1.0, gamma='scale', random_state=42)),
    ('RBF, C=100, γ=scale', SVC(kernel='rbf', C=100, gamma='scale', random_state=42)),
]

fig, axes = plt.subplots(2, 2, figsize=(14, 12))

for ax, (title, model) in zip(axes.ravel(), configs):
    model.fit(X_scaled, y)

    # Create mesh for decision boundary
    x_min, x_max = X_scaled[:, 0].min() - 1, X_scaled[:, 0].max() + 1
    y_min, y_max = X_scaled[:, 1].min() - 1, X_scaled[:, 1].max() + 1
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, 200), np.linspace(y_min, y_max, 200))
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)

    # Plot decision regions
    ax.contourf(xx, yy, Z, alpha=0.2, cmap=plt.cm.RdBu)
    ax.contour(xx, yy, Z, colors='black', linewidths=0.5, alpha=0.3)

    # Plot decision boundary + margins (only for linear kernel)
    if model.kernel == 'linear':
        w = model.coef_[0]
        b = model.intercept_[0]
        # Decision boundary: w[0]*x + w[1]*y + b = 0 → y = -(w[0]*x + b) / w[1]
        x_line = np.linspace(x_min, x_max, 100)
        y_boundary = -(w[0] * x_line + b) / w[1]
        y_margin_up = -(w[0] * x_line + b - 1) / w[1]
        y_margin_down = -(w[0] * x_line + b + 1) / w[1]
        ax.plot(x_line, y_boundary, 'k-', linewidth=2, label='decision boundary')
        ax.plot(x_line, y_margin_up, 'k--', linewidth=1, alpha=0.5, label='margin')
        ax.plot(x_line, y_margin_down, 'k--', linewidth=1, alpha=0.5)

    # Plot data points — highlight support vectors
    ax.scatter(X_scaled[y == 0, 0], X_scaled[y == 0, 1], c='#3498db',
              label='versicolor', edgecolors='black', s=40, zorder=3)
    ax.scatter(X_scaled[y == 1, 0], X_scaled[y == 1, 1], c='#e74c3c',
              label='virginica', edgecolors='black', s=40, zorder=3)
    # Support vectors (circled)
    ax.scatter(model.support_vectors_[:, 0], model.support_vectors_[:, 1],
              s=150, facecolors='none', edgecolors='lime', linewidths=2,
              label=f'support vectors ({len(model.support_vectors_)})', zorder=4)

    ax.set_title(f'{title}\nAccuracy: {model.score(X_scaled, y):.3f}')
    ax.legend(fontsize=8, loc='upper left')

plt.suptitle('SVM: Hyperplane + Margins + Support Vectors', fontsize=14, y=1.01)
plt.tight_layout()

print(f'\nResults:')
for title, model in configs:
    model.fit(X_scaled, y)
    n_sv = len(model.support_vectors_)
    print(f'  {title:25s}  accuracy={model.score(X_scaled, y):.3f}  support_vectors={n_sv}')

print(f'\nTHE MATHS:')
print(f'  SVM finds the hyperplane that MAXIMIZES THE MARGIN between classes.')
print(f'  Decision boundary: w·x + b = 0')
print(f'  Margins: w·x + b = ±1')
print(f'  Support vectors = points ON the margin (circled in green).')
print(f'  These are the ONLY points that matter — moving other points doesn\'t change the boundary.')
print(f'\n  C parameter (trade-off: margin width vs classification errors):')
print(f'    C=1: wide margin, allows some misclassifications (soft margin)')
print(f'    C=100: narrow margin, tries to classify everything correctly (might overfit)')
print(f'\n  Kernel trick (for non-linear data):')
print(f'    linear: straight line boundary')
print(f'    rbf (Gaussian): curved boundary — maps to infinite dimensions')
print(f'    poly: polynomial boundary')
print(f'  gamma: controls how far each training point influences the boundary')

Python output click Run to execute

Click "Run" to execute the code on the left.
The output will appear here.

🧠 How it works

SVM (Support Vector Machine) — VISUALIZED: GOAL: find the hyperplane that MAXIMIZES THE MARGIN between two classes. Not just any separating line — the one with the BIGGEST gap. DECISION BOUNDARY: w·x + b = 0 (solid line). MARGINS: w·x + b = ±1 (dashed lines). The gap between them = the margin. SUPPORT VECTORS: the points ON the margin (circled in green). These are the ONLY points that determine the boundary. Move any other point → boundary stays. That's why SVM is memory-efficient — you only need to remember the support vectors. C PARAMETER (regularization): C=1 (soft margin): wide margin, allows misclassifications. More robust to outliers. C=100 (hard margin): narrow margin, tries to classify everything. Might overfit. KERNEL TRICK (for non-linear data): linear: straight line. Fast, interpretable. Use when data is linearly separable. rbf (Gaussian): curved boundary. Maps data to INFINITE dimensions. Most popular. poly: polynomial boundary. Good for specific cases. GAMMA: controls how far each point's influence reaches. High gamma = each point only affects nearby region (can overfit). Low gamma = each point affects far away (smoother boundary). VERSICOLOR vs VIRGINICA: these species overlap in petal dimensions. Linear SVM with C=1 accepts some errors (soft margin). RBF kernel curves around the overlap — higher accuracy but might overfit.

✏️ Try these changes

Change kernel from 'linear' to 'poly' with degree=3
→ polynomial boundary — curved like RBF but different shape
Change gamma from 'scale' to 10
→ high gamma → each point has small influence → tighter, more complex boundary
Change gamma from 'scale' to 0.01
→ low gamma → each point has large influence → smoother boundary
Use all 3 iris classes (remove the mask)
→ SVM does multi-class via one-vs-one — 3 boundaries instead of 1

← Previous lab

PCA visualized — variance + projection (iris + digits)

Next lab →

Decision Tree visualized — the splits + overfitting (iris)