Concepts 🚀 Project 🧮 Labs

Logistic Regression visualized — sigmoid + decision boundary (iris)

SEE the sigmoid curve, SEE the decision boundary move as you change C. Uses the iris dataset. The maths becomes obvious when you can visualize it.

script.py v3.1

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

# ── Load iris (real dataset — 150 flowers, 4 features, 3 species) ──
iris = load_iris()
# Use only 2 features for 2D visualization: petal length + petal width
X = iris.data[:, [2, 3]]  # petal length, petal width
y = iris.target  # 0=setosa, 1=versicolor, 2=virginica
feature_names = ['petal length (cm)', 'petal width (cm)']

print(f'Dataset: iris (150 samples, 3 species)')
print(f'Features used: {feature_names}')
print(f'Species: {list(iris.target_names)}')

# ════════════════════════════════════════════════════════════════════
#  PLOT 1: The SIGMOID function — the heart of logistic regression
# ════════════════════════════════════════════════════════════════════
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Sigmoid: σ(z) = 1 / (1 + e^(-z))
z = np.linspace(-6, 6, 100)
sigmoid = 1 / (1 + np.exp(-z))
axes[0].plot(z, sigmoid, color='#a0c000', linewidth=3)
axes[0].axhline(0.5, color='gray', linestyle='--', alpha=0.5)
axes[0].axvline(0, color='gray', linestyle='--', alpha=0.5)
axes[0].set_xlabel('z = β₀ + β₁x₁ + β₂x₂ (linear combination)')
axes[0].set_ylabel('σ(z) = probability')
axes[0].set_title('Sigmoid function: squashes any number to [0, 1]')
axes[0].annotate('σ(0) = 0.5\n(decision boundary)', xy=(0, 0.5), xytext=(2, 0.3),
                fontsize=9, arrowprops=dict(arrowstyle='->', color='red'))
axes[0].set_ylim(-0.1, 1.1)

# PLOT 2: Decision boundary on iris data
# Scale features first (logistic regression is sensitive to scale)
X_scaled = StandardScaler().fit_transform(X)
model = LogisticRegression(C=1.0, random_state=42)
model.fit(X_scaled, y)

# Create a mesh to plot the decision boundary
x_min, x_max = X_scaled[:, 0].min() - 1, X_scaled[:, 0].max() + 1
y_min, y_max = X_scaled[:, 1].min() - 1, X_scaled[:, 1].max() + 1
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 200), np.linspace(y_min, y_max, 200))
Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

# Plot decision boundary (colored regions) + data points
axes[1].contourf(xx, yy, Z, alpha=0.3, cmap=plt.cm.Set1)
axes[1].contour(xx, yy, Z, colors='black', linewidths=0.5, alpha=0.5)
colors = ['#e74c3c', '#2ecc71', '#3498db']
for i, species in enumerate(iris.target_names):
    mask = y == i
    axes[1].scatter(X_scaled[mask, 0], X_scaled[mask, 1], c=colors[i],
                   label=species, edgecolors='black', s=40)
axes[1].set_xlabel(feature_names[0] + ' (scaled)')
axes[1].set_ylabel(feature_names[1] + ' (scaled)')
axes[1].set_title(f'Decision boundary (C=1.0)\nAccuracy: {model.score(X_scaled, y):.3f}')
axes[1].legend()

plt.tight_layout()
# Don't call plt.show() — the kernel captures the figure automatically

print(f'\nModel: LogisticRegression(C=1.0)')
print(f'Coefficients shape: {model.coef_.shape} (3 classes × 2 features)')
print(f'Intercepts: {model.intercept_.round(3)}')
print(f'Accuracy: {model.score(X_scaled, y):.3f}')
print()
print('THE MATHS:')
print('  Linear regression: y = β₀ + β₁x (predicts a NUMBER)')
print('  Logistic regression: P(class=1) = σ(β₀ + β₁x) = 1/(1+e^(-z))')
print('  The sigmoid SQUASHES the linear output to [0, 1] = probability')
print('  Decision boundary: where σ(z) = 0.5, i.e., z = 0')
print('  The LINE in the plot is where β₀ + β₁x₁ + β₂x₂ = 0')
print()
print('C parameter (inverse regularization):')
print('  C=1.0 (default): balanced')
print('  C=0.01: strong regularization → simpler boundary (might underfit)')
print('  C=100: weak regularization → complex boundary (might overfit)')

Python output click Run to execute

Click "Run" to execute the code on the left.
The output will appear here.

🧠 How it works

Logistic Regression maths — VISUALIZED: SIGMOID: σ(z) = 1/(1+e^(-z)). Squashes any number to [0, 1] = probability. At z=0, σ=0.5 (the decision boundary). z > 0 → σ > 0.5 → predict class 1. z < 0 → σ < 0.5 → predict class 0. DECISION BOUNDARY: the LINE where z = β₀ + β₁x₁ + β₂x₂ = 0. Everything on one side = class 0, other side = class 1. For 3 classes (iris): 3 binary classifiers (one-vs-rest), boundaries intersect. C PARAMETER (inverse regularization): C=1.0 (default): balanced. C=0.01: strong regularization → simpler boundary → might underfit. C=100: weak regularization → complex boundary → might overfit. KEY INSIGHT: logistic regression is LINEAR — the boundary is always a straight line (or hyperplane). If classes aren't linearly separable, it fails. That's why iris setosa (easily separable) gets 100% accuracy but versicolor/virginica overlap.

✏️ Try these changes

Change C from 1.0 to 0.01
→ strong regularization → simpler boundary, might misclassify more
Change C from 1.0 to 100
→ weak regularization → boundary fits tighter, might overfit
Use sepal features instead: X = iris.data[:, [0, 1]]
→ different feature pair — setosa still separable, but versicolor/virginica overlap more
Add max_iter=2000 to the model (some convergences need more iterations)
→ fixes convergence warnings

← Previous lab

Full ML workflow — the senior data scientist's process

Next lab →

PCA visualized — variance + projection (iris + digits)