Classification metrics — accuracy, precision, recall, F1

For classification models: accuracy, precision, recall, F1, confusion matrix. Each catches a different kind of failure.

script.py v3.1

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    confusion_matrix, classification_report
)

# Generate IMBALANCED data (90% class 0, 10% class 1) — common in real life
X, y = make_classification(
    n_samples=1000, n_features=10, n_informative=5,
    n_classes=2, weights=[0.9, 0.1],  # 90% / 10% split
    random_state=42
)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

lr = LogisticRegression().fit(X_train, y_train)
rf = RandomForestClassifier(n_estimators=50, random_state=42).fit(X_train, y_train)

# ── Compute classification metrics ──
def compute_cls_metrics(model, X_test, y_test, name):
    y_pred = model.predict(X_test)

    acc = accuracy_score(y_test, y_pred)
    prec = precision_score(y_test, y_pred, zero_division=0)
    rec = recall_score(y_test, y_pred, zero_division=0)
    f1 = f1_score(y_test, y_pred, zero_division=0)
    cm = confusion_matrix(y_test, y_pred)

    print(f'\n{name}:')
    print(f'  Accuracy  = {acc:.4f}   (overall correctness)')
    print(f'  Precision = {prec:.4f}   (of predicted positives, how many right?)')
    print(f'  Recall    = {rec:.4f}   (of actual positives, how many caught?)')
    print(f'  F1        = {f1:.4f}   (harmonic mean of P + R)')
    print(f'  Confusion matrix:')
    print(f'    TN={cm[0,0]:4d}  FP={cm[0,1]:4d}')
    print(f'    FN={cm[1,0]:4d}  TP={cm[1,1]:4d}')

compute_cls_metrics(lr, X_test, y_test, 'LogisticRegression')
compute_cls_metrics(rf, X_test, y_test, 'RandomForest')

# ── Why accuracy is misleading on imbalanced data ──
print('\n' + '=' * 60)
print('Why accuracy is MISLEADING on imbalanced data:')
print(f'  Class distribution: {np.bincount(y_test)}')
print(f'  If we ALWAYS predict class 0, accuracy = {np.bincount(y_test)[0]/len(y_test):.4f}')
print(f'  But recall for class 1 = 0.0000 (we never catch positives!)')
print(f'  Use F1 or recall for imbalanced data, NOT accuracy.')

Python output click Run to execute

Click "Run" to execute the code on the left.
The output will appear here.

🧠 How it works

4 metrics + 1 matrix: Accuracy — overall correctness. MISLEADING on imbalanced data (90% class 0 → 90% accuracy by always predicting 0). Precision — of all predicted positives, how many were actually positive? (spam filter: high precision = few false alarms) Recall — of all actual positives, how many did we catch? (cancer screening: high recall = few missed cases) F1 — harmonic mean of precision + recall. Use when you can't pick. Confusion matrix — 2x2 grid of TN, FP, FN, TP. The ground truth behind all metrics. ALWAYS look at the confusion matrix + class distribution before trusting accuracy.

✏️ Try these changes

Change weights from [0.9, 0.1] to [0.5, 0.5]
→ balanced data → accuracy becomes meaningful
Change n_estimators from 50 to 200
→ RF might improve recall
Add print(classification_report(y_test, rf.predict(X_test)))
→ per-class breakdown
Change the target to make_classification(n_classes=3, ...)
→ multi-class — metrics computed per-class

← Previous lab

Regression metrics — MAE, MSE, RMSE, R²

Next lab →

Cross-validation — more honest than a single split