Regression metrics — MAE, MSE, RMSE, R²

Compute the 4 standard regression metrics. Each tells you something different about model quality.

script.py v3.1

import numpy as np
from sklearn.linear_model import LinearRegression, RandomForestRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# Generate data + train 2 models
X, y = make_regression(n_samples=500, n_features=5, noise=20, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

lr = LinearRegression().fit(X_train, y_train)
rf = RandomForestRegressor(n_estimators=50, random_state=42).fit(X_train, y_train)

# ── Compute metrics for both models ──
def compute_metrics(model, X_test, y_test, name):
    y_pred = model.predict(X_test)

    mae = mean_absolute_error(y_test, y_pred)
    mse = mean_squared_error(y_test, y_pred)
    rmse = np.sqrt(mse)
    r2 = r2_score(y_test, y_pred)

    print(f'\n{name}:')
    print(f'  MAE  = {mae:8.2f}   (avg error — easy to interpret)')
    print(f'  MSE  = {mse:8.2f}   (squared error — punishes big errors)')
    print(f'  RMSE = {rmse:8.2f}   (sqrt of MSE — back in original units)')
    print(f'  R²   = {r2:8.4f}   (0-1, how much better than guessing mean)')
    return {'MAE': mae, 'MSE': mse, 'RMSE': rmse, 'R2': r2}

lr_metrics = compute_metrics(lr, X_test, y_test, 'LinearRegression')
rf_metrics = compute_metrics(rf, X_test, y_test, 'RandomForest')

# ── Compare ──
print('\n' + '=' * 50)
print('Comparison (lower is better for MAE/MSE/RMSE, higher for R²):')
for metric in ['MAE', 'MSE', 'RMSE', 'R2']:
    lr_val = lr_metrics[metric]
    rf_val = rf_metrics[metric]
    if metric == 'R2':
        winner = 'RF' if rf_val > lr_val else 'LR'
    else:
        winner = 'RF' if rf_val < lr_val else 'LR'
    print(f'  {metric:5s}  LR={lr_val:8.2f}  RF={rf_val:8.2f}  winner: {winner}')

Python output click Run to execute

Click "Run" to execute the code on the left.
The output will appear here.

🧠 How it works

4 metrics, 4 questions: MAE (Mean Absolute Error) — 'on average, how far off?' Same units as target. Most interpretable. MSE (Mean Squared Error) — squares errors, so big errors count more. Units are target² (hard to interpret). RMSE (Root Mean Squared Error) — sqrt of MSE. Back in target units. Like MAE but punishes outliers. R² (R-squared) — 0 to 1. How much better than just predicting the mean. 1=perfect, 0=mean, <0=worse than mean. Rule of thumb: report MAE to non-technical people, R² in papers, RMSE when comparing models with outliers.

✏️ Try these changes

Change noise from 20 to 50 in make_regression
→ harder data → all metrics worse
Change n_estimators from 50 to 200
→ RF might improve slightly
Add from sklearn.svm import SVR; compute SVR metrics
→ compare 3 models
Change test_size from 0.2 to 0.4
→ bigger test set → more stable metrics

← Previous lab

Measure inference time + throughput

Next lab →

Classification metrics — accuracy, precision, recall, F1