🚀 Stage 5 of 13

What does R^2 = 0.80 actually mean?

MAE, MSE, RMSE, R^2 — 4 metrics that answer 4 different questions about model quality.

🧠 The intuition R^2 = 0.80 means 'the model explains 80% of the variance in final_score'. That's an abstract statistical statement. For a real-world project, you want metrics in the SAME UNITS as the target. If final_score is 0-100, you want to say 'on average, the model is off by 5 points'. That's MAE. 4 metrics, 4 questions: MAE -> 'on average, how many points off?' (in target units) MSE -> 'how bad are the BIG errors?' (squared, penalises outliers) RMSE -> same as MSE but in target units (interpretable) R^2 -> 'how much better than just guessing mean?' (0 to 1, scale-free)

✏️ Tiny task — write this yourself Compute MAE for your trained model and interpret it. from sklearn.metrics import mean_absolute_error y_pred = model.predict(X_test) mae = mean_absolute_error(y_test, y_pred) print(f'MAE = {mae:.2f} points')

✅ Check — what you should see If MAE = 4.5, that means: 'on average, the model's prediction is 4.5 points off from the actual final_score'. Since final_score is 0-100, that's 4.5% error — pretty good!

🧮 Code files for this stage

📄 app/ml/evaluate.py python

import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np

df = pd.read_csv('data/students.csv').drop(columns=['student_id'])
X = df.drop(columns=['final_score'])
y = df['final_score']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LinearRegression().fit(X_train, y_train)
y_pred = model.predict(X_test)

print(f'MAE  = {mean_absolute_error(y_test, y_pred):.2f} points')
print(f'MSE  = {mean_squared_error(y_test, y_pred):.2f}')
print(f'RMSE = {np.sqrt(mean_squared_error(y_test, y_pred)):.2f} points')
print(f'R^2  = {r2_score(y_test, y_pred):.3f}')

# Example output:
#   MAE  = 4.32 points
#   MSE  = 28.50
#   RMSE = 5.34 points
#   R^2  = 0.798

How it works

MAE is the most interpretable — 'off by 4.3 points on average' makes sense to anyone. RMSE is similar but bigger because it squares the errors first (so a 20-point error counts 16x as much as a 5-point error — RMSE punishes big mistakes). R^2 is scale-free, so you can compare it across datasets — but it doesn't tell you 'how many points off'. Always report MAE to non-technical people; always report R^2 in academic papers; report both in production dashboards.

🚧 Routes — what's wired up at this stage Still no routes. Metrics are computed offline. Once the FastAPI app is live, we can surface these metrics on a '/model-info' page — but the metrics themselves are computed here, at training time.

🧮 Is a response model necessary here? NO — use response_class=HTMLResponse

Offline metric computation, no FastAPI. No response model needed.

⚠️ Common mistakes to avoid

Reporting only R^2 — your boss will ask 'but how many points off is that?' and you won't have an answer.
Confusing MSE and RMSE — MSE is in squared units (points^2), RMSE is in target units (points). Always report RMSE, never MSE directly.
Computing metrics on the TRAIN set — use the test set, always. Train metrics are optimism, test metrics are reality.

👀 What's next Next: Stage 6 is ERROR ANALYSIS — sort by absolute error, look at the worst predictions, and ask 'what kind of student does the model get wrong?'

Next — Stage 6

Find your worst predictions — error analysis

→