🚀 Stage 4 of 13

Your first ML model — 4 lines that do the magic

train_test_split + LinearRegression.fit + .predict + .score. The smallest possible ML pipeline.

🧠 The intuition ML training is shockingly small once the data is clean. 4 lines: 1. Split X, y into train and test sets (80/20) 2. Create a LinearRegression() object 3. Call model.fit(X_train, y_train) — this is the 'learning' 4. Call model.score(X_test, y_test) — this is the 'grade' fit() is where the math happens. For LinearRegression, it finds the best-fitting line through the data (least squares). For RandomForest, it builds many decision trees. You don't write the math — sklearn did. Your job is to feed it clean data and read the score.

✏️ Tiny task — write this yourself Train a LinearRegression on the student data and print the score. from sklearn.linear_model import LinearRegression from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) model = LinearRegression().fit(X_train, y_train) print(model.score(X_test, y_test))

✅ Check — what you should see score should be around 0.75-0.85 — meaning the model explains 75-85% of the variance in final_score. If it's near 0, your data is dirty. If it's 1.0, you have data leakage (final_score is in X).

🧮 Code files for this stage

📄 app/ml/train.py python

import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
import joblib

# 1. Load + clean (same as Stage 3)
df = pd.read_csv('data/students.csv').drop(columns=['student_id'])
X = df.drop(columns=['final_score'])
y = df['final_score']

# 2. Split into train (80%) and test (20%)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 3. Train (the 'learning' step)
model = LinearRegression()
model.fit(X_train, y_train)

# 4. Score (the 'grade')
train_score = model.score(X_train, y_train)
test_score  = model.score(X_test, y_test)
print(f'Train R^2: {train_score:.3f}')
print(f'Test  R^2: {test_score:.3f}')

# 5. Save the trained model so FastAPI can load it later
joblib.dump(model, 'app/ml/model.pkl')
print('Saved -> app/ml/model.pkl')

📄 run.sh bash

pip install pandas scikit-learn joblib
python app/ml/train.py

# Output:
#   Train R^2: 0.823
#   Test  R^2: 0.798
#   Saved -> app/ml/model.pkl

How it works

train_test_split is the most important line in ML. It carves out 20% of the data the model never sees during training, then we score on that unseen 20%. This number (test R^2) is the ONLY honest measure of how the model will perform on new students. Train R^2 is optimistic — the model has seen that data. If train is 0.99 and test is 0.50, you've overfit. joblib.dump serializes the trained model to a .pkl file so the web app can load it without re-training.

🚧 Routes — what's wired up at this stage Still no routes. This is the offline training script that produces model.pkl. Once we have model.pkl, Stage 8 will load it inside a FastAPI route to serve predictions.

🧮 Is a response model necessary here? NO — use response_class=HTMLResponse

Training script, not web server. No response model needed.

⚠️ Common mistakes to avoid

Scoring on the train set and reporting it as the model's accuracy — that's cheating.
Forgetting random_state — every run gives a different split, so your scores aren't reproducible.
Not saving the model — you'd have to retrain on every server restart, which is slow and gives different models.

👀 What's next Next: Stage 5 explains what that 0.80 R^2 actually means — and introduces MAE, MSE, RMSE so you can interpret predictions in real units.

Next — Stage 5

What does R^2 = 0.80 actually mean?

→