Evaluation metrics

Accuracy, precision, recall, F1, RMSE, MAE — pick the metric that matches the goal.

Code example

from sklearn.metrics import (
    accuracy_score, f1_score,
    mean_squared_error, r2_score
)

# classification
acc = accuracy_score(y_test, preds)
f1  = f1_score(y_test, preds, average='weighted')

# regression
rmse = mean_squared_error(y_test, preds, squared=False)
r2   = r2_score(y_test, preds)

How it works

Different metrics measure different things. Accuracy is intuitive but misleading on imbalanced data (90% accuracy is trivial if 90% of samples are class 0). F1 balances precision and recall. RMSE penalises large errors more than MAE. R² measures how much variance the model explains. OpenBenchML picks the metric based on task type — accuracy/F1 for classification, RMSE/R² for regression — and the leaderboard ranks by that metric.