Model Evaluation

No jargon, no math

Evaluation answers one question before you trust a model: how does it do on data it has never seen? First you hold out a test set, kept apart from training so the model cannot memorise it. Then you pick metrics that match the task: accuracy or F1 for classification, error in real units for forecasting, a judge model with a rubric for generative answers. The candidate is scored on the test set, then compared against two references: a simple baseline and the model currently in production. The average score is not enough; you also slice the results by region, product or customer type, because 95% overall can hide 60% on slow-movers. Pass the gate and the model deploys; fail it and it goes back with a report. Where it breaks: a test set that leaks into training, or metrics that reward the wrong behaviour, like accuracy on a fraud dataset where fraud is 0.1% of the rows and predicting “no fraud” always wins.