Benchmark
A benchmark is a standardised test that ranks models on a fixed task, dataset and scoring method, so different systems can be compared on the same footing instead of on vendor claims.
No jargon, no math
In plain words, what is it?
A benchmark fixes three things: the task, the dataset and the scoring method. Every candidate answers the same questions and gets a number comparable across models, versions and vendors, often published on a leaderboard. The score is only as honest as the test. If the benchmark's questions leak into a model's training data (contamination), the ranking flatters it. Models can also be tuned until they ace a famous benchmark without being any better at your job (overfitting to the test). And a high score on legal documents says little about your warehouse. So treat public benchmarks as a first filter, then run your own: same fixed task, your data, your scoring. That private number, not the leaderboard, is what should drive your choice.
Terms that live nearby.
Model Evaluation
Model evaluation is the process of measuring how well a model performs on data it has never seen, using metrics and test sets agreed in advance, before you trust it with real decisions.
ReadAI Model
An AI model is the finished artefact that comes out of training: a file of tuned numbers that maps inputs to predictions. You build it once and run it as many times as you like; running it is called inference.
ReadDataset
A dataset is the organised collection of examples an AI learns from or is judged on: thousands or millions of rows of text, images or sensor readings, cleaned, labelled and split into training, validation and test parts.
Read§ 03
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.