… visits todayONLINE
How to tell when an AI model quietly gets worse — and how we measure it across every major provider.
gpt-5.5 or claude-opus-5 at any time, and your code keeps calling the same string. If output quality drops, nothing in the API tells you. AI Stupid Level benchmarks every major model continuously and applies change-point detection to the results, so a sustained decline becomes a measurement instead of a hunch.AI drift is a measurable change in a model's output quality over time while its public identifier stays the same. It is distinct from prompt drift (your inputs changing) and from data drift in classical ML (the input distribution moving). Here the model itself is the thing that moved.
Drift is not always downward — models sometimes improve after an update. The problem is that neither direction is announced, so teams building on a model have no way to know their evaluation results are still valid. That uncertainty is the actual cost: you cannot tell whether a regression in your product came from your last deploy or from your provider.
A provider routes a stable alias to a new checkpoint. Behaviour changes; the model name does not.
Quantisation, distillation and serving-stack changes cut inference cost, and can cut output quality with it — particularly on long-context and multi-step reasoning work.
Tightened filters raise refusal rates on legitimate requests. The model is not less capable, but it is measurably less useful for the task you had. In our multi-turn reasoning suite, a prompt the provider declines outright is recorded as a refusal with the provider's stated category and that session's score is withheld, so a tightened filter shows up as a refusal rate rather than as invented incapability.
Under heavy load, providers may shift defaults such as sampling parameters or reasoning budget, which shows up as higher variance rather than a clean step down.
A single weak response is not evidence. Model output is stochastic, so scores vary run to run even when nothing has changed upstream. A naive threshold alarm on raw scores either fires constantly on that noise or is set so loose it misses real decline.
We use the Page-Hinkley test — a cumulative-sum change detector from statistical process control. Instead of judging each run in isolation, it accumulates how far each day's median score falls below the running mean, less a small tolerance, and tracks the gap between that running total and its lowest point so far. Random noise has no consistent sign, so it cancels out and the gap stays near zero. A genuine sustained decline pushes the deviations consistently in one direction, so the gap grows until it crosses a decision threshold and a drift event fires.
The threshold is what trades false alarms against detection lag. The tolerance and threshold were chosen by sweeping them against the full history of real benchmark scores, so ordinary day-to-day fluctuation stays quiet while a real shift is confirmed over days rather than hours. The statistic runs on each suite's own daily series — coding, tool use and reasoning are never blended, because they sit on different scales — and restarts whenever a suite's configuration changes. For sharp drops there is a second, faster instrument: an hourly canary of two fixed probes, tested over the last 6 and 24 hours against the prior week with Welch's t-test, which raises an incident at a 12-point fall with p < 0.01 and closes it when the gap closes. Every score is also published with a standard error measured from its own run-to-run repeatability, the leaderboard ties models whose lead is inside that noise, and a composite score is labelled with how many suites actually contributed to it. The full statistical approach, including the constants and the measured false-alarm rates, is documented on our benchmarking methodology page.
Drift is tracked per benchmark suite, because a model can decline in one dimension while holding steady in another — a common pattern after a cost-optimisation update.
Correctness, task complexity handling, code quality, stability, efficiency, edge cases, debugging, output format and safety, scored by executing the generated code rather than grading it by similarity.
Multi-step problem solving over a multi-turn session, scored on the code axes plus memory retention, hallucination rate, plan coherence and context-window use — usually the first place a quantised model shows decline.
Task completion, tool selection, parameter accuracy, efficiency, error handling, context awareness and safety compliance. Critical for agents, where a small drop in parameter accuracy compounds across a long chain of calls.
Providers publish benchmark numbers at launch. Nobody publishes them continuously afterwards, and no provider announces that a model got worse. Independent, ongoing measurement is the only way to know whether the model you evaluated is the model you are running — which matters for anyone with a model choice baked into production, a vendor contract to justify, or an AI feature whose quality they are accountable for.
Every model we track has its own page with a historical score chart, per-suite breakdown and drift status. Start from the live AI model leaderboard, or read the benchmarking FAQ for shorter answers to common questions.
AI drift detection is the practice of continuously measuring a model's output quality so you can tell when it changes. Because providers serve updated models behind stable API names, the version you call today may not behave like the one you tested against months ago. Drift detection replaces the anecdotal "it feels worse lately" with a measurement you can point at.
Several things: a provider silently swapping in a new checkpoint, changes to safety filters that increase refusals, quantisation or serving-stack changes made to cut inference cost, altered default sampling parameters, and capacity pressure during peak demand. From the outside these are indistinguishable — all you observe is that the same prompt now returns weaker output.
You need a fixed test set, repeated runs, and a statistical baseline. A single bad response proves nothing — model output is stochastic, so quality varies run to run even with no change at the provider. The signal you want is a sustained shift in the average across many runs, which is exactly what change-point detection is designed to isolate.
The Page-Hinkley test is a cumulative-sum (CUSUM-family) change-point detector. It keeps a running total of how far each new observation falls below the running mean, less a small tolerance, and compares that total with its own lowest point so far. Random fluctuations have no consistent sign and cancel out; a genuine sustained decline accumulates until the gap crosses a decision threshold and a drift event fires. That makes it far better suited to catching gradual decline than a simple threshold alarm, which either fires constantly on noise or misses slow degradation entirely. We run it on each model's daily median score, and the constants are published on the methodology page.
The code suite runs every four hours, the reasoning suite (all four tasks) and the tool-use suite (nine sessions) once a day, and a two-probe canary every hour. The Page-Hinkley statistic is computed per suite on daily medians, so it confirms a sustained shift over days rather than reacting to one bad hour; the canary's Welch test catches a sharp drop within the hour it lands. If a suite could not run — a provider outage, or a prompt the provider declined — that suite is left out of the score and the gap is shown next to it rather than filled in.
Current models across OpenAI, Anthropic, Google, DeepSeek, Moonshot AI and Zhipu AI — including the GPT-5 series, Claude Opus and Sonnet, Gemini, DeepSeek V4, Kimi and GLM. Every model in the leaderboard has its own page with a historical performance chart.
All benchmark data is free to browse — no account needed. Compare current models side by side, or open any model to see its full performance history and drift status.
VIEW LIVE MODEL RANKINGS →