Agent Accuracy Went Up. Agent Reliability Barely Moved.

Every agent vendor is currently pointing at a rising benchmark line. The pitch writes itself: accuracy is up, agents are production-ready, ship one.

A paper from a Princeton-led team gives you the direct rebuttal, and it is more useful than a rebuttal usually is, because it comes with something you can actually run against your own system.

The finding, in one line: accuracy improved steadily for eighteen months. Reliability barely moved.

What the paper does

The work is by Stephan Rabanser, Sayash Kapoor, Peter Kirgis, Kangheng Liu, Saiteja Utpala and Arvind Narayanan, with an interactive dashboard published alongside it.

The argument starts from a gap practitioners already feel. Benchmark accuracy keeps climbing, and yet agents keep failing in real deployments. The paper's diagnosis is that this is a measurement problem: compressing an agent's behaviour into a single success rate throws away exactly the information that predicts whether it will survive production.

So they decompose reliability into four dimensions and propose twelve concrete metrics across them, deliberately independent of raw accuracy so you can compare reliability between agents of different capability levels. Then they apply the metrics to 14 agentic models across two benchmarks, spanning roughly eighteen months of model releases.

The result: steady accuracy improvements, and only modest overall improvement in reliability. The gap between the two is widening.

The four things a success rate hides

This is the part worth internalising, because it converts directly into things you can test.

Consistency. Does the agent do the same thing twice? Same input, same conditions, same outcome? A benchmark run once tells you nothing about variance across runs. An agent that succeeds 70% of the time is a very different product from one that succeeds on 70% of tasks reliably every time.

Robustness. Does it survive small perturbations? A reworded prompt, a slower API, a field arriving in a different order, a slightly malformed input. Benchmarks are clean. Production is not.

Predictability. When it fails, does it fail in a way you could have anticipated and handled? Predictable failure is designable. Arbitrary failure is not.

Safety, in the sense of bounded error severity. When it goes wrong, how bad is the worst case? An agent that occasionally returns a wrong summary and one that occasionally deletes a table have the same success rate and completely different risk.

The pre-deployment checklist, for teams without an SRE budget

The paper is academic. Here is what I would actually do with it if I were shipping an agent with a small team and no dedicated reliability engineer, which describes most teams I know, and nearly every team I know working in Freetown, Accra or Lagos.

1. Run the same task ten times and measure the spread. Not once. Ten times. If the outcome varies, you do not have a 70%-accurate agent, you have a coin flip with extra steps. This single test costs almost nothing and tells you more than any benchmark.

2. Perturb the input deliberately. Reword the request. Add a typo. Reorder fields. Give it a slightly stale value. If small, realistic noise breaks it, it will break the first day a real user touches it.

3. Log every failure and classify it. You are looking for whether failures cluster into a few recognisable modes (good, designable) or scatter randomly (bad, undesignable). This is predictability, measured cheaply.

4. Cap the blast radius before you cap anything else. Dry-run mode by default. Human approval on anything irreversible, anything that spends money, anything that deletes. Hard timeouts. Spending limits. This is the bounded-severity dimension, and it is the one you can control completely regardless of which model you use.

5. Prefer the boring agent. Given two options, one more capable and erratic, one less capable and consistent, take consistency. You can build around a predictable limitation. You cannot build around a surprise.

That last point is the practical heart of it. When you have one engineer and no 24/7 monitoring, reliability is worth more than capability, because you personally absorb every failure. The frugal choice and the correct choice line up again, as they usually do.

How this fits with choosing a framework

I argued recently that most teams reach for more agent machinery than their problem needs. This paper is the empirical companion to that argument. If reliability lags capability across the board, then adding more autonomous moving parts multiplies your exposure to exactly the dimension that has not improved.

The conservative reading, which I hold: use the least autonomy that solves the problem, measure the four dimensions above on whatever you do deploy, and let the blast-radius controls be the part you over-engineer.

What would make me wrong

Two benchmarks are not the world, and agents evaluated in a benchmark harness are not your agent on your data with your tools; the paper measures a slice, carefully, and I am generalising from it. The metrics themselves are new, and a framework proposed in 2026 will be argued with and revised, so treat the twelve as a well-reasoned starting proposal rather than a settled standard. It is also possible that reliability improves sharply as scaffolding and tooling mature, independent of the underlying models, in which case the lag closes without anything in the models changing. And my checklist is opinion informed by the paper, not a finding from it. The paper measures the gap. What to do about it is my argument, and you should push on it.

Key takeaways

1. Twelve metrics across four dimensions: consistency, robustness, predictability and bounded severity, all independent of raw accuracy.
2. Across 14 agentic models and two benchmarks, reliability gains lag capability gains. Eighteen months of accuracy improvement produced only modest reliability improvement.
3. A single success rate hides the failure modes that break deployments, which is why benchmark-led buying decisions disappoint.
4. Run it ten times, perturb the inputs, classify the failures, cap the blast radius. That is a reliability review a small team can actually do.
5. With a small team, reliability beats capability. Prefer the boring, consistent agent.

Before your next agent goes live: have you run the same task ten times and looked at the spread? That one test changes most people's plans.

---

Source: Stephan Rabanser, Sayash Kapoor, Peter Kirgis, Kangheng Liu, Saiteja Utpala and Arvind Narayanan, "Towards a Science of AI Agent Reliability", arXiv

.16666, with an interactive dashboard at hal.cs.princeton.edu/reliability. Figures quoted (twelve metrics, four dimensions, 14 agentic models, two benchmarks, roughly eighteen months of releases) are from the paper.