Your Agent Sees the Contradiction, Then Answers Confidently Anyway

The skill your eval never measures

Picture a research agent you built for a client. It searches the web, finds a source that says one thing, and its own parametric memory says another. It notices. It keeps going. Five tool calls later it writes a confident final answer that never mentions the disagreement. The answer might even be right. But the uncertainty it found along the way is gone, overwritten by the next step of the trajectory.

This is the exact failure mode a new paper from Johns Hopkins and NYU sets out to measure. The authors call the missing skill epistemic humility: the willingness to recognize, act on, and communicate one's own uncertainty during task execution. Not calibration on a quiz. Not a refusal classifier. A trajectory-level behavior in agents that read tools and write answers over many turns.

The argument is hard to dismiss. Existing agent evaluation is almost entirely outcome-based: it grades whether the final answer is right, not whether the agent flagged the uncertainty it should have surfaced along the way. And as the paper puts it, a trajectory that flags a conflict "receives no more credit than one that confidently commits to an answer." If nothing in your scoring rewards the flag, do not be surprised when the flag stops appearing.

What humility looks like, operationalized

The authors turn an unobservable idea into three observable behaviors, abbreviated ISE:

• Identify: does the agent recognize the conflict during its trajectory? Scored as F1 over intermediate turns by a GPT-5 judge, which also records the turn where recognition first appears.
• Solve: after recognizing the gap, does the agent try to close it with a fresh tool call and finish cleanly? This scores the attempt, not whether the attempt produced the right answer.
• Escalate: conditional on the final answer being wrong on a conflict instance, does the final turn acknowledge any residual uncertainty? This is the honest-admission rate.

Each facet can fail independently, and that is the point. An agent can score well on Identify and still fail Escalate. That combination, recognizing the gap mid-run and delivering a confident wrong final answer, is the paper's central character.

The measurement setup is unusually careful for agent research. Three judges from different model families (GPT-5, Claude Sonnet 5, GLM-5.2) score the trajectories, agreeing with Gwet's AC1 of 0.93 to 0.97 on correctness and 0.82 to 0.92 on Identify and Escalate. Human annotators, blind to the judge labels and agent identities, agree with the judges on 76.0% of Identify turns and 85.3% of Escalate answers. Every conflict instance is paired with a matched no-conflict control. The authors release every trajectory and every per-turn judgment as a public dataset. The experiments cost more than $3,000 and took over 1,000 GPU hours.

Who was tested

Four harnesses, five backbone configurations, each run with a 50-iteration cap across both conflict settings:

Harness	Backbone
Nemotron-ToolOrchestra	Nemotron-Orchestrator-8B
Claude Code	Claude Sonnet 4.6
OpenHands	GPT-5
Qwen-Agent	Qwen3.5-9B
Qwen-Agent	Qwen3.5-27B

Two settings. Controlled conflict uses factual questions from ConflictQA and WikiContradict with conflicts supplied in the task context, either between sources or between the context and the model's memory. Naturally occurring conflict runs the agents on GAIA, MoNaCo, and BrowseComp, where contradictions can surface mid-execution from tool results or the model's own outputs. The naturally occurring split is observational (different questions per split), so the controlled split carries the cleanest causal read.

Finding 1: controlled conflict is a bloodbath

On the controlled-conflict split, where the context hands the agent evidence both supporting and contradicting its memory, accuracy collapses relative to the matched controls:

• Nemotron-ToolOrchestra: 3.0% on conflict vs 39.0% on control
• OpenHands: 3.2% vs 58.4%
• Qwen-Agent (9B): 4.0% vs 65.0%
• Qwen-Agent (27B): 6.5% vs 60.0%

Every reported configuration loses more than 30 percentage points. These are systems that answer the same questions comfortably when the evidence agrees with them, and fall to low single digits when it does not. This matches earlier single-turn findings for bare LLMs, but it is worth seeing the size of the drop inside full agent loops with tools available: the tools do not save them.

Finding 2: accuracy and humility run in opposite directions

Here is the finding that should change how you read leaderboards. Plotting conflict-split accuracy against the humility metrics across agent-dataset cells, the fitted curves decline: Solve drops from near 45% at low accuracy to under 10% at high accuracy, and Escalate from near 38% to under 10%. Identify shows no clear monotone relationship. In other words, the better the configuration is at the task, the less it tries to resolve conflicts or warn the user about them.

The individual cases make it concrete. Claude Code reaches an Identify F1 of 90.0% on BrowseComp, the best conflict recognition in the study, yet its incorrect final answers often do not acknowledge the unresolved uncertainty. OpenHands hits 51.0% accuracy on the GAIA conflict split but, in the authors' words, "tends to ignore the conflict for the instances where it gets wrong." The strongest agents see the problem most clearly and are the least likely to tell you about it.

The paper's proposed mechanism deserves a slow read. Agents are trained and scored on outcomes, and "providing an uncertain answer could still have the possibility to get it correct, but escalating to the user will be graded as incorrect." An uncertain answer is a gamble with upside; an escalated answer is a certain zero. Any optimizer facing that payoff structure learns the same lesson: commit, do not flag. Two agents can tie on benchmark accuracy and differ completely on whether they will warn you, and the benchmark cannot tell them apart.

Finding 3: the conflict is spotted early, then dropped

Where in the trajectory does recognition happen? Conflict-relevant answer mentions peak in the first 10% of execution and then fall sharply, with a long tail of additional steps. The agent reads the contradiction, registers it, and keeps going. The follow-up never arrives.

This is the most actionable pattern in the paper. The problem is not that agents fail to detect the gap; detection is the one thing they do early and reliably. The problem is that the doubt has a half-life measured in turns. Each new tool call, each new observation, overwrites it. The paper's term for the compatibility here is simplicity bias: the trajectory settles toward the easy answer and the early complication quietly disappears.

Finding 4: one prompt clause buys humility at the price of accuracy

The authors test the most optimistic explanation: maybe the backbone simply lacks the behavior, and a nudge fixes it. They prepend one clause to each system prompt asking the model to consider alternative readings of the evidence, name unresolved conflicts, and report its humility scores. Tools, planning loop, and weights stay untouched. The clause is a minimal test, not a full solution.

It works, and it costs. On MoNaCo, Escalate rises from 1.6 to 60.7 for GPT-5, from 13.9 to 60.8 for Claude Code, and from 16.7 to 44.9 for Qwen-Agent (9B), while GPT-5's accuracy falls from 30.1 to 23.2. On BrowseComp, GPT-5 gains 28 points of Escalate and loses 6 points of accuracy. Most configurations land in what the paper calls the humble-but-inaccurate quadrant: higher escalation, lower task score.

That trade-off is the paper's real thesis. The backbones can express uncertainty when asked, but the behavior "emerges from the interaction among the backbone model, agent harness, and evaluation environment." There is no single component to patch. A prompt buys you humility at retail price; fixing the harness and the scoring is the wholesale route.

What this means for data and AI practitioners

1. Judge trajectories, not just final answers. Log where conflict-relevant information appears in your agent's runs and check whether it survives to the final answer. If mentions peak early and vanish, you have the exact pattern in Figure 4.
2. Give the agent an explicit escalate or abstain action. The paper's first design recommendation: a first-class action so a doubt raised mid-trajectory survives into the final answer instead of being overwritten by the next tool call. A system prompt is a bandage; an action is architecture.
3. Score uncertainty-awareness in your own evals. A confident wrong answer should not systematically outscore an honest admission. The Identify/Solve/Escalate trio is a concrete starting point, and the released trajectories plus per-turn judgments are ready-made signal for process supervision.
4. Prompt for alternatives where stakes are high, and measure the price. The one-clause nudge works, and every configuration paid in accuracy. For a customer-facing research agent, that trade can be worth it. Know the exchange rate before you commit.
5. Stop reading leaderboards as trustworthiness rankings. Same accuracy, different escalation behavior: the benchmark cannot distinguish an agent that warns you from one that papers over the dispute. Ask vendors for trajectory-level honesty metrics, not just task scores.

In favour

The design is unusually honest for agent eval work: matched no-conflict controls, three judges with reported agreement, blinded human validation, and a full public release of trajectories and per-turn judgments. The inverse accuracy-humility curve is fitted across cells with bootstrap bands, not drawn from one anecdote. The result extends a consistent prior literature from single-turn LLMs (Kadavath et al. on suppressed self-knowledge, the ConflictQA and WikiContradict conflict studies) into multi-step agent loops, and the convergence is reassuring rather than surprising.

Against

The coupled backbone-harness design means the headline interaction claim is argued, not isolated; a factorial study could still pin humility mostly on the backbone. The Escalate metric conditions on the final answer being wrong, so confidently-phrased wrong answers and the style of hedging are entangled in ways the agreement statistics only partly address. And knowledge conflict is the most controllable elicitation channel by design, which means it is also the one where the desired behavior is easiest to define; real deployments mostly live in the channels this paper sets aside, where what counts as humble is genuinely ambiguous.

What would make me wrong

• A backbone-by-harness factorial showing humility tracks the backbone alone, which would break the interaction claim.
• Escalation scored on all instances rather than only wrong ones, if that erases the inverse accuracy-humility curve.
• Human re-judgment of the trajectories showing the judge penalizes a confident writing style rather than measuring real uncertainty acknowledgment.
• A harness-level abstain action that raises Escalate without the accuracy drop the prompt clause caused, which would shrink the trade-off to a design bug rather than a structural one.

Related on Everyday Data Science

• Your Agent's Tests Are Lying to You
• ThinkingBox: Why Your Agent Passes Once and Fails on Repeat
• ReplayBench: The Attack Where Real Tool Results Land in the Wrong Task
• RAG Collapse: When Your Retriever Quotes Your Own Model Back at You

When was the last time your agent told you it was not sure? If you cannot answer that from your logs, you do not have an eval gap. You have a visibility gap.

Sources: Accurate but Not Humble (arXiv:2610.12360) · full text (HTML) · PDF · trajectories and code · aiweekly coverage · dev.to explainer