Agent Reflection Does Not Beat a Retry. The Numbers From 16,946 Trials.

On September 11, 2026, a team from Zhongguancun Laboratory, Tsinghua University, and China Mobile published a paper that asks a question most agent builders never test. When your agent fails a task, is it better to have it write down what went wrong and try again with that lesson in mind, or to just try again?

The paper is VRL-Bench (arXiv

.12404). It introduces a harness for fair comparison of trial-and-error learning, then runs the comparison: Reflexion and two newer verbal-memory methods against memory-free retry, with every method given the same number of complete trials. The evaluation spans three models (DeepSeek-V4-Flash, GPT-5.4 nano, GLM-4.7-FlashX), two benchmarks (MiniWoB and WebShop), 2,220 paired cases, and 16,946 trial executions. The answer is uncomfortable.

1. Every existing memory method helps somewhere and hurts somewhere else

None of the three existing updates beats memory-free retry in all six model-environment settings. Reflexion posted positive retry-relative gains in 4 of 6 settings. DC-Cu*, an adaptation of Dynamic Cheatsheet, managed 2 of 6. ACE*, adapted from Agentic Context Engineering, reached 5 of 6.

The concrete cases matter more than the counts. ACE*, the strongest of the three, dropped from retry's 50% success rate to 45% on WebShop with GLM-4.7-FlashX. DC-Cu* fell below retry in all three MiniWoB settings. The direction of improvement changes with the model, not just the update: Reflexion improved success rates on both environments with DeepSeek-V4-Flash and GPT-5.4 nano, but lowered them on both with GLM-4.7-FlashX. The same update can be a win or a loss depending on which model runs it.

2. The gains are real, and so are the losses

The paired 95% bootstrap intervals support both directions, so this is not noise on either side. Reflexion gained 16 percentage points on GPT-5.4 nano WebShop (95% CI [+8, +25]), a clear win. But DC-Cu* had negative intervals excluding zero for GLM-4.7-FlashX in both environments, meaning its losses are statistically distinguishable from noise too. A reflection pipeline that helps on your model by 16 points is real. A reflection pipeline that hurts another model is also real. Average over settings and you learn nothing.

3. Replaying reflections can actively reduce success

The authors revisited Reflexion's AlfWorld and HotPotQA experiments with replay-exposure sweeps spanning 3,324 recorded episodes, holding the reflection writer fixed while varying how often its advice reaches the actor. Using reflections can reduce success rates, and different replay schedules recover different failed cases.

One figure in the paper makes the mechanism visceral. In a WebShop task, Reflexion carried a compact failure account forward and stayed unsolved through all six trials, while plain retry explored a different product and succeeded on trial two. The reflection anchored the agent to the wrong route while the fresh rollout found a way through. Under a finite trial budget, failure-derived memory redirects the remaining attempts, and a redirect is not always an improvement. The authors frame this as an empirical exploration-exploitation tradeoff: exploiting experience versus continuing to explore alternatives.

4. The method that won treats exploration as a scheduling decision

The authors propose VEX2, a verbal exploration-exploitation scheduler that uses a language model to jointly select which policies to pursue and how to allocate the remaining trial budget. It was the only evaluated update with positive observed success-rate gains over retry in all six settings: gains ranging from 1.6 to 6.6 percentage points on MiniWoB and from 8.0 to 17.0 on WebShop, with lower censored average trials in every setting.

The per-setting numbers show how it won. On DeepSeek-V4-Flash WebShop, VEX2 solved 16 goals that retry missed while retaining every retry success. On MiniWoB with the same model, it recovered 118 of the 157 shared first-trial failures, compared with 76 for retry. The scheduler does not write better reflections. It decides when reflection deserves another trial and when a fresh exploration is the better bet.

The honest caveat: on GLM-4.7-FlashX, both VEX2 intervals include zero (MiniWoB +1.6, CI [-1.1, +4.4]; WebShop +8.0, CI [-1.0, +17.0]). Positive everywhere on point estimates, statistically distinguishable on four of six. And even in the win column, reflection loses individual cases: on GLM-4.7-FlashX WebShop, VEX2 solved 15 goals that retry missed but missed 7 that retry solved, for a net gain of 8.

What this means for data and AI practitioners

1. Benchmark your agent against memory-free retry with the same trial budget. This is the paper's central demand, and it is cheap to adopt. If your reflection pipeline cannot beat a dumb retry under matched budgets, the memory is theater. Several published methods fail this test on some models.

2. Reflections are not free, and the paper does not price them. VRL-Bench matches trials and interaction opportunities, not tokens or model calls. A reflection pipeline that wins on success while burning multiples of the tokens is a different cost story. Run your own token accounting before you celebrate.

3. Re-run your evals per model. The most consistent result in this study is model-dependence. The same update helped GPT-5.4 nano by 16 points and hurt GLM-4.7-FlashX. A lesson learned on one model is not a lesson.

4. If you keep reflection, treat its influence as a dial, not a switch. The replay sweeps show that how often advice reaches the actor changes which failures get recovered. Schedule the exploitation. Do not just accumulate memory and hope.

Broader context

In favour

This result rhymes with earlier findings. LATS reported similar WebShop performance for repeated ReAct and Reflexion (Zhou et al., 2024). Kapoor et al.'s "AI Agents That Matter" (TMLR, 2025) showed resampling could match elaborate agents on code generation. The paired design here is genuinely strong: shared first trials across conditions, matched trial and interaction budgets, setting-specific 95% paired bootstrap intervals over 64 MiniWoB task-family means and 100 WebShop goals. And the authors tested genuinely recent methods (Dynamic Cheatsheet and ACE, both from 2026), not a strawman version of a 2023 idea.

Against

Coverage is narrow: three models, two benchmarks, six-trial budgets. The harness does not account for token or model-call cost, which is where reflection methods are most expensive. The replay sweeps used separate agent configurations and task sets from the main study, so they establish schedule sensitivity rather than a universal schedule. ExpeL was excluded because it requires successful trajectories. And the headline VEX2 claim, "positive in all six settings," leans on point estimates where two confidence intervals cross zero. On four of six settings it clears the bar cleanly. On two it is suggestive.

What would make me wrong

An independent replication with more models and environments showing Reflexion-style updates consistently beating retry under matched budgets. Evidence that the replay-exposure tradeoff disappears with better reflection prompts or stronger reasoners. A cost accounting showing verbal-memory methods win on tokens as well as trials. Or a larger VEX2 evaluation where the GLM-4.7-FlashX intervals clear zero.

Sources

• VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets, Yu Bai et al., arXiv
  .12404 (2026)
• Reflexion: Language Agents with Verbal Reinforcement Learning, Shinn et al., NeurIPS 2023
• AI Agents That Matter, Kapoor et al., Transactions on Machine Learning Research (2025), https://openreview.net/forum?id=Zy4uFzMviZ

Are you running memory-free retry as your baseline, or are you comparing your reflection pipeline against nothing?