Reasoning Models Explained: What Test-Time Compute Actually Costs You

For a decade, the way you made a language model better was to make it bigger. More parameters, more data, more training compute. That dial still works, but in 2024 the field found a second dial, and it points in a different direction: instead of training a bigger model, let the model you already have think for longer before it answers.

That is what "test-time compute" means, and "reasoning models" are the models built to use it. The concept is simple. Whether you should pay for it is the part that actually matters, and almost nobody explains that part honestly, so let me.

What test-time compute actually is

A standard language model answers in essentially one pass. You give it a prompt, it produces the next tokens, and whatever it commits to early it is mostly stuck with.

A reasoning model is trained, usually with reinforcement learning, to do something different first: generate a long internal working-out, a chain of thought where it tries an approach, checks it, notices a contradiction, backtracks, and tries again, before it writes the final answer you see. The "test-time compute" is exactly that working-out. The model is spending processing at inference, the moment you query it, rather than relying only on what training baked in.

The insight underneath the trend is that for some problems, letting a model deliberate produces answers that a bigger model answering in one pass simply cannot reach. Thinking longer beats thinking bigger, on the right kind of task.

This is real, and here is the receipt

Healthy scepticism is the correct starting position for anything in AI, so here is the concrete evidence rather than a vibe.

In January 2025, DeepSeek released R1, and its research showed that pure reinforcement learning, with no supervised examples of good reasoning, could teach a model to reason. On the AIME 2024 mathematics benchmark, pass@1 accuracy rose from 15.6% to 71.0%, and climbed to 86.7% with majority voting, matching OpenAI's o1. The work was later peer-reviewed and published in Nature, which is not a thing that happens to hype.

DeepSeek-R1 mattered for a second reason too. It proved the capability was not the private property of one American lab. The weights were open. OpenAI's o-series (o1, then o3) is the commercial lineage of the same idea. Within a year, "let the model think" went from a single product feature to a general technique anyone could use.

So the capability is genuine. Now the part the benchmark charts leave out.

You pay for the thinking, every time

The chain of thought is made of tokens, and tokens cost money and take time to generate. A reasoning model working through a hard problem can emit many times the tokens of a standard answer before it says a word to you. That has two consequences you feel immediately in production.

Cost. If you pay per token, the internal deliberation is on your bill even though the user never sees it. A reasoning model is not a little more expensive than a standard one. On a hard query it can be several times more expensive.

Latency. All that thinking happens before the first useful token comes back. For anything interactive, a chat box, a support widget, a live tool, the delay of a full reasoning pass is often too long to be usable. For batch and overnight jobs, the same delay does not matter at all.

The decision: when to pay the tax

Here is the framework I would actually give a junior engineer.

Use a reasoning model when the task is genuinely multi-step and a wrong answer is expensive. Mathematical or scientific problems, non-trivial code, planning, multi-constraint analysis, anything where the model needs to hold several things in its head and check its own work. This is where the extra compute earns its cost.

Do not use one for generation, classification, or extraction. Writing an email, summarising a document, translating text, tagging a ticket, detecting sentiment, pulling fields out of a PDF. On these, a standard model gives you comparable quality at a fraction of the price and a fraction of the wait. Paying for deliberation here buys you nothing but a slower, dearer version of the same answer.

Let latency decide the rest. If a human is waiting on the response, lean standard and reserve reasoning for the one hard step. If it is a background job, the latency is free and you can afford to let the model think.

Why this matters more on a tight budget

If you are building somewhere compute is metered and every token is real money, in Freetown, Accra, or any team without a hyperscaler's budget, this is not an academic distinction. The reasoning tax is the difference between a service you can afford to run and one you cannot.

The good news is that the frugal choice and the correct choice are the same choice. Default to the small model. Escalate deliberately. Treat inference budget as an architecture decision you make on purpose, not a surprise you discover at the end of the month. The same discipline that lets you ship on hardware you can actually afford is the discipline that uses reasoning models well.

What would make me wrong

Benchmarks are not your workload. A model that tops AIME may or may not help with your particular problem, and the only way to know is to test it on your own task with your own cost and latency budget, not to trust a leaderboard. The field is also moving fast: hybrid models that decide for themselves how long to think are blurring the clean line between "reasoning" and "standard," and the routing may increasingly happen inside the model rather than in your code. I have deliberately not quoted you specific per-second latencies or per-query prices, because they differ by model and version and would be stale by the time you read this. The shape of the tradeoff is stable. The exact numbers are yours to measure.

Key takeaways

1. Test-time compute means a model thinks longer at inference, generating an internal chain of thought before answering.
2. It is a real capability, not hype: DeepSeek-R1 lifted AIME accuracy from 15.6% to 71.0% with pure reinforcement learning, and it was published in Nature.
3. You pay in tokens and latency on every query, whether or not the task needed thinking.
4. Use reasoning for hard, multi-step, high-cost-of-error problems. Use standard models for generation, classification, and extraction.
5. Route, do not default. Cheap model first, escalate only the genuinely hard queries. On a tight budget this is both the frugal choice and the right one.

What is one task in your system where you are paying for reasoning it does not need? That is usually the fastest cost saving in the whole pipeline.

---

Sources: DeepSeek-AI, "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning" (arXiv

.12948), and the peer-reviewed version in Nature (nature.com/articles/s41586-025-09422-z), for the AIME 15.6% to 71.0% (86.7% majority-vote) result and the pure-RL method. OpenAI o1 and o3 as the commercial reasoning-model lineage.