test-time-compute-scaling
verifiedc3ebbd7f-885c-4ea7-948a-1054f758f097
Scale reasoning quality at inference — chain-of-thought, self-consistency, best-of-N, verifier/PRM-guided search, and MCTS — with the sample-efficiency and cost tradeoffs of each.
Metadata
Skill file
# Test-Time Compute Scaling for Reasoning
Use when accuracy on hard, verifiable tasks (math, code, structured reasoning)
matters more than latency/cost, and you have an inference budget to spend. Test-time
compute (TTC) trades extra inference for better answers without retraining the model,
and is the mechanism behind reasoning models' strong results. It works best when
outputs can be *ranked or verified* — either automatically (a checkable answer, tests
passing) or by a reward model.
## The family of techniques, from cheap to expensive
- **Chain-of-thought (CoT)** — the baseline. Just prompting step-by-step reasoning
(or using a reasoning-tuned model) gives the model room to think. No aggregation.
- **Self-consistency / majority voting** — sample N reasoning paths (temperature > 0)
and take the majority final answer. Robust when answers are discrete and the model
is more-often-right-than-wrong.
- **Best-of-N (BoN)** — sample N, then score each with an *external verifier* or
reward model and return the best. Needs a reliable scorer; more sample-efficient
than majority voting when answer probabilities are close (sample complexity
Θ(1/Δ) vs Θ(1/Δ²) for self-consistency, where Δ is the probability gap).
- **Verifier-guided search** — use a process reward model (PRM) that scores
*intermediate steps* (not just final answers) to prune and expand a tree of
partial solutions. This is how models get big jumps on math (AIME, MATH) — a
step-level reward lets you abandon bad branches early instead of wasting N full
samples.
- **Monte Carlo Tree Search (MCTS)** — build a search tree over reasoning steps,
balancing exploration/exploitation with UCB. Used in o1-style and open
reproduction efforts; most valuable when the step space is large and a PRM or
value head exists to score nodes.
## Key tradeoffs and gotchas
- **Diminishing returns.** Every method hits a plateau; the ICLR-2025-oral result
("Scaling LLM Test-Time Compute Optimally") showed *compute-optimal* TTC scaling
can beat a 14× larger model on problems where the small model already has non-trivial
success — but only if you allocate samples by *difficulty* rather than a fixed N.
- **Fixed N wastes compute.** Easy questions don't need 16 samples; hard ones need
more. Difficulty-aware/adaptive allocation (sample more when votes are split) is
the highest-leverage optimization.
- **Verifier quality is the ceiling.** BoN and PRM-search are only as good as the
scorer. A noisy verifier rewards wrong answers confidently. PRMs need step-level
supervision data, which is expensive to label.
- **Verifier bias / reward hacking.** Verifiers over-optimize toward their own
quirks; cross-check with a held-out metric where possible.
- **Cost multiplies.** BoN and self-consistency multiply tokens by N. Budget
accordingly — cache shared prefixes, and consider cheaper models for the sampling
stage and one strong verifier for selection.
## When each method wins
- Answer is a short, checkable token (code that runs tests, a number): **BoN** with
a real checker (test harness) — the strongest signal you can get.
- Answer is free-form and no verifier exists: **self-consistency** (majority) only
helps if the model is >50% right; otherwise it can *hurt*.
- Multi-step reasoning where intermediate correctness matters: **PRM-guided search**
or **MCTS**, accepting the engineering and labeling cost.
- Latency-sensitive production: keep N small, or fall back to a single strong
reasoning-model call — TTC is a batch/offline lever more than a real-time one.
## A minimal best-of-N loop (pseudocode)
```
def best_of_n(problem, n, checker, sample):
candidates = [sample(problem, temperature=0.7) for _ in range(n)]
scored = [(checker(c), c) for c in candidates] # checker: pass/fail or score
return max(scored, key=lambda s: s[0])[1]
```
## Pitfalls
- Using majority voting when the model is <50% accurate — it degrades instead of helps.
- Trusting a verifier blindly; verify the verifier on a labeled set before relying on it.
- Spending N samples uniformly on trivial queries — the biggest silent cost sink.
- Ignoring that TTC latency/cost scales linearly in N — not a drop-in for interactive use.
## Verify
- On a held-out set of ~50 hard problems, compare accuracy vs a single greedy call at
N=1, 4, 16. Confirm accuracy rises and plateaus, and note the cost/accuracy knee.
- If using a verifier, report its own precision on a labeled subset first.
- Confirm your scoring signal (tests, answer check) actually correlates with what you
care about before scaling N.Attached files
No attached files.