DeepSeek's "1017" refresh of V4.1 Flash is a weights-only update: same architecture, tokenizer, and prompt format, advertised to reduce output length. We benchmarked the old weights before the cutover and the new ones after, with everything else fixed: same problems, seeds, sampling, hardware. Only the weights changed.
Setup
Three suites with exact, machine-scored answers, no LLM judges:
- AIME 2024 (10 problems): hard competition math, the longest reasoning chains.
- MATH-500 (10 problems): everyday math, stratified by difficulty.
- BFCL (20 tool-calling tasks): single calls, choosing between functions, parallel calls, abstention.
Each problem ran three times per arm, 120 requests per arm. We ran the old weights twice back to back to measure our own noise floor; any claim below has to clear that gap.
The update reasons less on hard problems
On AIME, the median response fell from 1,681 to 1,124 output tokens: 21% less reasoning for the same answers. Counted as reasoning text, the drop is 30% (4,841 to 3,378 characters). Both are far outside the noise floor.
The longest chains get cut hardest. Our longest problem went from 3,814 to 1,880 tokens, a 51% cut, while short problems barely moved. The refresh pruned redundant deliberation rather than compressing everything uniformly.
Correctness did not move: 101 correct out of 120 paired requests on the old weights, 100 on the new, and the two flips sit inside the noise floor. On the everyday-math suite the cut was similar, about 27% fewer tokens.
The reasoning-effort dial works now
V4.1 Flash accepts a reasoning_effort budget from 1 to 100. On the old weights the dial was mostly decorative below the default: budget 1 still produced a median of 1,974 tokens on hard problems, and the curve even peaked above the default at effort 10 (2,202 tokens).
The update honors it. At budget 1 the median drops to 790 tokens, 60% below the old model at the same setting, and the worst single chain shrinks from 10,280 to 5,940 tokens.
Accuracy held at every setting we tested. At effort 1 the update still solves 29 of 30 hard problems. If your traffic is cost- or latency-sensitive, reasoning_effort: 25 is now a real option: full marks on both math suites, roughly 20% cheaper than the default.
Tool calling did not change
Every BFCL suite scored identically before and after: 41 of 66 on both sides, same passes, same failures.
| Suite | Old | New |
|---|---|---|
| Simple (single call) | 15/18 | 15/18 |
| Multiple (choose 1 of 2–4) | 9/12 | 9/12 |
| Parallel (same function) | 12/12 | 12/12 |
| Parallel-multiple | 3/12 | 3/12 |
| Abstain (no tool applies) | 2/6 | 2/6 |
Two weak spots show up across models (we ran the same fixtures against GLM-5.3 and Kimi-K3):
- Composing parallel calls to different functions is hard for this generation: every model we tested sits at 3–5 out of 12 on the parallel-multiple suite. Single-call accuracy does not predict it.
- Models rarely abstain. Handed tools that don't fit the question, V4.1 Flash invents a call rather than answering directly, as do the others. Keep the tool list tight, or say explicitly that answering without a call is allowed.
Where it leaves the field
| Model (default effort) | AIME | MATH-500 | Tool tasks | $ / correct hard answer |
|---|---|---|---|---|
| DeepSeek V4.1 Flash (1017) | 29/30 | 30/30 | 41/66 | $0.0032 |
| Kimi-K3 | 28/30 | 30/30 | 39/66 | $0.0181 |
| GLM-5.3 | 26/30 | 29/30 | 40/66 | $0.0108 |
GLM-5.3 writes the tersest reasoning of the three, about half of V4.1 Flash's tokens on the same problems, but pays for it with accuracy on hard problems, and its output price is nearly 3× higher. The refresh extended V4.1 Flash's cost lead: $0.0032 per correct hard answer, down from $0.0037 on the old weights.
What it means for your bill
Pricing is unchanged: $0.50 per million input tokens, $1.50 per million output, $0.05 cached. What changed is how many output tokens the model needs before it answers. On reasoning-heavy traffic, spend per solved task drops roughly 20–30% with no action on your side; the model id under existing requests is unchanged. reasoning_effort: 25 saves another ~20% on top.
Method and caveats
All numbers come from 1,440 requests across eleven arms, run on 2026-10-17 against our production serving stack, paired on task and seed. AIME and MATH-500 are likely in every frontier model's training data, so correctness may run at ceiling; the token findings are unaffected. Tool calls were scored by exact name-and-argument match against BFCL ground truth, never executed. With ten problems per math suite, read exact scores as ±1 and trust directions, which were consistent across every problem.