The Model Got Smarter.
Your Ceiling Didn't.

The model, and what its own card doesn't say

Gemini 3.8 Flash was published on 2 September 2026, model code gemini-3.8-flash, built on 3.7 Flash. It takes 1,048,576 input tokens and returns up to 65,536. Thinking is supported at low, medium and high. There is no audio or image generation, no Live API, and computer use is in Preview. Its self-reported knowledge cutoff is March 2026.

That is the specification. Two smaller things turned up while measuring it, and both are the kind of detail that only appears if you call the API rather than read about it.

The minimal thinking level did not change at 3.8

The documentation reads as though minimal was dropped for this release. It was not. Measured across all three models:

gemini-3.6-flash   thinkingLevel=minimal   ACCEPTED
gemini-3.7-flash   thinkingLevel=minimal   HTTP 400
                   "Thinking level MINIMAL is not supported for this model"
gemini-3.8-flash   thinkingLevel=minimal   HTTP 400  (identical message)

The change landed at 3.7. If you are still passing minimal anywhere, it has been failing since the previous release, not this one — and if your code treats a 400 as a transient error and retries, it has been paying for retries that cannot succeed.

The version string is inconsistent

Ask each model to identify its version and the older two answer in a stable format. The new one does not:

Cosmetic on its own. It stops being cosmetic if anything in your stack routes, logs or gates on that string.

What the launch page publishes, and what it doesn't

Four benchmark claims appear on the model's page. One carries a number.

There is no comparison table on the page. That is not an accusation — marketing pages are marketing pages — but it does mean that if you want to know whether 3.8 is better for your workload, the published material cannot tell you. Which is the entire reason this class exists.

Three identical numbers are not three findings

The first battery ran seven phases against all three models. Here is what four of them returned.

It is tempting to write that up as “3.8 matches 3.7 on structured output and coding”. That sentence is true and worthless. The tests could not have produced any other result. An instrument that returns the same reading for every input is not measuring the input — it is measuring itself, and reporting its own constant.

Four identical rows are not four findings. They are one failure of instrument design. The correct response is not to publish them as a result; it is to raise the difficulty until the instrument can distinguish, and to keep raising it until something separates.

Which of these is a measurement?

Work these before reading on. Each is a real result from this research, and the question is the same each time: could this test have told you the models differ, if they did?

caps.json — one call per model

schema.json — n=12 per model

coding.json — 10 tasks per model

thinking_gap.json — 5 tasks x n=2

thinking.json — 4 levels x 3 models

coding_hard.json, then confirmed at n=5

n=8 said twice as slow. n=20 said no.

The latency phase ran first, because latency is easy to measure and everybody wants the number. At eight samples per model it produced a clean, quotable, wrong headline.

Nothing changed between those two rows except the number of samples. The n=8 run was not corrupted, mistimed or unlucky in any way that could have been spotted at the time. It contained a 5,087 ms outlier — a ten-fold spread across eight calls — and with eight samples one outlier moves the median.

This is the ordinary way benchmark claims go wrong. Not fraud, not incompetence — a sample too small to see past the tail, run by someone who stopped as soon as the result looked interesting. The defence is boring and reliable: decide the sample size before you look at the result, and for anything timing-related assume the distribution has a tail until you have seen enough of it to know otherwise.

Which number describes what a user experiences?

four figures, all arithmetically correct

the ceiling has to cover something

the token counts are recorded

Raise the threshold until it can fail

After the easy battery returned 10/10 three times, the choice was to publish “no measurable difference” or to build a test that could actually fail. The second battery kept the same shape — ten tasks, code returned in a fenced block, graded by executing it — and changed one thing: each task carries between four and eight adversarial vectors, and passes only if every vector passes.

The vectors are the cases people actually get wrong. Empty input. A pattern that requires backtracking. A quoted comma. A doubled quote inside a quoted field. Banker's rounding on a negative tie. A tie in a longest-palindrome search where the leftmost must win. A pre-release version suffix that sorts before the release it belongs to.

easy battery      3.6  10/10     3.7  10/10     3.8  10/10
hard battery      3.6  10/10     3.7  10/10     3.8   9/10   <-- h08_version_full

One task, on the newest model. Which is exactly the result most likely to be an artifact, so it was not reported until it had survived two more methods.

Second method: does it reproduce?

same prompt, same grader, n=5 per model

gemini-3.7-flash    5/5 PASS
gemini-3.8-flash    0/5 PASS

Reproducible, and not marginal. At that point the failure is real. What it means was still unknown, and the obvious interpretation — that 3.8 is worse at comparing version strings — turned out to be wrong.

Claim against measurement

Six statements. Each pairs something it would be reasonable to believe with what the runs actually recorded.

Capability, or budget?

A model fails a task. There are two explanations, and they call for opposite responses:

• Capability limit — it cannot do the work. Change models, change the prompt, or do it another way.
• Budget limit — it can do the work but ran out of room. Change one number in your config.

They look identical from the outside. Both produce a missing answer. And there is exactly one way to tell them apart: raise the allowance and change nothing else. Same prompt, same grader, same vectors, same model. One variable.

It is a budget limit. 3.8 can compare version strings, including the pre-release ordering rule that makes the task awkward. It simply could not do it inside an allowance that 3.6 and 3.7 both fit comfortably.

The tail, not the centre

Having established that on one task, the obvious next question is whether it generalises. Five more hard tasks were run against all three models at the same 6,144 ceiling, recording thinking tokens on every call.

Read the medians alone and you would write “3.8 thinks a bit more than 3.7, less than 3.6”, which is true and useless. The distributions differ in their tails, not their centres. 3.7 never exceeded 2,679 thinking tokens on any task tested. 3.8 reached 5,894 — against a ceiling of 6,144.

None of which means “3.8 truncates”. One task in six did, at n=2 to 3. That is a signal, not a rate. What it does mean is that the variable you never tuned — the ceiling you set once, against a different model, probably by copying an example — is now the thing most likely to break, and it will break on your hardest inputs rather than your typical ones.

HTTP 200. STOP. Zero bytes.

Here is what the two failing calls actually returned. Both are successful HTTP requests. Neither raised an exception. One of them is detectable and one is not.

The second failing call at least set finishReason: MAX_TOKENS and returned 242 bytes of a half-written function — ugly, but detectable, and a parser would have choked on it loudly. The first is the dangerous one:

HTTP status              200
finishReason             STOP
candidatesTokenCount     7
thoughtsTokenCount       5,898
parts[0].text            ""        <-- length 0

Every check most code performs passes. The request succeeded. The model reported that it stopped normally. There is a candidates array, a content object, a parts array. It is all there. It is just empty.

Anatomy of the response you actually get

A generateContent response from the failing callHTTP status?finishReason?candidatesTokenCount?thoughtsTokenCount?parts[0].text?The exception you were hoping for?

What to actually check

The gauntlet below is the order the checks have to run in. Each gate lets some failures through; the point is which ones, and what is left standing at the end.

Note what gate five is for. It never fails a response; it fails a configuration. If your hardest real prompt is consistently spending 5,894 of a 6,144-token allowance, you do not yet have a bug — you have four percent of headroom and a bug scheduled for whenever your inputs get slightly harder.

Same price. More thinking.

The last piece is the one that turns a curiosity into a budget line. Read from the pricing page on 4 September 2026, Standard tier:

Identical to the cent. And the phrase “output price (including thinking tokens)” is not our gloss — it is the column heading on the pricing page. Reasoning is billed as output.

Put the two facts together. Same list price, and on hard problems 3.8 spends up to 2.2× 3.7's worst-case thinking. On the tasks that matter most, the identical advertised rate buys you a larger bill for the same work — and, if your ceiling is unchanged, a chance of no work at all.

Size your own ceiling

Fill in what you measured. The output is a guard you can paste into your own code: it sizes the ceiling from your observed worst case, and it fails loudly on the response that otherwise arrives as a successful, empty call.

What we would change tomorrow

1. Re-run the hardest real prompts at the current ceiling before switching models. Not the test suite — the actual worst inputs, the ones that already make the model work. Those are the only ones that can show this.
2. Log thoughtsTokenCount in production. It is free, it is on every response, and it is the only field that tells you how close you are running to the edge. Alert on the ratio, not on failures.
3. Assert a non-empty body. One line. It converts an invisible failure into a visible one.
4. Size ceilings from observed maxima with a multiplier, and write down the date and the sample size beside the number, so the next person knows whether to trust it.