• AI

Jex 1.0, Part 1: The Frozen Engine

A frozen 31-billion-parameter model, a leaderboard read wrong, an engine that had never touched a GPU, and a reproducibility audit that turns every speed shortcut into a liability: the first phase of the campaign, from a broken engine to a banked 58.09.

The Frozen Engine and the Number to Beat

The first thing we did was read the leaderboard wrong.

Not badly. Just wrong in the way you always read a leaderboard when you assume the numbers on it are the same kind of thing. There was the reference, “Decider chat · Gemma-4-31B,” at 57.33. There was the leader, Surogate Rune 26B-A4B v3, at 57.44. There was Jev at 57.89, sitting in the middle of the sheet like it belonged there. We spent a day planning against those three numbers as though they measured the same species of thing.

They do not. It took reading the meta blocks to find out, and nobody on the team had read them.

Let me back up, because the benchmark matters more than usual here: its shape is what traps you.

The benchmark is a form to fill in

The Decision Index is a reproduction kit for what it calls typed decision engines. You get a state and a bundle of typed questions: multiple-choice choice primitives, or yes/no noul primitives with explicit criteria. You return one answer per question, with a probability for every option. That is the whole interface. No prose. No rubric. No partial credit for a nice explanation. If you have ever filled in a scantron with a confidence estimate scrawled next to each bubble, you have the idea.

The edition in play at the time was 0.2.1, dated 2026-09-27. Thirty-eight benchmarks, five areas, every score chance-corrected (random guessing is zero, perfect is a hundred) and coverage-adjusted (an unanswered question is wrong before the correction even starts). The area weights are not flat: Arts & Human Taste is pinned at 10%, and the other four split the rest by the square root of their benchmark count: Tools & Automation 18.3%, Retrieval & Classification 20.0%, Knowledge & Reasoning 25.8%, Language Understanding 25.8%. Thirteen benchmarks get a gold star and weigh 1.2 inside their area: MMLU-Pro, BBH, GPQA Diamond, HLE, ANLI, WinoGrande, HellaSwag, BANKING77, CLINC150, BRIGHT, BFCL, API-Bank, ForecastBench. The index is 100 times the weighted mean of the five areas. Fine. That is a scoring document, not a story.

Two things about it are stories.

The first is that the index is mostly knowledge-bound. The benchmarks that move the needle are multiple-choice and classification tasks where the hard part is knowing the answer, not reasoning your way to it. Normally that would just mean “use a bigger model.” But our model was frozen (official weights, no training, no quantisation on the audited path) because the whole point was to beat a field on a fixed artifact. A frozen model cannot learn a fact. It can only change how it reads out the facts it already has. That is a narrow door, and it turned out to be the only one open.

The second thing is in the scorer’s source. This is the part we should have seen on day one:

def prediction(q, a):
    return a["choice"] if q["type"] == "choice" else a["noul"] >= 0.5

The scorer takes choice verbatim. It never looks at probabilities on a choice question. Which means the mapping from model output to answer is entirely ours to write, and validate() does not demand that choice be the argmax. A threshold is a decision. It is not filtering, it is not rendering, and the rules do not forbid it. Hold that thought; it becomes a free +0.47 index and then tries to become a free +0.62 and gets caught.

The number we had to beat was a piece of code

Here is the meta block for the 57.33 entry, straight from the board’s index.json:

{ "engine": "decider-chat-gemma4-31b",
  "meta": { "kind": "inference technique",
            "trained_bytes": null,
            "base_model": "google/gemma-4-31B",
            "served_checkpoint": "google/gemma-4-31B-it",
            "code_url": "https://github.com/Mapika/decider" } }

trained_bytes: null. kind: "inference technique". The 57.33 is not a model. It is a method, running on the exact checkpoint we were loading, with its source on GitHub.

That reframes everything, and it does it in our favour. We are not racing a trained artifact for the frozen crown; we are racing a public recipe on identical weights. The recipe is readable. The gap that matters is method versus method.

The leader is the opposite case. Surogate Rune 26B-A4B v3 has kind: "full fine-tune" and trained_bytes: 52173413928: fifty-two gigabytes of training to buy the 0.11 points from 57.33 to 57.44. We are not going to buy that with inference. The gap above the leader is somebody else’s problem. The gap below it is ours.

So we read the recipe. Mapika/decider, commit 23579f7. The prompt is:

Context: <state>
Question: <text>
Options:
(A) <option>
(B) <option>
Answer: (

It reads the letter at that final ( token, one forward pass, no generation. Its decision rule is a bare argmax: j = max(range(len(p)), key=p.__getitem__). Its only threshold is abstain_below, default 0.0, so choice is never None. Its only calibration is a scalar temperature, which is argmax-preserving, so it cannot change a choice or a score. The reference leaves the decision rule, option-order debiasing, prior correction, and any second pass sitting on the table.

That is the good news. The uncomfortable corollary is that our own renderer was byte-identical to the kit’s transformers engine, not to the reference’s. The 57.33 we had to beat came out of a different prompt format. Nobody had scored our format. We had been carefully validating ourselves against the wrong ruler.

The rules we could not bend

Before spending anything we wrote down the constraints. They come from the kit README and from how the maintainers behave, and they come back around every time a hypothesis dies:

1. Determinism, at the answer level. The maintainers re-run the engine on a private held-out sample, one request at a time, “the fastest path the model’s code supports,” and compare answers. A non-deterministic engine is a rejection, not a lower score.
2. Latency. Median per-request latency over 1,000 ms on one RTX PRO 6000 and the model is not added to the board: “at that speed they are no longer Jev-like.”
3. One rendering, no truncation, no option filtering. An engine that cannot fit a request raises Unsupported and the row counts as wrong. Unanswered is wrong before chance correction.
4. Data discipline. No suite rows in training, because the suite has no training split: the kit’s adapters stamp every row test, evaluation or evaluation-only by construction. Outputs go to a private dataset.

The hardware is one RTX PRO 6000 (96 GB, Blackwell, sm_120). Multi-GPU is not a submission path, because the maintainers reproduce on a single card. It is an accelerator for experiments, and only after a cross-host gate says so.

There was also a deadline shape we nearly missed. The board’s 0.2.1 submissions were paused by the maintainer on 2026-09-28 (PR #21) while v0.3 (private held-out benchmarks, a penalty for public-to-private gaps) was built. No rush to submit. The freeze is a runway: bank something solid, then improve it with general mechanisms that survive the move to v0.3.

What the gap was worth, and what the ceiling was

We rebuilt the kit’s aggregation from source and got 57.3294 against the published 57.33. Close enough to trust. Then the sensitivity, which is where the campaign’s ambition gets calibrated. A uniform +1 percentage point of raw accuracy is worth +1.393 index. The old rank-1 target of 57.70 was just +0.37 away. And if four target benchmarks were driven to skill 1.0 there was +5.767 on the table. The gap was 6.4% of the headroom. Small.

Then the directive arrived and changed the size of the problem. Beat Clef (Cloudflare): > 61.21, a target set on edition 0.2.1, before the board moved. At the time that was +3.1 over the 58.09 we would bank, +3.9 over the reference. The readout was not going to get us there. We close that whole frontier later in this part, with data. The one legal door left was to let the frozen model think when it was unsure and re-read its answer with the thought in front of it.

The ceiling for any of this is fixed by the banked run. Over 240,442 questions we had 37,148 errors, worth +32.88 index if every single one were fixed. Everything after this part is an attempt to convert some measured slice of that.

The reason this is a reproducibility story in a performance costume

Everyone wants to hunt score. Our actual spine is reproducibility, and the reason is mechanical.

The moment the engine generates tokens (which is the entire premise of thinking) it opens a numerical surface that did not exist before. Greedy decoding at temperature zero is supposed to be deterministic. It is not, across batch sizes. The culprit is not atomics; it is the reduction order inside RMSNorm, matmul and attention kernels, which changes with the batch shape. Perturb the last bits of the logits and a long autoregressive chain turns that into a different token a few steps later.

On a normal benchmark, a rounding error. Here, a rejection, because the maintainers batch it their way and compare. Which fuses the two questions into one: the only speedups worth having are the ones that leave the answers identical under a different batch shape. That is why the decisive engineering of this campaign is not a clever reasoning trick. It is a serving-engine flag that makes batch 1 and batch 32 produce the same tokens. Everything else hangs off that.

We did not know any of that yet on day one. On day one we had an engine that passed all its tests, a reference we had misread, and a plan that assumed the scorer could not be gamed by a threshold. Two of those three turned out to be wrong.

Where this leads (0.3 revision note): the 61.21 target above belonged to edition 0.2.1. The campaign ran on to edition 0.3, where the frozen stack finished at a 60.29 public index, the best of all no-training entries, submitted complete and re-runnable bit-exact. Parts 1–3 tell the story of the first phase; the resolution is in Part 4.

The two that were wrong had been wrong the same way: we believed them because we had not run them. So before the next launch we ran them: the engine against a real GPU, and the kill list against actual evidence. That is the ledger of lost levers, and it starts with the one test nobody had written.

The Ledger of Lost Levers

Seventy-six tests. All green.

We were proud of that for about four hours. Then somebody wondered why the first real GPU run had never been part of the plan, and we went looking, and it turned out the engine was, in the code path that actually touches a GPU, completely broken.

The torch forward pass scored option rows by indexing rows_lp[predict[cursor + j]]. For one-token options the predicted-token list is [0]. So the second option of every question raised:

IndexError: list index out of range

Why did 76 tests pass? Because every one of them ran through MockBackend, which keeps its own private implementation of scoring and never calls _pack, _forward, _inner or _full. The GPU path was 100% untested. The plan’s first real-backend test had been a paid 200-row run on a 62 GB model, which is precisely how a defect like this survives a test suite. We wrote a CPU test instead: _pack → _forward → score against a tiny random causal LM, in seconds, offline.

Then we ran it, and got seven defects instead of the five we had written down. Two of them would not have crashed. They would have answered differently:

• The fp32 path normalised over candidate rows instead of the vocabulary. That changes the argmax.
• logits_to_keep was accepted by the Gemma-4 text tower and then ignored, so fp32 hidden states were never actually sliced.
• Hugging Face handed a provided 4-D mask back as-is, so our block-diagonal mask silently disabled the sliding window on 50 of 60 layers. Wrong hidden states, every question, no error message.

The KV-cache bug went the other way for once. We built a bare DynamicCache() where we needed a config-aware one, and our first estimate of the damage was ~14.6 GB versus ~1.7 GB at long context. We were wrong by 3.4× in the alarming direction: measured, it was 55 GiB bare versus 5.78 GiB config-aware at 65,536 tokens, because the estimate had assumed full-attention layers use head_dim=256 when they use 512. We had made an inference instead of reading the config. The project now has a rule about this, and the rule is: no claim about the engine without a file:line or an execution. Ten of the ten substantive errors so far were inferences a grep would have settled.

After the repair, the equality check passed on a tiny local Gemma-4 with max |diff| = 9.54e-07 against a plain forward, for the answer row and multi-token option rows, in both precision paths. 92 of 92 checks, on CPU, no GPU. Now the engine read the logits it claimed to read.

The kill list was mostly vibes

Before we spent a cent on experiments, we inherited a list of sixteen “dead” techniques. Somebody (us, in an earlier session, with more confidence than data) had already ruled them out. An adversarial audit went through them by evidence type and found that only six were supported by mathematics or by an execution. The other ten were mechanism arguments, assumptions, or “the docs say so.”

Three of the kills had no artifact at all. Mid-depth readout, cascades, NVFP4: cited with the same tone as measured verdicts, but nothing you could reopen. description_channel was flatly wrong on two counts: it is a second forward pass, not a free one, and the theorem cited to kill it doesn’t apply. And ACOS (positive rate 0.54%, the most misplaced decision boundary in the entire suite, 2.04 index of headroom) had never been measured. It was deprioritised because a paper about a different task didn’t mention it. We read absence of literature as evidence of low value. That is the same mistake that had just eaten the iSarcasm verdict.

The new instruction was blunt: re-examine before trusting, prefer the stable rule over the higher-scoring one, and put ACOS in the next readout pass. We mostly followed it.

The biggest lever in the project, and the bit it died on

iSarcasm looked like a small target. Its F1 metric sat on a score that ranked things well; the boundary was in the wrong place. Plain argmax at 0.5 selected 505 rows and scored F1 0.5078. The probabilities were saturated: 63.3% of rows sat below 0.1 and 34.9% above 0.9, and in that top band the mean probability was 0.998 while the actual positive rate was 36.5%. The ordering had signal. The values were lying.

So we moved the cut. A fixed scalar threshold:

threshold	selected	F1
0.5 (argmax)	505	0.5078
0.999	428	0.5637
0.99999	325	0.6210
0.999999	289	0.6585
0.99999999	220	0.7095

At iSarcasm’s sensitivity of 0.0314 index per raw point, +15.1 points is ≈ +0.47 index. For zero forward passes. More than the entire gap to rank 1 at the time.

Two things tried to stop it. The first was our own arithmetic: the script multiplied a fraction by a sensitivity expressed in index per percentage point and printed a number 100× too small. Caught by reading the output instead of trusting the script. The second was real, and only showed up because we looked at the ranking below the cut. The apply rule was “best key with probability ≥ τ, else the global argmax.” Once τ is above 0.5, the global argmax is the positive, so the fallback returns “yes” for every row below the cut and the whole lever is a no-op. The fallback has to be the best non-positive key. That one correction turned the lever on. None of the analyses that only looked at the ranking could have seen it.

Then the collapse, which is the part worth keeping.

The rows separating the cut were things like 0.999999997269 and 0.999999997183. The gaps are about 1e-10. fp32 carries roughly seven significant digits. We had selected on an ordering the model could not represent. A perturbation test at the fp32 noise scale made it clean: the curve held flat with standard deviation exactly zero up to k=289, and only the last step to k=220 rode on noise. So the real lever was +15.1 (fixed τ) or +16.4 out-of-fold, not the +20.2 headline. Still real. Still sample-independent, which is why the shippable form is a frozen τ and not a frozen count. Worth about +0.474 index.

The lesson was bigger than the lever. An independent replication, a cross-fitted estimate, and a legality analysis had all blessed the result, and all three had shared the same float64 serialisation. None of them asked whether the numbers were resolvable. A result that lives on the last bit of a bf16 logit is not a result, and the test that shows it is four lines of np.spacing.

We spent ten euros to close seven doors

The audit said fund the cheap tests rather than argue about them. So we did. Seven arms over the A0/A1/A2 readout panel (6,111 rows across GPQA-Diamond, iSarcasm, HLE and HoVer), each one the baseline plus exactly one option: mid-depth readout at layers 20, 30, 40, 50 of 60; word keys instead of label keys; a space-letter slot; a key-addressed rendering.

First we re-ran the baseline to make sure the comparison meant anything. It came back bit-identical across all 6,111 answer slots. Good. Then every arm lost. Mid-depth readout broke about 61% of all decisions at every depth we tried, exactly as the prior said, because the final lm_head wants final-layer states. Word keys lost. Space slot lost. Key-addressed lost. The deltas ran from −0.89 to −3.42 index, with 752 to 3,762 changed decisions. The shipped combination (reference rendering, paren slot, label keys, final-layer readout) was locally optimal among everything we could think of.

Ten euros, seven closures. slot=space alone would have cost ~1.35 index if it had shipped unmeasured. I will take that trade every day of the week.

A smaller negative closed the ANLI threshold candidate. On a 500-row probe it looked like +4.87pp. On the full edition the cross-fitted gain fell to +0.46pp, the fold band was unstable (stable_band: false, k across folds [908, 939, 874, 914]), and the “rarest gold wins” heuristic picked class A on the probe and class C on the full edition, because ANLI’s labels are near-balanced (A 1070 / B 1068 / C 1062). It was sampling noise wearing a mechanism’s clothing. The threshold lane closed.

Reading the field, and correcting ourselves in public

While the GPU arms ran we read. The board’s meta blocks told us the reference was a method. One external survey told us something bleaker: sealed-set scores collapse for everyone: 20–37% against a 29.3% chance floor, with Jev itself at 36.7% sealed versus 86.6% public. Every public decision number overstates generalisation, which is precisely the trap v0.3’s private held-out set exists to spring.

We also spent a decent chunk of the week correcting our own briefing. Jev is not a board entry; it is the reference standard. The leader is a full fine-tune. The 120,340 in the board data is not an anomaly; it is the kit’s base request count. And a PLAN.md rewrite had confidently listed 150,317 among “discarded false figures” when 150,317 was the kit’s own scoreable count. We wrote a false correction into the plan while enforcing a rule against exactly that.

By the end of this stretch the shape was clear. The readout is fixed. The thresholds that help are label-free and deterministic. What is left is the knowledge-bound benchmarks, and the only legal way to move those without training is to let the model think. But thinking opens a new numerical surface, and there was a shipping run to land first.

Landing it meant renting silicon, which meant finding out whether any of this ported off the machines we had been testing on. The run that followed is the one where the ledger finally closes: where a cheaper venue, a full-edition score, and one missing scalar turn a stack of experiments into a floor we could submit from.

Fifty-Eight Point Zero Nine

The first paid launch died before it ran a single row.

The wrapper needed an SSH key. There wasn’t one. We had written an entire portability harness on the assumption that we, or the machine, or somebody, owned a keypair, and nobody had checked. That is the kind of thing that costs you an afternoon and teaches you to verify the boring prerequisite first. We generated a key, moved on, and launched again.

The second launch died at its first pip call. ubuntu24.04-cuda13.0 ships Python 3 without pip, plus the PEP 668 “externally managed environment” guard, while the HF PyTorch image ships pip, which is why a launch script that had only ever run on HF had never noticed the gap. The fix is now [fix 0] in the bootstrap: apt-install python3-pip, set PIP_BREAK_SYSTEM_PACKAGES=1, put ~/.local/bin on PATH. A later manual relaunch would fail for the cousin of the same reason (no ninja on PATH on a warm VM) and succeed only when run detached.

We were doing this portability work for a boring reason: money. The same RTX PRO 6000 was about $0.95/GPU-h on Nebius against $2.75/h on HF Jobs. Three times. Over a campaign of dozens of arms, three times is the difference between “we can afford to be wrong a few more times” and “we cannot.”

Then the account appeared to deny every compute write, uniformly, for both the user and the service account. The first honest recommendation was to stop: account-level IAM, nothing we could do. That recommendation was wrong, and the correction is the useful part. The tenant and project were showing suspension_state: SUSPENDED on the v1 Container reads, while container_state: ACTIVE looked perfectly fine. The field exists only on v1, the v2 API has no equivalent, the only documented trigger is a failed payment, and there is no API to read the reason or clear it. A billing fix cleared it and nothing else changed. New rule: when Nebius denies every compute write the same way, go read the v1 suspension block before you write a sad memo about IAM.

The most educational failure came on the third run. The wrapper stages the repo with git archive <commit>, which silently drops untracked files. The kit’s decision_index/data/ was never in git ( the top-level .gitignore carries data/ under a “licence-critical … NEVER commit” header that matches at any depth), so ten files (the 0.2/0.2.1 scoring subsets, index panels, benchmarks, chance baselines) existed only in working trees. Every HF run had shipped the whole working tree, so nothing had ever noticed. We respected the licence rule: no data went into git. We scp’d a side-car kit-data.tar.gz, overlaid it into the staged repo, verified all ten files sha256-identical, and made the bootstrap uninstall the kit before reinstalling it so a warm VM could not keep a stale copy.

Run three reproduced decisions exactly. 2,500 of 2,500 rows, zero errors, 3,101 shared answer slots, 0 decision changes, and 519 slots differing only in the last float digit: max absolute probability delta 2.22e-16, one ULP, concentrated in catalogs 37/39/41. Nebius was now a drop-in venue at about a dollar per round trip. A full-edition re-run would cost ~€23 instead of ~€59.

The run that became the floor

runs/full-rules is the full edition with the shipped rules on. 150,759 of 150,759 rows, complete: true, Decision Index 58.09. Raw 67.76, breadth 56.77. It finished 2026-09-30 18:44:13Z, after 85,582 seconds (23.8 hours) on an HF rtx-pro-6000 (torch 2.8.0+cu128, transformers 5.17.0). Coverage 1.0 across all five areas, all 38 index benchmarks present. Area skills: knowledge .445, language .645, retrieval .630, tools .755, arts .352.

We had pre-registered 58.22 and got 58.09, which is inside the ±0.3 band and therefore fine. Jev at 57.91 was +0.18 behind us. The board leader at 57.44 was +0.65 behind. The reference at 57.33 was +0.76 behind. Median latency 141.1 ms against the 1,000 ms bar; p95 1,734 ms; mean 557 ms. It was banked, submittable the day the freeze lifted, and it stayed the floor for everything after.

Two footnotes from that run, both worth more than they look.

The 23 errors were all ToolRet rows dying on name 'q_ids' is not defined, a NameError in the evaluation path, not the engine. ToolRet is also the catalog that OOM-crawled at ~20 s/row while the engine’s option-block halving ground through it. The 23 errors sit outside ToolRet’s 685-case scoring set, so the index never saw them. Clean fallback artifact.

The watcher died silently. After a restart the re-armed poller had no process and no output file and never delivered its completion ping; we caught the finish by hand about an hour later. Results upload permanently so nothing was lost, but the rule went into the notes anyway: after you arm a poller, verify its first poll line actually lands.

ANLI dies on the full edition

One candidate was still breathing from the threshold lane. On a 500-row probe, a far-tail cut on ANLI lifted macro-F1 by +4.87pp and changed 36 answers. The standing instruction was finalise against the full edition before believing it, so we did. The full edition killed it: cross-fitted macro went from 0.73477 to 0.73934, +0.46pp; the frozen in-sample cut was +0.86pp; the folds disagreed with each other (k across folds [908, 939, 874, 914], range 65, sd 23.2, stable_band: false). And then the punchline: the positive class itself flipped. “Rarest gold wins” picked A on the probe and C on the full edition, because ANLI’s labels are near-balanced. The lever class was built for rare-positive F1 benchmarks. ANLI is not one. Even the optimistic reading priced at ~+0.04 index. It did not ship, and the threshold lane closed.

Buying a base model: we looked, and no

The research sweep had handed us a beautiful story: at decision tasks, small beats big. A 340M encoder reportedly beating 4B LLMs. A 12B Gemma read through one-token option logits ranking #4–6 of 89 on JevBench. A 770M T5 beating few-shot 540B PaLM. If any of it held, a 340M model would make our entire budget question disappear.

The counter-evidence arrived with it. Every one of those small-model wins needs labels, and the suite defines no legitimate training split: zero training rows, ever, by construction. Training on suite data is training on the test set. Using an already-trained public model is model choice, which is legal, but the sealed-set collapse (Jev 36.7% sealed versus 86.6% public) says the public numbers are mostly fiction, and that is exactly what v0.3 checks.

The same-size comparison was worse. Through the reference letter-slot readout, Gemma-4-31B bf16 scored 59.84 and Qwen3.6-27B bf16 scored 55.73, same technique, Gemma wins by ~4 points. The board’s own Qwen3.8-27B entries sat at 56.40 and 55.74, below our banked 58.09. We preflighted the best-fitting recent open model (Qwen/Qwen3.8-27B, official FP8 W8A8, KLD 0.0052) and concluded the base-swap prior was negative: capability is not readout score, the hybrid Gated-DeltaNet attention needed a new engine path, and every board entry below us was a hint. The preflight was held, not run.

The summary: no Gemma-4-31B derivative publishes a sourced improvement on this suite; a 4B sits at 40–65% of the 31B on every row we can price (we ran Spark-X2.5-4B through the engine to confirm); no open decision model beats the leader by a confirmed margin. The base was right. The lever had to be the method.

The temperature hole

Then we audited the run per-benchmark against the reference decider row, same checkpoint. Parity within ±0.011 everywhere except our three rule targets (iSarcasm +0.256, ACOS +0.115, FinEntity +0.036 skill) and one screaming exception: ForecastBench, catalog 48, ours 0.0 versus the reference’s 0.1916.

A gold-star arts benchmark, scored entirely to zero.

The cause was a missing piece of the reference’s public config. It sets temperature: 6.972 and a per-option-count schedule T(n) = max(0.05, 10.124 - 1.633·ln n). It softens its probabilities. Our replication ran temperature 1.0, so probabilities saturated around 1e-7, Brier came out 0.262 (above the 0.25 baseline) and the clipped skill was 0. Because temperature is argmax-invariant, every choice-scored benchmark matched anyway. Only the one Brier-scored benchmark fell through the hole.

The offline verification over all 10,139 rows was about as clean as this project gets:

temperature	skill
T = 6.972	0.156
T = 8.99 (their T(n) at n=2)	0.1909 (their published 0.1916)
T = 12	0.2134
T = 16	0.2142 (theoretical optimum)

Catalog 48 is the only probability-scored benchmark in the suite, so the transform cannot hurt anything else. Value: +0.31 to +0.35 index, pushing a rerun to ~58.40–58.44, above the pre-registered 58.22. We validated it live on 1,013 rows (runs/ffb-val): Brier 0.19997 → skill 0.2001, matching the simulation to five decimals, with probabilities finally landing in a sane 0.17–0.75 range instead of 1e-7.

The freeze becomes a runway, and the target gets bigger

The full calibrated run (full-cal) was launched and then paused by the user. The freeze was still on, there was no urgency, and 58.09 was already submittable. The relaunch command went into the notes, staged for the day we wanted to ship it. The engine already carried a temperature_by_options knob, default off.

Then the user changed the size of the problem. On 2026-10-02: Stage 1 only, inference-time compute on the frozen gemma-4-31B. Nebius only. Go straight to Route B. Target, on edition 0.2.1: beat Clef (Cloudflare), > 61.21. If B breaks, or an opportunity beyond B shows up, stop and ask. Training was closed with evidence. Adoption of other models was closed with evidence. The banked 58.09 stayed as the floor.

The arithmetic forced the conclusion. The benchmarks that dominate are knowledge-bound, and a frozen model cannot add knowledge. But it can be asked to think about a question it is unsure of, and then re-read its answer with the thought in front of it. That is the only legal road from 58.09 to 61.21.

Which raised a question we had no business being confident about: would it work at all, and what would it cost?

A 0.3 revision note, because the target above is history: the campaign did not stop at edition 0.2.1. When the board moved to 0.3 the frozen stack was re-run on the new suite and finished at a 60.29 public index (v1.0 at 59.1), complete on all 140,178 requests, unsupported zero, re-scoring bit-exact, the best public index of any no-training entry, and +2.10 over the next frozen engine, deck31b, which runs the same checkpoint. The 58.09 floor and the 61.21 target belong to the era this part describes.

---

Next: Jex 1.0, Part 2, The Gate A gate curve that says +20 index is on the table, a latency wall at 40% coverage, and an external anchor that also says thinking hurts the one probability-scored benchmark. We built the control arm that could catch ourselves cheating.