We built the control arm of our own ablation and shipped it under the model's name

This is the most instructive failure the project has produced, and almost none of it is about machine learning. It is about a sentence in an architecture document that narrowed a specification, was implemented faithfully, and was never written down as a narrowing.

What the paper specifies

The thesis names two differentiators. The first is heterogeneous operator-valued regional state: each region i has its own state space, not merely its own parameters within a shared one. The components, verbatim, “need not have equal shape or even be ordinary dense tensors — they may be fields on a cortical mesh, arrays indexed by depth and cell class, graphs of local populations, point processes of spike events…”paper/body.tex §2.1 Nine operator types are registered, to be assigned per region.

The second is non-nested, source-native spatial and temporal resolutions.

What we actually built

A single string in a config file.scwbd/foundation/config.py:32

local_core: str = "learned"

One backend, resolved once at construction, applied to every region. Six backends exist and are genuinely interchangeable — but the switch is global, not per region. Regional heterogeneity entered only through a conditioning vector: as parameters of one operator, never as different operators.

The state was dense and uniform — the same 28 components for every parcel. That is the paper's state with the region index deleted from the space itself.11In the paper's notation, X_i ∈ 𝒳_i: the space is indexed by region. What we implemented is X ∈ 𝒳 with a region axis inside it. Those are different mathematical objects, and the difference is the first differentiator.

So when the artifact was evaluated and lost to five of six baselines, the result did not test the thesis. The treatment arm was never built. We had built the control arm of the very ablation the paper requires, and reported its measurement under the model's name.

The part that is not a mitigation

A first draft of the internal report continued: “— it is the expected behaviour of the null arm, measured correctly.” That was wrong on two counts, and the correction is the more interesting half of this story.

It is not expected behaviour of that class. A 1.76M-parameter pooled-vector model losing to persistence is not what the thesis predicts of a pooled-vector model. A 4,160-parameter autoregressive baseline is a class the thesis equally expects to lose, and it wins by 0.54 nats. Re-scoping the result removes its standing as a test of the thesis. It does not convert it into a design choice.22This distinction is the whole of the correction. “The result does not test what we claimed” and “the result is fine” are different sentences, and the first does not imply the second. The failure remained an open, unexplained defect and was escalated as a P0 blocker on run 2.

Run 1 is not run 2's control arm. It is a control-class artifact from a different protocol — synthetic anatomy, an unproven split, no matched search budget, one seed. Treating it as the control would license an unmatched comparison. Run 2 trains its own control.

And the consequence that made this a precondition rather than a footnote: if the cause of the failure lives in shared infrastructure, it damages run 2's treatment arm identically. It did. Essay 02 is what we found.

How it happened

The architecture document specified per-parcel structured state, “each with E/I rates, adaptation, spectral modes, hemodynamic compartments, uncertainty channel.”

“Each with” — every parcel gets the same list. That phrase is a narrowing of the specification from operator-valued heterogeneous state to a uniform feature vector. It was implemented exactly as written.

The corrective was structural rather than cultural: the architecture document gained a Declared Narrowings register. Every divergence from the paper is listed with the section it narrows, the reason, and whether it is permanent or scheduled. Anything not listed there is a defect by definition. Any contributor may add a row; none may remove one; adding a row is explicitly not approval.

What was not wrong

Stated so the gap is not overclaimed, because overclaiming a failure is the same error pointing the other way:

• The multirate co-simulation is real, and its semigroup residual is measured.
• Six backends are implemented and interchangeable by config.
• The compiler, its eleven refusals and the schema kernel are built and fail closed.
• Restriction and prolongation machinery exists — it was undeclared, not missing.
• The identifiability, calibration and gate infrastructure is real, and it is what detected most of the defects in this list.

The gap is in assembly and declaration, not in the component library.

Then the treatment arm turned out to be 97% identical to the control

Having identified the gap, the obvious fix was to give regions different operators by family. Re-deriving the family assignment on the real 414-parcel prior produced this:

Eleven families, four distinct operators, and every cortical parcel running the same one. Heterogeneity was 12 regions — 2.9% of the brain, all subcortical, with hippocampus and thalamus at n=2 each. A declared cerebellar family held zero parcels.

Which meant the planned ablation was underpowered by construction: even if the hypothesis is true, the measurable effect is bounded by the 2.9% of regions that differ, and the paired contrast has to find it against the between-participant variance of the other 97.1%. A null result would have said nothing about the thesis.

The honest resolution, which was not the convenient one

That put a sharp question to the anatomy prior: does the evidence support giving cortical networks distinct operators? It was answered by a pre-fixed rule — ship the finest candidate partition in which every pair of families separates — under a spin-based null with 1,000 spins and FDR control.33A spin null preserves the spatial autocorrelation of a cortical map while randomising its alignment to the partition. Without it, almost any contiguous split of cortex “separates”, because neighbouring parcels resemble each other regardless of what you call them.

The landed partition is nine families: 138 unimodal cortical parcels, 262 association, and seven subcortical families of two parcels each.44The 138/262 split is recorded in an internal report rather than pinned by a test — the test suite asserts the total of 414, disjointness and exhaustiveness, but not the per-family counts. We are citing it because it is traceable and reproducible from the partition, and flagging that it is the weakest-pinned number on this page. 138 of 400 is a large enough contrast for the ablation to have power. The earlier concern was right in form and wrong in magnitude.

Two negative results drove the design and both are load-bearing. Cytoarchitectonic classes failed globally on every evidence block we hold, so cytoarchitecture is carried as description and is barred from justifying a family. And somatomotor versus visual cortex do not separate — meaning the separate visual and auditory families the paper names are not resolvable on the evidence we hold. We shipped fewer families than the paper lists, and put auditory, cerebellar and brainstem/hypothalamic into an explicit declared_absent list with reasons, because the validator refuses empty families. “We have no cerebellum” cannot be encoded as “we have a cerebellum family”.

We also refused a laminar-differentiation join that would have produced a field plausible in shape and wrong in content: the crosswalk put precentral and postcentral gyri in the paralimbic class, at 0.00 agreement both ways. The field is None everywhere, and the check ships as a test that fails if the join ever becomes defensible.

And the guards were verified by mutation, not by assertion

This is the standard the project now holds itself to, and it is stricter than the one it had been recommending. Writing a test that constructs a bad input and asserts the guard goes red proves the guard responds to a bad input you thought of. Mutating the implementation proves the guard is load-bearing on the code it protects.

With the validator stubbed to a no-op, all 14 guard tests fail. With the cortical family definition replaced by an anatomy-free regrouping, the separation test fails — against a size- and smoothness-matched spun control that must separate less. Without that control, “a regrouping fails” could just mean “any regrouping fails”, which would prove the test discriminates nothing.

The question that is still open

Both cortical families currently resolve to the same backend, so operator-wise the cortex is still homogeneous. The measured separation is on receptor profile, intrinsic timescale and myelin — properties that an operator's parameters carry perfectly well. Whether the evidence supports different operator families is precisely the distinction this whole essay is about, and it is the ablation's hypothesis, not a premise the ablation may assume.

With two cortical families, “region-indexed state space” is a binary distinction over 400 parcels — much closer to the paper's pooled-vector control than the phrase suggests.

That is filed in the register, in those words, asking to be attacked. It should be read as a measurement of how little the prior resolves, not as a design preference.

What we changed

• A declared-narrowings register, with the rule that anything absent from it is a defect by definition.
• The ablation's arms vary state structure only. Operator assignment is held identical across arms, because otherwise two different ablations are confounded and neither is testable.
• A third arm that receives exactly the features the spin test used, so the control is not handicapped by being denied information the treatment gets.
• The claim narrowed to what the evidence can support: a two-family cortical state partition beats a uniform one at matched capacity — not “operator-valued heterogeneous regional state”.

Run 2 inverted it, and the comparison is still unavailable

The obvious fix is to build the other arm. We did: run 2 is the candidate — family-indexed heterogeneous regional state, with per-family operators. So the arms are no longer swapped.

The comparison still cannot be made. Reading the pre-registration properly, before its numbers landed, the ablation has six arms:

Run 2 trains one of six. Its evaluation compares that arm against generic forecasting baselines — persistence, an AR model, a VAR model, a pooled Gaussian, a dense network. Nothing in that comparison holds the structure fixed while varying it.

Beating persistence would not show that heterogeneous regional state helps. Losing to it would not show that it does not. Both sentences are unavailable from this run, in the same way and for the same reason.

This was written down before the numbers existed, which is the only time it can be written honestly. Afterwards, one of those two sentences always looks like the obvious reading.

Next: the most expensive 134 seconds →