Foundation model
SC-WBD: an integrated whole-brain foundation model across modalities, scales and dynamics
One subject's brain as a 414-parcel dynamical system that EEG, MEG, fMRI and stimulation all supervise at once — each modality constraining only what it observes, with the rest marginalized rather than imputed.
Every dot is a region of one person's brain; the lines are the
white-matter tracts connecting them. Signals enter and leave through the body at
the bottom. Drag to turn it.1414 regions — 400 cortical, 14 subcortical —
at their real coordinates in fsLR_32k surface RAS, with the strongest
900 edges of the measured connectome.
SC-WBD builds a working model of one person's brain, and keeps it consistent across every way you can measure that brain — electrical, blood flow, stimulation, and what the person actually does.
The hard part is that these measurements disagree. Electrodes read the brain a thousand times a second; an MRI scanner reads it once every two. Most people are only ever measured one way. And what someone looks at, or types, or says is not a brain measurement at all — but it is evidence about the brain.
SC-WBD keeps one shared model and lets each kind of signal constrain the part it can actually speak to. A subject recorded with electrodes alone still improves the shared model; nothing is invented to fill the gaps.
| Kind of signal | What it is | Examples |
|---|---|---|
| Going in | the world acting on the person | video, audio, speech, a magnetic pulse |
| Reading the brain | an instrument measuring the brain itself | EEG, MEG, fMRI |
| Coming out | what the person does, measured outside the head | eye movements, typing, speech, heart rate, test answers |
| Background | slow conditions that shape everything else | time of day, session, medication |
Sorting signals this way is what makes partial data usable.2In the
schema these are stimulus, observation,
boundary_output and context, and the distinction is
enforced: an observation must declare the forward operator it is
measured through, or it is refused. Without that axis, eye tracking and button
presses could only be declared as measurements of neural state, which is a
different and wrong claim. Someone's gaze is real evidence, but it is not a
reading of their cortex, and a model that confuses the two learns the wrong
thing.
400 cortical + 14 subcortical
stimulus · observation · boundary output · context
captured by a 3-vector moment, vs 5.6% scalar
each with a test that makes it fire
all five claim gates could_not_run
One model, three jobs
One model, three jobs. Each drawing shows which parts of the brain a task actually involves, and which signals it reads or sends.
Choosing where to stimulate
Magnetic stimulation is used to treat depression, and where you aim it matters. The model predicts what a pulse at a given spot will do, using that person's own head shape rather than an average one.
Reading what someone responds to
Show someone a film and you know exactly what they saw, where they looked, and what they said about it — alongside what their brain did. Four different kinds of evidence about one response.
What it does today
These run now, on this repository, not on a roadmap.3In order:
LoadedModel.from_checkpoint(...).predict();
sources.parcellate_bold.parcellate_run();
observe.leadfield.build_lead_field(); scwbd.intervene;
scwbd.infer.identifiability.
- Predicts what the brain does next, and says how confident it is rather than returning a bare number.
- Puts one person's brain scan into a shared map, marking the regions the scanner never covered instead of guessing them.
- Models how a signal travels from cortex to scalp through that individual's own head, not a template.
- Compares stimulation targets, keeping the dose and the expected effect separate.
- Tells you what an experiment can and cannot resolve before anyone is scanned.
What is in the box
Stated at the granularity someone could check. Every row below is a thing you could run.
| Component | State | What it is |
|---|---|---|
| Schema and compiler | built | Typed contracts for units, clocks, frames and transform lineage, with eleven refusals (R01–R11) that fail closed rather than warn.scwbd/schema/refusals.py A twelfth, R12, is local to this repository and fires at checkpoint emission. |
| Anatomy prior | built | 414 parcels — 400 Schaefer cortical, 14 Tian subcortical — carrying receptor density, intrinsic timescale, myelin and thickness. Partitioned into nine families by a pre-fixed rule under a spin null.4The rule was fixed before the test ran: ship the finest candidate partition in which every pair of families separates. Yeo-7 separated 6 of 21 pairs and was rejected. The surviving split is binary over cortex — cortex_unimodal (138 parcels) and cortex_association (262) — plus seven subcortical families of two parcels each. |
| Dynamics backends | built | Six generative backends — Wilson–Cowan, Jansen–Rit, reduced Wong–Wang, Stuart–Landau, Kuramoto, Linear–Gaussian — plus an equal-capacity learned control, interchangeable by one config key.5Twelve backends are registered in total once the four engineered per-family backends are counted. Only three of the six generative ones are typed mechanistic in the code; Stuart–Landau and Linear–Gaussian are effective and Kuramoto is functional. The distinction is in the source and we keep it. |
| Simulated corpus | built | 37,888 trajectories over 414 parcels at 125 Hz, generated from five of the backends.reports/run2_corpus.md |
| Real EEG corpus | built | 109 PhysioNet participants, 64 channels, split 71/11/27 and verified participant-disjoint. This is real human recording and is kept strictly separate from the simulated corpus.6The repository is emphatic about this and so are we: a simulated corpus can never be evidence that the model has learned anything about biology. Conflating the two corpora would be the most basic error available to this project. |
| TMS impulse-response path | partial | A validated E-field solver joined to the latent dynamics as an additive drive. The field computations pass their gates; the mapping from drive to neural response is explicitly flagged unvalidated in code, because no checkpoint has been trained on perturbational data.scwbd/intervene/impulse_response.py |
| Trained model that beats baselines | does not exist | Run 1's artifact lost. Run 2's pilot is training now. Nothing on this site should be read as a claim that we have one. |
| SC-WBD-002 | training | The first model on real anatomy — and, as it turns out, trained
on simulation alone.7Found at step 6166 of 8700, while the run was
still going. Run 2 renamed its training stages; the trainer gates
measured-data losses on the previous run's stage names, so
the gate never opened. The real-EEG loader was built and its split
fingerprinted, and no stage ever took a gradient on it — for nine hours the
only loss field emitted was sim_forecast_nll. Nothing raised,
because an unmatched stage name means "no real loss" rather than "unknown
stage". Two further mechanisms were inert for the same reason: per-stage
gradient permissions all fell back to the wildcard, and no individualizer was
ever built, so the stage named T1_individualisation ran 900 steps
of ordinary simulator training. 2.5M parameters over 414 parcels with
per-family engineered backends, heterogeneous per-family state widths, and
state-dependent predictive variance.8This entry previously said "a
ragged state layout (52% of the padded plane removed)". Both halves were
wrong. Run 2 ships the padded layout — the checkpoint records
layout: family_padded — with 47.34% of the
(414 × 59) plane being pad, because two hippocampal parcels of width
2 set the width for all 414 regions. The ragged layout is built and tested but
is not what these weights use, and the 52% figure was computed against an
11-family partition that no longer exists. Distinct from
001-beta in a way that is not a judgement call: the run-1
checkpoints refuse 414-region input, so the two do not share an
input space and cannot be compared. |
| Impulse response | measured | Two TMS coil poses produce measurably different predicted EEG, and a 200-permutation shuffled-normal null puts the real contrast above every permutation (p = 0.005): orientation carries the difference, not field magnitude. The criterion was committed while the checkpoint directory was still empty. |
Published artifacts
Everything below is on the Hugging Face Hub, public, with a card that states what it is and what it is not. The licences differ per artifact because the data underneath them differs — that is computed per artifact, not asserted once.
| Artifact | What it is | Licence |
|---|---|---|
| anatomy-prior-414 | The 414-parcel group-average prior: connectome, geometry, cortical normals, folding coherence. | Not NC-SA — it omits the regional maps, which are the one production asset carrying a non-commercial share-alike term. |
| sim-corpus-414-subset | Simulated trajectories over the same 414 parcels, for anyone reproducing the training curve without the measured corpus. | Generated by this repository. |
| scwbd-001-beta | Run-1 weights, published as a negative result: it loses to copying the last observed sample forward. Structurally the control arm of our own ablation. | Inherited from the corpora it was trained on. |
| scwbd-002-pilot | Run-2 weights — the treatment arm, family-indexed
heterogeneous regional state. Published as a negative result:
it loses to every baseline on both columns, NLL and MSE.9NLL
3.179 against 2.045 for the best baseline; MSE 36.27 against 4.53. Every
paired participant-clustered interval excludes zero. Run 1 at least won the
conditional mean — run 2 does not, so the "the loss is all in the variance
channel" diagnosis does not transfer.
And 3.179 flatters it. SC-WBD is scored on target/s with
the Jacobian folded into the log-variance; every baseline is scored on the raw
target. NLL_scaled = NLL_raw − log s, and mean(log s) =
0.5694 on this fold — roughly 17× the 0.035-nat spread across the three
non-trivial baselines. In their units the NLL is ≈ 3.75 and the gap ≈ 1.70
nats. No verdict changes; every interval moves further from SC-WBD.
Two more things the model card carries. subject_specific_ar is
bit-for-bit identical to ar16 — the participant-disjoint split
leaves no test participant with a fitted model, so the thesis's hardest
baseline never ran, while its own describe() reports
fallback_subjects=0. Read the table as five
comparators. And individualisation cannot be measured on this holdout at all:
no held-out person has a fitted person effect, so the between-participant
spread of the applied θ shift is exactly 0.000e+00. That is a
property of the split, not a defect of the run. The number that explains it: it scores
0.565 on simulation and 3.179 on measured
EEG, because five of six curriculum gates were wrong and no gradient was ever
taken on a recording.
And that is the smaller half. 88.7% of the model could not receive a
gradient at all — 2,231,447 of 2,516,530 trainable parameters,
including the entire family-indexed regional model that is the treatment arm's
whole thesis.10The regional modules were renamed local →
family_local, residual → family_residual,
readout → family_readout when the family-padded
architecture landed. The source cards still grant local.*,
residual.*, readout.*, and
fnmatch("family_local.ports.out_proj.weight", "local.*") is
False. An unmatched glob is not an error — it is an empty
permission set, which is a legal permission set. The 285,083 parameters that
were reachable are enough to fit something, so the loss fell, the run
finished, and five separate audits passed over it. Mechanism and measurement
agree independently: those exact modules are bit-identical across every
consecutive pair of the five stage checkpoints. So the negative result does
not say heterogeneous regional state fails to help. That part of the model was
a random initialisation participating in the forward pass for 8,700 steps. |
Inherited from the corpora; the anatomy prior's terms apply. |
Run 2 is done and listed above with its measured result, which is a loss. That was the commitment made here while it was still training — "it will be listed with its measured result, whatever that result turns out to be" — and the result turned out to be worse than run 1's.
What you would use it for
- Building a multimodal model of one subject. You have EEG for some sessions, fMRI for others, a stimulus log, maybe eye tracking. SC-WBD fuses them without requiring every subject to have every modality: each channel supervises the part it observes, and what was not measured is marginalised rather than filled in.
- Choosing where to stimulate. TMS and tFUS targets compared under an E-field model on the subject's own head geometry, with the dose and the predicted effect separated rather than collapsed into one score.11The planner refuses by default when a plan declares intent to drive real hardware or be applied to a person, and a valid authorisation record is necessary but explicitly not sufficient. The gate does not open by a calendar date passing — there is a test that runs the clock forward to 2027 to prove it.
- Designing an experiment before running it. The identifiability laboratory answers what a proposed design can actually resolve — per parameter, per modality — so a session that cannot separate two hypotheses is visible before anyone is scanned.
- Reading a brain-derived signal into software. The runtime
returns typed outputs with their predictive variance, and an output whose
parameters were not restored comes back as
Unresolvedrather than as a plausible number.
It is a modelling and engineering programme. Nothing here is a medical device.
The paper
The full technical description, built from LaTeX source in this repository rather than shipped as
a stale PDF.12Built with make paper, which runs
tectonic over
paper/sc_wbd_frontiers.tex. 46 pages including the implementation
supplement, which ships inside the same PDF.