SC‑WBD

The identifiability benchmark returned a verdict

Three regions, linear-Gaussian, five designs, a decision rule written before the run. C1 and C2 are the thesis's first differentiator, and as measured they do not hold.

Applies to no checkpoint — a method benchmark on a three-region linear-Gaussian system with ground truth. The verdict is about the joint-inference claim itself, so it stands until that claim is re-tested.

SC-WBD's first claim is that fusing EEG and fMRI at their native sampling rates identifies coupling and delay parameters that neither modality resolves alone, and that resampling to a common clock destroys the advantage. That is testable on a system small enough to have ground truth, so it was tested on one: three regions, linear dynamics, Gaussian noise, a 12 ms conduction delay and an explicit haemodynamic lag.

The rule, written first

Five criteria, each quantified over the minimum eigenvalue of the profile information matrix for the pre-registered parameters, evaluated in every regime.

CriterionRule
C1_fusion_informationjoint-native ≥ 1.05 × max(EEG-only, fMRI-only), in every regime
C2_native_beats_resampledjoint-native > the naive-resampling estimator
C3_intervention_informationimpulse-matched ≥ 1.05 × joint-native
C4_calibrated_recoverynominal 95% inside the Wilson interval of empirical coverage
C5_recovery_improvementdelay and θ RMSE ≤ the better single modality

The result

verdict: INCOMPLETE

C1_fusion_information        FAILED    in every regime
C2_native_beats_resampled    FAILED    (passes in 1 of 3)
C3_intervention_information  FAILED    (passes in 1 of 3)
C4_calibrated_recovery       NOT EVALUATED
C5_recovery_improvement      NOT EVALUATED
regimeC1C2C3
referencefailfailfail
low_snr_short_delayfailpasspass
weak_coupling_long_delayfailpassfail

C1 never passes. Joint native-clock inference does not reach 1.05× the better single modality in any regime tested. C2 and C3 pass where the problem is hard — low SNR, weak coupling — and fail in the reference regime, which is the one where the parameters are easiest to see.

The rule says every regime, so partial passes are failures. That was decided before the numbers existed, which is the only time it can be decided honestly.

Why two criteria are unevaluated rather than failed

The benchmark reports its own reasons, and both are properties of the experiment rather than of the model.

Convergence gating. MAP recovery ran a fixed number of Newton steps. The preconditioner is the expected information at the prior mean and is never refreshed, so the iteration converges linearly, not quadratically — a fixed step count cannot certify convergence. Two regimes were gated out on that basis. The loop now stops on the Newton decrement, sqrt(gTH-1g), the remaining distance to the optimum in posterior standard deviations, and records how many steps it actually took.

A degenerate reference regime. Its truth sits exactly on the prior mean — 0.000 prior sd on all four pre-registered parameters.

There, an estimator that ignores the data entirely and returns the prior mean scores zero bias, zero RMSE and 100% coverage. Recovery metrics cannot discriminate anything.

It is not hypothetical: fmri_only, whose information about the delay is 5e-05, posted a delay RMSE of 0.028 ms against EEG's 0.233 ms — eight times "better" purely by not moving. The report now measures each regime's offset from the prior mean in prior standard deviations and marks anything under 0.25 as unable to discriminate.

What this settles, and what it does not

It is a measured negative on the thesis's first differentiator, from a benchmark that ran to completion on a system with ground truth. It is independent of anything about the trained model — different code path, different system, no neural network involved.

It does not say fusion cannot help. It says that on this three-region linear-Gaussian system, with these five designs and this pre-registered rule, the information advantage did not appear at the 1.05× threshold the rule demanded. A larger system, a different parameterisation, or a different threshold are all live possibilities — and each would need its own rule written first.

The artifact is reports/identifiability/results.json, with the per-design decomposition in results_by_design_checkpoint.json and the narrative in reports/identifiability/summary.md.