SC‑WBD

Engineering

What went wrong, and what it was worth

Six essays on defects found while building SC-WBD. Most were found by instruments we had built to find defects; several were found inside those instruments.

These are written for engineers and researchers, and they assume you would rather read the failure than the summary. Nothing here is retrospective tidying — every claim is traceable to a committed file, including the ones that make us look bad.

The one heuristic worth taking away

If you read nothing else here, take this. It found two of the most expensive defects in the list above, before either instrument had ever been run, and it costs nothing:

The operational check

For every quantity your code names as varying, ask what it is a function of — and read the answer off the expression rather than the docstring. If the expression's free variables do not include the thing it claims to depend on, it is decorative. No run required, no data required.

The companion question, for guards specifically:

The companion check

Take the failure this guard exists to catch. Apply it to the guard's input. Does the reading move? If not, the guard is decorative — however sound its mechanism, however green its tests, and especially if it has never been run.

And the corollary that is harder to see, because from inside the apparatus every check is green and every check is correct:

Ask of any guard not only “can it fire” but “is the failure it targets representable in the model it runs against”. A verification apparatus built on a surrogate inherits the surrogate's assumptions as blind spots.