bugarach · the learned detector

The data is hard to label reliably, so the benchmark had to be built first.

A pipeline that measures real recordings, simulates a data set with the answers planted in it, and scores every detector against the same truth on the same terms. The deliverable is the apparatus. A 1,149-parameter network is one of the things that has been through it, and what the apparatus says about that network is more interesting than its score.

A calcium-imaging movie gives you, per cell, a list of times at which that cell fired. The question is when a group fired together — a coordinated event, with a start and an end. Six detectors in this project answer it, each by a rule somebody wrote down: most compare a windowed count of active cells against a shuffled null and threshold the excess, though two do neither — rate+context compares a short window against its own longer context, and SPIKE-synch works from a spike-timing coincidence profile.

Training a network to answer it instead runs into the thing that makes the neighboring fields tractable and this one not. Sleep-EEG spindle detectors, hippocampal ripple detectors and clinical-EEG discharge detectors predominantly learn from events a human expert marked.

Coordinated calcium events are difficult to label reliably, and the reason is not that nobody got around to it. Show two experienced people the same raster and they will not mark the same coordinated events. How many cells make a group, how far apart their onsets may drift and still count, whether a busy stretch with no tight alignment is one long event or none — there is no observer-independent answer, so a hand-marked corpus would record one person's threshold rather than a fact about the tissue. The nearest thing that exists is DeepCINAC's four-expert corpus, and it labels when a cell fired, not when the group did — a question with a much sharper answer.

That is the whole reason this project exists, and the six detectors are the evidence for it: they are six people's attempts to write the threshold down, and they disagree with each other on real recordings.

So scoring a detector against a human's marks would measure agreement with that human, and scoring it against another detector measures agreement with that detector. Neither is correctness, and no amount of labeling effort turns one into the other. The way out is to stop looking for a truth to discover and manufacture one instead — events placed at times the generator records, in recordings whose statistics were measured from the lab's own, so that the answer is exact by construction and the question is not a toy.

Getting that wrong is not a hypothetical cost. Detector settings tuned on a dense benchmark — a coordinated event every 14 seconds — collapsed when the same settings met sparse data, because planted events sat inside the context window the detectors measure their null against. Binned SCE's precision fell from 74% to 10%, and finding out took two weeks. Both benchmarks were synthetic, which is the point: a simulator that does not match the recordings can mis-tune a detector all by itself.

Four stages. Three constraints in them are the reason the output means anything, and each was a decision rather than a default.

No detector runs in the measurement stage. The assessor reads 85 real recordings and reports four things directly: what fraction of cells joins an event, how tightly their onsets align, how often events happen, and how often a cell fires overall. Event jitter and cluster count are each tested against 1,000 circular-shift surrogates; per-cell rate and participation are measured outright, for two different reasons: a circular shift preserves each cell's onset count exactly, so it cannot be a null for a rate; and it is a null for how much coactivity there is, not for who, so it says nothing about participation either. No detector's decision rule runs in this stage, so nothing it produces inherits a detector's blind spot. (Three utilities are shared with the detector modules — a percentile, a recording-extent helper, and the window selector — but no detection rule.)

Baseline windows only. A window is a labelled stretch of one recording, and every non-baseline one — drug, high-potassium, washout — is counted and skipped. Taking the properties of coordination from a treated slice assumes the answer, and this preparation is one where that assumption has already produced a wrong conclusion.

The scan ships beside the choice. The assessor reports a scan across K — the minimum number of cells that must fire together before the assessor will call it a group — and a person picks one. Here K=3. A scan is not a decision, so the whole scan travels with the number rather than the number alone:

Every rate here is a median across the 85 slices, not a total. K=4 cuts it to roughly a quarter of K=3's and would build a different benchmark; by K=8 the median slice has nothing, and only 8 of 85 have enough events to measure an onset spread at all. The last column — the count of slices with a measurable spread — is what decides the choice, because the spread itself does not fall cleanly as K rises: past K=4 it is measured on a shrinking, increasingly unrepresentative subset. Observed spread sits below its shuffled null at every K, which is the direction real coordination should show.

Then the generator plants events at known times, so a miss and a false alarm are counted rather than inferred. Every detector is fitted on three folds and scored on the fourth, all four rotations, by one scorer. The hand-written six sweep the single threshold each declares; the networks train and pick an operating point. That operating point is chosen on the training folds — never on the fold being scored, and never re-picked on the recording being analyzed, which would hide exactly the failure the transfer test below measures.

Provenance, stated because it bounds everything below. This assessment read the FAST stream only — the lab separates each recording's events into fast and slow calcium transients, coordination behaves differently between the two, and only the fast half was measured here. It came from the project's .mat event store. That store has since been closed in favor of an approved export folder, because it carries recordings the lab later withdrew, and the difficulty axis it produced was re-derived from the folder on 2026-08-20. The pipeline drawn above is what runs today; the numbers below came out of it before that correction.

A benchmark that only plants events measures whether a detector can find them. It says nothing about what else the detector fires on, and that is where coordination detection goes wrong — the failure is not missing a crowd, it is calling a busy stretch a crowd.

So the generator plants two kinds of negative as well. Distractors are correlated bursts labeled as coincidence rather than coordination, and a detector is charged for firing on them. And each recording carries a probe block — 1,200 s to 1,500 s of a 3,525 s recording, to which the generator adds 0.06 Hz per cell on top of a background of 0.0097 Hz, with nothing planted in it. About seven times the baseline activity and no coordination at all: the exact thing a coordination detector must not confuse.

⚠ The distractors are weaker than that description implies, and the page reports the probe rather than them for that reason. The generator draws a distractor with the same code it draws a planted event — same cell sampling, same jitter — differing in participation and in where they may land, and at 0.18 a distractor involves more cells than two of the three planted levels (0.14 and 0.08). So "coincidence, not coordination" is a label applied by the generator, not a property a detector could discriminate on, and every leading detector duly fires on nearly all of them. That caps the precision of any detector that fires more often than it hits — which is every leading detector here — and the ceiling is a fact about the generator, not about detectors.

The published score cannot see the probe block, on purpose. Firings inside it are removed from the precision denominator rather than charged as false alarms, so the block reports rather than penalizes. That is a defensible rule and it has a cost: the headline metric is blind to the behavior the block was built to expose. The honest thing is to print both numbers, which is what the table below does.

Look down a tube as the recording slides past. The tube is dark and single firings are specks; when several cells fire together a bright spot crosses the center. Jitter dims it. A busier background raises the whole field, but a real spot still stands above its own surround.

Cells are max-pooled over the narrowest fitted center width, averaged into one brightness trace — the mean is what makes the model indifferent to cell count — and convolved with four differences of Gaussians whose widths are free parameters. (The model's docstring says the cells are summed. They are averaged, and the division is what buys the invariance the docstring credits to the sum — a second stale comment, named here rather than quietly fixed, on the same principle as the one below.) Of the model's 1,149 parameters, the dilated stack that follows holds 1,128 and the final 1×1 layer 9; the center−surround stage the architecture is named for holds twelve.

Three properties are often claimed for this shape. Each was checked — two by probes built for the purpose, on inputs the benchmark does not contain, and one not at all — and what the checks support is one and a half of them.

Cell count — holds, with a caveat worth more than the claim. The trace is the fraction of cells active, so the model runs unchanged on any number of cells. But what transfers is that fraction, not the event: six cells firing together is a participation of 0.19 on a 32-cell recording and 0.02 on a 300-cell one, and the benchmark's smallest planted events sit at 0.08. The model has only ever been scored at 32 cells, and no sweep over cell count exists.

One cell, one vote — the cap holds and the conclusion drawn from it does not. The max-pool caps each cell's contribution exactly, in amplitude. Across 10 independently trained fits, two cells bursting reach a pooled brightness of 0.0625 however often they fire, and four distinct cells reach 0.1250 — the same two values at every seed, to four decimals. The cap is not the problem.

⚠ The behavioural results below are measured on a synthetic raster that is empty apart from the cells named — the strongest contrast a center−surround kernel can be handed, and not a condition the benchmark contains. They say what the architecture does with a clean stimulus; they are not a detection rate.

The problem is that the model does not decide on brightness. It decides on the length of the run. On 10 of 10 fits, two cells firing five times clear the operating point while the same two cells firing once are rejected — identical amplitude, opposite verdict, decided entirely by duration.

Take the docstring's claim literally — one cell — and the picture is stranger. A single cell firing once clears threshold on 0 of 10 fits; firing five times it clears on 5; firing twenty times it clears on 1 again. The model has a preferred run length, and bursting past it is rejected as surely as not bursting at all. So one cell can imitate a crowd on a minority of fits, and only if it bursts by about the right amount — which is a weaker claim than "a single cell cannot", a stranger one, and the one the measurement supports.

⚠ A tempting stronger claim does not survive the same test. Two bursting cells outscore four distinct ones on 6 of 10 fits and lose on the rest — both scores are saturated sigmoids above threshold, separated by a median of 0.0002 and never more than 0.006, so which one wins is seed noise. An earlier version of this page reported the single fit where the burst won. That is the failure this page keeps finding in its own project, committed by the page itself.

So the model's source comment is right that the cap is exact and wrong in what it concludes. It says a single cell bursting cannot imitate a crowd; two cells bursting reliably do, and one cell does on a minority of fits. That is the same phenomenon as the probe-block firings below.

Rate invariance — true of the kernel, false of the model. Two area-matched Gaussians subtracted give a kernel that integrates to zero, so a flat field cancels. That is the figure's flat field cancels label, and it is arithmetic. But the raw brightness trace bypasses that stage and enters the stack as a fifth channel — the dashed path in the figure — carrying the absolute background level with it.

Pushing a rate change through the whole trained model rather than through its kernel says what that costs. On recordings with nothing planted in them at all, raising the background eightfold takes the model from 66 frames over its own operating point to 890, out of about 35,000. A rate-invariant detector would return roughly the same count at every multiple. Zeroing the bypass channel and re-scoring the same recordings gives 39 and 692 — four fifths of the baseline firings depend on that channel, but at eightfold background 298 of 486 survive without it. What remains is variance: a difference of Gaussians is a difference of two weighted means, so it cancels a shift in level but not the extra fluctuation a higher rate brings with it.

⚠ The ablation does not isolate the bypass, and it points the other way. The counts fall at every multiple, but the fold-change does not: 35× from baseline to eight-fold background with the channel, and 99× without it. Removing the channel this section blames makes the relative rate sensitivity roughly three times worse. Zeroing an input while keeping a threshold fitted with it also takes the model out of the distribution it was calibrated on, so the drop in absolute counts is at least as consistent with a global downward shift as with anything rate-specific. Re-picking the threshold on the ablated model would separate those; it has not been done. What survives is the first measurement, which needs no ablation: the model is not rate-invariant. Which of its two channels is more to blame is unsettled.

⚠ This page carries four kinds of fit of the same architecture, and only the first produced the table. The four scored folds; the ablation grid behind the scale and clamp results — twenty fold-fits over five variants, several of which are provably the same computation; the two regime fits behind the transfer numbers, the busier of which is also the refit whose surround ratio reaches 37.7; and the 10-seed grid behind the one-vote probe, of which seed 7 is the single fit this rate sweep used. Each demonstration is about the architecture rather than about a published number, but the distinction is worth naming, because conflating two of them is a mistake this page has already made once.

⚠ The rate sweep above is that one fit, so it is an order of magnitude rather than a measurement — the same limitation every learned number here carries. It is reported because the alternative in an earlier version of this page was a figure panel that convolved a step with a numpy copy of the kernel and called it the test. That panel could not have failed.

The fitted kernel widths are not quoted here. An earlier version of this page reported them as the model recovering the timescale of a real event on its own. The project had already withdrawn that reading: refitted on a quieter background with identical events, the same four widths spread over a 3.8× range where the busier fit spans 1.6×. They are a property of the background, not a measurement of the event, and the withdrawal is older than the claim was.

F1 is the harmonic mean of recall (what fraction of planted events were found) and precision (what fraction of calls were real); 1.0 is perfect, and it charges equally for a miss and a false alarm. A call counts as finding a planted event if it lands within 1.5 s of it, or if the detector's own reported span contains it. Recall, precision and fold ranges for every row are in the full run. The two F1 columns describe the same detections — nothing was re-run; one arithmetic choice is reversed.

Read the two columns together, and the ordering is not stable. On the published rule the top three are a tie: four folds of thirty planted events cannot separate 0.681 from 0.638, and all three fold ranges overlap. Charge the probe block and the learned model separates from the other two, downward: it loses 0.138 of F1 where CoactDetect loses 0.011. The two hand-written detectors stay tied with each other — 0.025 apart, well inside the same fold spread.

The column also reorders the field. locust ranks fifth of nine on the published metric and seventh once the probe is charged, because it fires 215 times into a block containing nothing — against CoactDetect's 1.25. It stays above only the two learned models that barely fire at all. That is the pipeline doing the job it was built for: a detector that keys on activity rather than coordination is invisible to the headline metric and obvious in the trap.

⚠ Which of these two columns should be the score is an open question in this project, and this page does not answer it. Both rules are already implemented in the tree, they pick opposite winners for more than one detector, and a decision is pending — it blocks a planned re-fit. Excluding the block has a real argument behind it: fold it in and a severe enough probe stops measuring the detector and starts measuring how hard the probe was set. Printing both is the honest position while that is undecided, not a verdict.

And the two learned models reading zero in that column are not restrained. The per-cell bank swallowed the block whole: it emits two detections per fold, and those two calls hit twelve distractors and two planted events across two recordings — one span per recording, covering essentially all of it. Its probe count is zero because a detection matched to a planted event never enters the false-alarm set at all, so the count is structurally zero for any detector whose every call is a hit, however much of the block that call covers.

The pooled trace reads zero on one of its four folds for both reasons at once, which is worth more than either. There it emitted ten calls. Two of them — one per recording — span about 89% of their recording and swallow the probe block whole; both matched a planted event, so neither could ever enter the false-alarm set. The other seven are short spans at the two ends, and they happened to miss a window that is 8.5% of the recording. Not restraint, and not one clean mechanism.

So the trap has a second blind spot, and it is worth more than the first. It cannot charge a firing that matched a planted event, however wide that call was; it cannot charge a detector that fires once, forever; and it cannot charge one whose false alarms simply landed elsewhere. A zero can be manufactured by more than one of those at the same time, which is the strongest statement available of why the column cannot be read on its own. Neither F1 column sees any of it: the charged rule differs from the published one only by putting probe firings back into the denominator, so with zero of them the two rules are the same arithmetic. That is why these two rows alone are identical across the two columns while every other row moves. ⚠ In seven of their eight fold-fits the threshold also lands on the bottom edge of the searched grid, which this project treats elsewhere as a search that stopped too early rather than an answer, so their F1 is reported for completeness and is not an operating point. They train at a tenth the learning rate of the model they are contrasted with, so the architecture comparison is uncontrolled and no conclusion is drawn from it here.

The model transfers asymmetrically. Fitted on quiet recordings it scores 0.772 on held-out quiet ones and 0.532 on busy ones. Fitted on busy recordings it scores 0.727 on quiet and 0.612 on busy. Fitting on the wrong regime costs 0.080 of F1 when the recordings are busy and 0.045 when they are quiet, which points at a rule — a lab fitting a model on its own recordings should fit on the busier ones — while falling short of establishing it. ⚠ The page's own floor forbids the stronger reading. Seed variance is unmeasured and the caveat below says nothing under 0.049 of F1 should be believed; 0.045 is under it, each of the four cells is a single training run, and the rule is minimax rather than dominance — on quiet data the quiet fit is still the better one. Both regimes also sit on the difficulty axis that was later re-derived.

At four folds the scale count is not a variable this benchmark can resolve. One scale scores 0.649 at 1,068 parameters against four scales at 0.681 and 1,149. Two scales scores 0.664, so the three are ordered by scale count. ⚠ That ordering is new, and it is the second thing this ablation has said. Re-run on 2026-08-28 after the operating point stopped being chosen on the fitting recordings, it reversed: one scale had scored above four, and two scales had sat outside both, which no monotone reading predicted. The old grid is what the webapp README's “ship one scale” guidance was derived from. Neither ordering is a result. All three still sit inside a fold spread of 0.049, so none of the three differences is one this benchmark can distinguish from noise — and a grid whose ranking flips when an unrelated defect is fixed is a grid reporting that it cannot resolve the variable. Four is what ships, because four is what was measured end to end — not because a sweep chose it, and not because it now happens to lead.

A second ablation raised the ceiling on how much wider the surround may be than the center, from 40 to 200, and the score did not move by a digit. That is not a null result — it is a no-op. The largest ratio fitted anywhere is 24.8 at four scales and 28.6 at one, so the ceiling never binds and each pair of runs is a bit-identical computation: 0.649 against 0.649 at one scale, and the same equality at four. An earlier version of this page reported the no-op as evidence the clamp was harmless. ⚠ And the question that prompted this ablation no longer reproduces. It was written because a fitted ratio sat at 38 against a ceiling of 40 — by this project's own rule, a search reporting that its range was wrong. After the 2026-08-28 threshold fix nothing fits above 28.6, so the ceiling is clear at every scale count and there is nothing left to raise. The earlier page also said the clamp did bind in the single-scale runs and that their scores “differ as they must”; they no longer differ, because it no longer binds there either.

Applying the same test to the busier regime says the same thing. A refit there reaches 37.7, close enough to 40 to look like a parameter pinned against its bound — and that observation is why max_ratio exists at all. But raise the ceiling to 200 on that refit and it is bit-identical too. It stops at 37.7 because that is where the fit lands, not because 40 held it there.

Where the ceiling genuinely binds is the single-scale runs. Those fits push their raw ratio to 41.3 against a clamp of 40, and reach 59.3 once it is raised to 200 — which is why their two scores differ where the four-scale pair's do not. So the clamp is inert at four scales in both regimes and active at one, and the interesting thing it reveals is not about the clamp: a bank of four scales settles on ratios a single scale runs well past.

None of the six is this project's method. Each is a port, and several are modified; the names in the table are the ports, not the originals. Cite the sources below rather than this repository for any of them.

The architecture is not unprecedented either. A center−surround receptive field as a difference of Gaussians is Rodieck (1965) and Enroth-Cugell & Robson (1966); as a detection kernel, Marr & Hildreth (1980); a bank of them with scale as the selected quantity, Lindeberg (1998), with octave spacing from Lowe (2004). Making the widths learnable has a direct precedent in Pogoncheff, Granley & Beyeler (NeurIPS 2023). And the obvious next move on the rate sensitivity above — divide by the surround instead of subtracting it, so the threshold scales with the background rather than sitting at a fixed offset from it — is STA/LTA, the standard seismological event trigger: Allen (1978), BSSA 68(5):1521–1532, doi:10.1785/BSSA0680051521. Same ratio, same geometry, forty-eight years old.

Nor is the pipeline's shape. Simulation-derived benchmarks with planted ground truth are established practice — NAOMi and MEArec build them for calcium imaging and extracellular recording, SpikeForest scores against them. The per-lab retraining loop is CASCADE's stated central move, published in 2021. What differs here is narrower and worth stating exactly: the substrate and the source of truth — a generator fitted to one lab's own baseline recordings, so the benchmark is rebuilt per lab and the ground truth is exact.

The intent was that a new idea could be dropped in and land on the same accuracy-versus-cost curve as everything already measured, with nothing else edited. A registry carries the architectures, and the trainer and the figure tool read from it. The center−surround model is registered as tube, after the metaphor above:

⚠ "Nothing else edited" is the intent and not yet the fact. The registry has two consumers — the trainer and the architecture figure. A new architecture today costs edits in four more files before it reaches this page's curve: the bake-off, the regime-shift test, the figure builder and the lab server each carry their own learning-rate table, and three of them their own list of which architectures to run. Folding the learning rate into the registration and having the sweep iterate the registry would make the claim true; it has not been done, and stating the intent as an achievement would be the same defect this page catches elsewhere.

That is the registration verbatim, including the note that still says rate invariance by construction — the claim the section above demolishes. It is quoted unedited rather than tidied, because a page that corrects one stale source comment by name and silently trims another is choosing which of its own errors the reader gets to see. Correcting it is filed, not done: the docstring and the model disagree, and making them agree means deciding whether to change the words or the network.

Torch is an optional extra, and outside the training code only the lab server's capability probe imports it, inside a try/except — so the measurement half of the package (the assessor, the generator, the six detectors, the scorer) installs and runs with no deep-learning stack at all. That split is deliberate: the apparatus has to be usable by a lab that will never train anything.

The full run with every caveat is docs/learned/bakeoff.md; the architectures are src/bugarach/learn/nets.py and the training loop is src/bugarach/learn/train.py. The three claims this page makes about the model each have a tool behind them and a store in front of them: the second F1 column is tools/probe_inclusive_f1.py, the rate sweep is tools/probe_rate_invariance.py, and the one-vote grid is tools/probe_one_vote.py. All of it regenerates from the tools in the repository, and each of those three writes its whole store.