how it was measured

The protocol in brief. The full text is in the paper and in the repository’s research record; every rule below is enforced in code and most are pinned by a test.

the split

Each natural movie is shown in two blocks per session, tens of minutes apart. One epoch boundary per session is set at the start of the earliest late block. Segments before it train (first 80%) and validate (last 20%); segments after it are test, in full; segments straddling it are discarded. The test period is therefore strictly in the future and shows the same stimulus the model trained on — which removes the confound between elapsed time and a change of stimulus that a plain chronological split would have on this dataset.

Primary resolution 0.05 s, fixed before any comparison. Context 40 bins (2 s). Multi-step horizons 1, 5, 10, 25, 50 bins.

the score

Metrics are computed on spike counts after mapping predictions back from the normalised space. The headline skill is R² against each unit’s training-period mean rate: referencing the test mean would credit a model with information it does not have. Every metric is per unit, reported as a distribution and as a rate-weighted mean.

Uncertainty resamples whole sessions, never individual bins, because adjacent 50 ms bins are strongly dependent. Model comparisons are paired. When a family’s best entrant is needed it is chosen by validation score.

leakage, and what stops it

Risk	Control	Test
Shuffled time bins	Splits are assigned by recording segment on the session clock, never by row.	`test_train_val_test_are_chronologically_ordered`
A window straddling two splits	A window is admissible only if every bin it reads or predicts carries its own split's code.	`test_no_window_straddles_a_split_boundary`
Train and validation windows sharing bins	An embargo of one context length between them.	`test_embargo_separates_train_and_val_windows`
A window across a segment join	Window indices are per segment.	`test_no_window_crosses_a_segment_boundary`
Test statistics in normalisation	The normaliser is fitted on training bins only.	`test_normalizer_uses_only_training_statistics`
Smoothing that mixes in the future	Only one-sided kernels exist in the code; none is on by default.	`test_causal_boxcar_never_looks_forward`
An architecture that peeks ahead	Causal masks, and the encoders are probed directly.	`test_convolutional_and_attention_encoders_do_not_look_ahead`
An empty split making checks vacuous	The split validator refuses any plan with an empty split.	`test_validator_rejects_a_shuffled_plan`
Flagged recording intervals	Probe flags drop that probe's units; session flags mask the bins.	`test_masked_bins_are_never_touched_by_any_window`
A biased inverse transform	Retransformation bias is added back from training residuals.	`test_sqrt_inversion_is_biased_without_correction_and_unbiased_with_it`

tuning, and its limits

Classical models tune lag, ridge penalty and latent dimension per session on validation. Each neural architecture was offered four capacity and regularisation settings, scored on the validation periods of two sessions. Every architecture chose the largest setting on the grid, which means the neural models may sit below their ceiling; a separate capacity sweep measures how much a wider network would have helped.

Neural configuration chosen on validation
Architecture	Hidden	Dropout	Weight decay
rnn	256	0.3	0.001
gru	256	0.3	0.001
lstm	256	0.3	0.001
tcn	256	0.3	0.001
transformer	256	0.3	0.001

Completed runs per experiment arm
Arm	Runs
primary	126
covariates	80
binwidth	30
scaling	32
capacity	18
latent	8
generalization	1
filtering	36

reproduce it

make setup
make train-smoke      # synthetic, no network, < 1 min
make preprocess       # stream 8 sessions from DANDI
make train            # tune on validation, then the benchmark
make ablations generalization latent robustness
make evaluate analyze figures paper site
make test