one-step benchmark

Predict every unit’s spike count in the next 50 ms bin from the previous two seconds, on a block recorded tens of minutes after training. 8 sessions; neural models averaged over seeds.

skill on the future block

Trivial baseline
Classical
Neural

R² against each unit’s training-period mean rate, averaged over units. Faint dots are sessions; the bar is a 95% interval from resampling whole sessions. Zero is the mean-rate predictor. Persistence is omitted from the plot (mean −0.472) because it would compress the axis; it is in the table.

Table view

Model	Family	R² mean	95% CI	Median unit R²	Rate-weighted R²	Corr.	Poisson dev.	Seed SD	Params	Fit (s)
AR (per unit)	classical	0.187	[+0.172, +0.201]	0.169	0.227	0.372	0.733	—	3,960	29
Smoothed persistence	trivial	0.160	[+0.144, +0.175]	0.142	0.196	0.343	0.818	—	1	0
VAR (ridge)	classical	0.144	[+0.123, +0.166]	0.135	0.183	0.341	0.955	—	57,720	8
GRU	neural	0.142	[+0.124, +0.163]	0.131	0.179	0.340	0.899	0.0011	715,896	62
Transformer	neural	0.138	[+0.120, +0.158]	0.122	0.177	0.333	0.940	0.0030	1,247,608	66
TCN	neural	0.130	[+0.115, +0.146]	0.117	0.165	0.323	0.956	0.0020	1,534,328	55
LSTM	neural	0.123	[+0.106, +0.141]	0.109	0.155	0.316	0.940	0.0025	944,248	35
RNN	neural	0.123	[+0.108, +0.138]	0.110	0.158	0.314	0.966	0.0026	259,192	67
PCA + VAR	classical	0.103	[+0.088, +0.119]	0.099	0.131	0.300	0.968	—	12,184	1
LDS (Kalman)	classical	0.086	[+0.079, +0.094]	0.068	0.118	0.252	0.940	—	16,952	14
Mean rate	trivial	0.000	[0.000, +0.000]	0.000	0.000	0.000	0.982	—	120	0
Persistence	trivial	−0.472	[−0.505, −0.440]	−0.492	−0.390	0.225	6.752	—	0	0

cost against skill

Classical
Neural

Mean one-step R² against parameter count and wall-clock fit time (neural models on Apple-silicon GPU, classical on CPU; fit time includes each model’s own validation search). Trivial baselines are omitted because they have no fitted parameters or fit time to place on a log axis.

Table view

Model	Params	Fit (s)	R² mean
AR (per unit)	3,960	28.9	0.187
VAR (ridge)	57,720	8.3	0.144
GRU	715,896	62.4	0.142
Transformer	1,247,608	66.4	0.138
TCN	1,534,328	54.8	0.130
LSTM	944,248	34.8	0.123
RNN	259,192	67.0	0.123
PCA + VAR	12,184	1.4	0.103
LDS (Kalman)	16,952	14.3	0.086

does the ranking survive

Four sensitivity arms: how much training data each model needs, whether a wider network helps, whether the temporal resolution changes the ordering, and whether the unit-selection rules do. Sections still computing are simply absent.

training data

Classical
Neural

Each model refitted on the most recent contiguous fraction of the training period; validation and test unchanged. Gap between the best neural and best classical model at the smallest fraction: −0.091; the gap narrows monotonically with data (Spearman 1.00).

capacity · GRU

A wider network on the same data: validation is the near future of the same block, test is a block recorded tens of minutes later. A rise in validation without a rise in test is interpolation bought at no benefit to extrapolation.

capacity · Transformer

temporal resolution

Trivial baseline
Classical
Neural

Context and horizon held fixed in seconds while the bin width changes. Skill rises with coarser bins for every model, because averaging removes Poisson noise; the question is whether the ordering moves.

unit selection

One-step R² under relaxed and strict quality criteria and a smaller unit cap
Variant	Units (mean)	AR (per unit)	GRU	VAR (ridge)
cap60	60	0.227	0.193	0.203
relaxed	120	0.196	0.149	0.150
strict	79	0.201	0.175	0.179
default	—	0.196	0.157	0.161

paired comparisons

Differences are computed within each session, then summarised. “A better in” is the share of sessions (or units) where A scores higher. The unit-level rows resample units within resampled sessions, so their intervals honour the session structure.

Session-level paired differences (A − B)
Comparison	Mean ΔR²	95% CI	d_z	A better in	n
Best neural vs best classical (chosen on validation)	−0.0328	[−0.0529, −0.0127]	−1.02	13%	8
RNN vs AR (per unit)	−0.0646	[−0.0774, −0.0541]	−3.48	0%	8
RNN vs VAR (ridge)	−0.0214	[−0.0275, −0.0153]	−2.20	0%	8
RNN vs Smoothed persistence	−0.0378	[−0.0549, −0.0239]	−1.52	0%	8
GRU vs AR (per unit)	−0.0448	[−0.0590, −0.0325]	−2.15	0%	8
GRU vs VAR (ridge)	−0.0016	[−0.0060, +0.0034]	−0.21	38%	8
GRU vs Smoothed persistence	−0.0180	[−0.0361, −0.0022]	−0.67	38%	8
LSTM vs AR (per unit)	−0.0644	[−0.0784, −0.0524]	−3.18	0%	8
LSTM vs VAR (ridge)	−0.0212	[−0.0268, −0.0154]	−2.32	0%	8
LSTM vs Smoothed persistence	−0.0377	[−0.0560, −0.0220]	−1.42	0%	8
TCN vs AR (per unit)	−0.0574	[−0.0692, −0.0467]	−3.29	0%	8
TCN vs VAR (ridge)	−0.0141	[−0.0210, −0.0066]	−1.23	13%	8
TCN vs Smoothed persistence	−0.0306	[−0.0465, −0.0165]	−1.29	0%	8
Transformer vs AR (per unit)	−0.0495	[−0.0614, −0.0382]	−2.78	0%	8
Transformer vs VAR (ridge)	−0.0063	[−0.0114, −0.0001]	−0.69	13%	8
Transformer vs Smoothed persistence	−0.0227	[−0.0387, −0.0087]	−0.96	13%	8

Unit-level paired differences (A − B), sessions as blocks
Comparison	Mean ΔR²	95% CI	A better in	units
RNN vs AR (per unit)	−0.0644	[−0.0774, −0.0539]	7%	936
RNN vs VAR (ridge)	−0.0214	[−0.0277, −0.0151]	21%	936
GRU vs AR (per unit)	−0.0444	[−0.0591, −0.0321]	17%	936
GRU vs VAR (ridge)	−0.0014	[−0.0059, +0.0035]	48%	936
LSTM vs AR (per unit)	−0.0641	[−0.0783, −0.0520]	8%	936
LSTM vs VAR (ridge)	−0.0210	[−0.0268, −0.0152]	26%	936
TCN vs AR (per unit)	−0.0571	[−0.0693, −0.0463]	11%	936
TCN vs VAR (ridge)	−0.0141	[−0.0211, −0.0066]	29%	936
Transformer vs AR (per unit)	−0.0492	[−0.0613, −0.0381]	14%	936
Transformer vs VAR (ridge)	−0.0062	[−0.0114, +0.0001]	36%	936