release 6e71f00 · 8 sessions · DANDI:000021

cortexflow

long-horizon stability

Teacher forcing hands the model the true past before every step; a free rollout feeds it only its own predictions. They agree at one step by construction. How far apart they drift is the real forecasting test.

free rollout

  • Trivial baseline
  • Classical
  • Neural
−1.5−1.0−0.50.0125102050AR (per unit)Transformerhorizon (bins of 50 ms)R² vs training mean
Mean over sessions (seed 0). Horizon in 50 ms bins, log scale. Below zero, the forecast is worse than predicting each unit’s average rate. Hover or focus the chart and use the arrow keys to read values.

teacher forced

  • Trivial baseline
  • Classical
  • Neural
0.000.050.100.15125102050AR (per unit)Smoothed persistenceGRUhorizon (bins of 50 ms)R² vs training mean
The same windows with the true history supplied at every step: an h-step curve here is h independent one-step forecasts.

how rollouts fail

Rank agreement between the one-step and the 50-step orderings (Spearman): 0.26. The lowest free-rollout R² at any horizon flags divergence: below −1 the forecast is more than twice as wrong as predicting the mean. The temporal-SD ratio compares how much the forecast moves with how much the recorded counts move; a rate forecast should sit below 1, and near 0 means it has flattened out.

Per-model rollout summary
ModelR² at 50 msR² at 2.5 s (free)Teacher-forced − freeFirst horizon ≤ mean rateLowest free R²Temporal SD, forecast / recorded
AR (per unit)0.185+0.032+0.157never+0.0320.09
Smoothed persistence0.157−0.053+0.2141.0 s−0.0530.01
GRU0.144−0.148+0.2980.55 s−0.1480.19
VAR (ridge)0.139+0.012+0.133never+0.0120.11
Transformer0.137−0.110+0.2530.35 s−0.1110.19
TCN0.128−0.542+0.6760.75 s−1.4510.45
LSTM0.123−0.099+0.2260.55 s−0.0990.19
RNN0.122−0.149+0.2770.40 s−0.1490.18
PCA + VAR0.100+0.023+0.080never+0.0230.09
LDS (Kalman)0.084+0.005+0.082never+0.0050.08
Mean rate0.000+0.0000.000never+0.0000.00
Persistence−0.501−0.786+0.3070.05 s−0.8080.00