long-horizon stability
Teacher forcing hands the model the true past before every step; a free rollout feeds it only its own predictions. They agree at one step by construction. How far apart they drift is the real forecasting test.
free rollout
- Trivial baseline
- Classical
- Neural
teacher forced
- Trivial baseline
- Classical
- Neural
how rollouts fail
Rank agreement between the one-step and the 50-step orderings (Spearman): 0.26. The lowest free-rollout R² at any horizon flags divergence: below −1 the forecast is more than twice as wrong as predicting the mean. The temporal-SD ratio compares how much the forecast moves with how much the recorded counts move; a rate forecast should sit below 1, and near 0 means it has flattened out.
| Model | R² at 50 ms | R² at 2.5 s (free) | Teacher-forced − free | First horizon ≤ mean rate | Lowest free R² | Temporal SD, forecast / recorded |
|---|---|---|---|---|---|---|
| AR (per unit) | 0.185 | +0.032 | +0.157 | never | +0.032 | 0.09 |
| Smoothed persistence | 0.157 | −0.053 | +0.214 | 1.0 s | −0.053 | 0.01 |
| GRU | 0.144 | −0.148 | +0.298 | 0.55 s | −0.148 | 0.19 |
| VAR (ridge) | 0.139 | +0.012 | +0.133 | never | +0.012 | 0.11 |
| Transformer | 0.137 | −0.110 | +0.253 | 0.35 s | −0.111 | 0.19 |
| TCN | 0.128 | −0.542 | +0.676 | 0.75 s | −1.451 | 0.45 |
| LSTM | 0.123 | −0.099 | +0.226 | 0.55 s | −0.099 | 0.19 |
| RNN | 0.122 | −0.149 | +0.277 | 0.40 s | −0.149 | 0.18 |
| PCA + VAR | 0.100 | +0.023 | +0.080 | never | +0.023 | 0.09 |
| LDS (Kalman) | 0.084 | +0.005 | +0.082 | never | +0.005 | 0.08 |
| Mean rate | 0.000 | +0.000 | 0.000 | never | +0.000 | 0.00 |
| Persistence | −0.501 | −0.786 | +0.307 | 0.05 s | −0.808 | 0.00 |