Bureau of Transportation Statistics (BTS)
Printable Version

Table 1 - Statistical Performance of the Four Forecasting Methods

Excel | CSV

Model method Naive model Long-term memory model ARIMA model Nonlinear model
RMSE 597.8 904.5 637.7 471.5
RMSE (154.7) (158.8) (184.7) (107.9)
MAPE 9.16 13.92 9.41 7.59
MAPE (1.96) (2.33) (2.08) (2.11)
Rank correlation of original 0.768 0.59 0.757 0.809
Rank correlation of original (0.087) (0.093) (0.090) (0.071)
Significant number of positive errors, 5% 4 11 0 56
Significant number of positive errors, 10% 11 22 4 70
Wilcoxon location on original, 5% 0 0 0 1
Wilcoxon location on original, 10% 0 0 0 7
Wilcoxon variance on original1, 5% 0 2 0 14
Wilcoxon variance on original1, 10% 0 4 0 34
Rank correlation of difference 0.064 0.113 0.082 0.098
Rank correlation of difference (0.138) (0.130) (0.121) (0.119)
p(Bin)<5% (much better than chance), p2)>10% 19 54 21 17
p(Bin)<5% (much better than chance), p2)>5% 19 51 19 15
p(Bin)<10% (better than chance), p2)>10% 28 76 30 30
p(Bin)<10% (better than chance), p2)>5% 28 72 25 28

1 The number of days on which this test is valid is 184 minus the number of days on which there was a significant difference in the locations of the original and forecast series, e.g., for nonlinear at the 10% level this is 184-7=177 days.