CFA论坛 - Powered by Discuz! Board

LOS g: Contrast in-sample and out-of-sample forecasts, and compare the forecasting accuracy of different time-series models based on the root mean squared error criterion.

Which of the following statements regarding an out-of-sample forecast is least accurate?

A)	There is more error associated with out-of-sample forecasts, as compared to in-sample forecasts.

B)	Forecasting is not possible for autoregressive models with more than two lags.

C)	Out-of-sample forecasts are of more importance than in-sample forecasts to the analyst using an estimated time-series model.

Forecasts in autoregressive models are made using the chain-rule, such that the earlier forecasts are made first, permitting later forecasts to depend on these earlier forecasts.

Frank Batchelder and Miriam Yenkin are analysts for Bishop Econometrics. Batchelder and Yenkin are discussing the models they use to forecast changes in China’s GDP and how they can compare the forecasting accuracy of each model. Batchelder states, “The root mean squared error (RMSE) criterion is typically used to evaluate the in-sample forecast accuracy of autoregressive models.” Yenkin replies, “If we use the RMSE criterion, the model with the largest RMSE is the one we should judge as the most accurate.”

A)	Batchelder is correct; Yenkin is incorrect.

B)	Batchelder is incorrect; Yenkin is correct.

C)	Batchelder is incorrect; Yenkin is incorrect.

The root mean squared error (RMSE) criterion is used to compare the accuracy of autoregressive models in forecasting out-of-sample values (not in-sample values). Batchelder is incorrect. Out-of-sample forecast accuracy is important because the future is always out of sample, and therefore out-of-sample performance of a model is critical for evaluating real world performance.

Yenkin is also incorrect. The RMSE criterion takes the square root of the average squared errors from each model. The model with the smallest RMSE is judged the most accurate.

William Zox, an analyst for Opal Mountain Capital Management, uses two different models to forecast changes in the inflation rate in the United Kingdom. Both models were constructed using U.K. inflation data from 1988-2002. In order to compare the forecasting accuracy of the models, Zox collected actual U.K. inflation data from 2004-2005, and compared the actual data to what each model predicted. The first model is an AR(1) model that was found to have an average squared error of 10.429 over the 12 month period. The second model is an AR(2) model that was found to have an average squared error of 11.642 over the 12 month period. Zox then computed the root mean squared error for each model to use as a basis of comparison. Based on the results of his analysis, which model should Zox conclude is the most accurate?

A)	Model 1 because it has an RMSE of 3.23.

B)	Model 2 because it has an RMSE of 3.41.

C)	Model 1 because it has an RMSE of 5.21.

The root mean squared error (RMSE) criterion is used to compare the accuracy of autoregressive models in forecasting out-of-sample values. To determine which model will more accurately forecast future values, we calculate the square root of the mean squared error. The model with the smallest RMSE is the preferred model. The RMSE for Model 1 is √10.429 = 3.23, while the RMSE for Model 2 is √11.642 = 3.41. Since Model 1 has the lowest RMSE, that is the one Zox should conclude is the most accurate.

Consider the estimated AR(2) model, x_t = 2.5 + 3.0 x_t-1 + 1.5 x_t-2 + ε_t t=1,2,…50. Making a prediction for values of x for 1 ≤ t ≤ 50 is referred to as:

A)	an in-sample forecast.

B)	an out-of-sample forecast.

C)	requires more information to answer the question.

An in-sample (a.k.a. within-sample) forecast is made within the bounds of the data used to estimate the model. An out-of-sample forecast is for values of the independent variable that are outside of those used to estimate the model.