Page 1 of 1

### Forecast evaluation output

Posted: Tue Oct 09, 2018 6:23 am
Hi!

I use the forecast evaluation "fcasteval" but the tabel is puzzling. In the output table showing evaluation statistics there is no constant relationship between RMSE and TheilU2.

I use the following command where "gr_f" contains around 50 forecasts for PC_GDP. (I use Eviews10)

PC_GDP.fcasteval(evalsmpl=%wfend-!NRF %wfend, mean, trmean, median, trim=15) gr_f

What am I missing?

### Re: Forecast evaluation output

Posted: Tue Oct 09, 2018 12:45 pm
viewtopic.php?t=17447

The last few posts.

### Re: Forecast evaluation output

Posted: Tue Oct 09, 2018 1:43 pm
Sorry, I still don’t get it. If you refer to the discussion about the sample size I don’t see how that could have different impact on TheilU2 for different series (and all my forecast series have the same sample). The forecast rank should still be the same according to RMSE and TheilU2.

### Re: Forecast evaluation output

Posted: Tue Oct 09, 2018 3:08 pm
Sorry, I'm not sure I understand what you're asking.

I don't believe there it is the case, using the definition of U2 given, that U2 and RMSE have to give the same ranking of forecasts.

### Re: Forecast evaluation output

Posted: Wed Oct 10, 2018 6:08 am
Perhaps I'm missing something in the previous topic. Here´s an example to explain my problem:

Series a = actual series
Series f = forecast of a
series fpe = (f-a)/a
series ape = (a-a(-1))/a
genr sq_fpe = (fpe)^2
genr sq_ape = (ape)^2
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
RMSE_fpe = @sqrt(fpe2)
RMSE_ape = @sqrt(ape2)
U2= RMSE_fpe / RMSE_ape

Since RMSE_ape does not change for a given sample, RMSE and U2 must give the same ranking of the forecasts. But this is not the case in the output table.

### Re: Forecast evaluation output

Posted: Wed Oct 10, 2018 1:15 pm

Code: Select all

`create u 100rndseed 10Series a = nrndseries f1 = nrndseries f2 = nrndseries e1 = f1-aseries e2 = f2-aseries fpe1 = e1/a(-1)series fpe2 = e2/a(-1)series ape = (a(-1)-a)/a(-1)series sq_fpe1 = (e1/a(-1))^2series sq_fpe2 = (e2/a(-1))^2genr sq_ape = ((a(-1)-a)/a(-1))^2series sq_e1 = e1^2series sq_e2 = e2^2smpl 90 @lastscalar fpe21 = @sum(sq_fpe1)scalar fpe22 = @sum(sq_fpe2)scalar ape2 = @sum(sq_ape)scalar U21 = @sqrt(fpe21/ape2)scalar U22 = @sqrt(fpe22/ape2)smpl 89 @lastscalar rmse1 = @sqrt(@sum(sq_e1)/12) scalar rmse2 = @sqrt(@sum(sq_e2)/12)a.fcasteval f1 f2`

### Re: Forecast evaluation output

Posted: Thu Oct 11, 2018 7:20 am
Ok, thanks for this clarification. Googled TheilU2 and there seems to be different definitions, not sure which one is the orginal.