Forecast evaluation output

For requesting general information about EViews, sharing your own tips and tricks, and information on EViews training or guides.

Moderators: EViews Gareth, EViews Moderator

Maglin
Posts: 4
Joined: Tue Oct 09, 2018 2:22 am

Forecast evaluation output

Postby Maglin » Tue Oct 09, 2018 6:23 am

Hi!

I use the forecast evaluation "fcasteval" but the tabel is puzzling. In the output table showing evaluation statistics there is no constant relationship between RMSE and TheilU2.

I use the following command where "gr_f" contains around 50 forecasts for PC_GDP. (I use Eviews10)

PC_GDP.fcasteval(evalsmpl=%wfend-!NRF %wfend, mean, trmean, median, trim=15) gr_f

What am I missing?

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11899
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation output

Postby EViews Gareth » Tue Oct 09, 2018 12:45 pm

viewtopic.php?t=17447

The last few posts.
Follow us on Twitter @IHSEViews

Maglin
Posts: 4
Joined: Tue Oct 09, 2018 2:22 am

Re: Forecast evaluation output

Postby Maglin » Tue Oct 09, 2018 1:43 pm

Sorry, I still don’t get it. If you refer to the discussion about the sample size I don’t see how that could have different impact on TheilU2 for different series (and all my forecast series have the same sample). The forecast rank should still be the same according to RMSE and TheilU2.

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11899
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation output

Postby EViews Gareth » Tue Oct 09, 2018 3:08 pm

Sorry, I'm not sure I understand what you're asking.

I don't believe there it is the case, using the definition of U2 given, that U2 and RMSE have to give the same ranking of forecasts.
Follow us on Twitter @IHSEViews

Maglin
Posts: 4
Joined: Tue Oct 09, 2018 2:22 am

Re: Forecast evaluation output

Postby Maglin » Wed Oct 10, 2018 6:08 am

Perhaps I'm missing something in the previous topic. Here´s an example to explain my problem:

Series a = actual series
Series f = forecast of a
series fpe = (f-a)/a
series ape = (a-a(-1))/a
genr sq_fpe = (fpe)^2
genr sq_ape = (ape)^2
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
RMSE_fpe = @sqrt(fpe2)
RMSE_ape = @sqrt(ape2)
U2= RMSE_fpe / RMSE_ape

Since RMSE_ape does not change for a given sample, RMSE and U2 must give the same ranking of the forecasts. But this is not the case in the output table.

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11899
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation output

Postby EViews Gareth » Wed Oct 10, 2018 1:15 pm

Code: Select all

create u 100
rndseed 10
Series a = nrnd
series f1 = nrnd
series f2 = nrnd

series e1 = f1-a
series e2 = f2-a
series fpe1 = e1/a(-1)
series fpe2 = e2/a(-1)
series ape = (a(-1)-a)/a(-1)
series sq_fpe1 = (e1/a(-1))^2
series sq_fpe2 = (e2/a(-1))^2
genr sq_ape = ((a(-1)-a)/a(-1))^2
series sq_e1 = e1^2
series sq_e2 = e2^2
smpl 90 @last
scalar fpe21 = @sum(sq_fpe1)
scalar fpe22 = @sum(sq_fpe2)
scalar ape2 = @sum(sq_ape)
scalar U21 = @sqrt(fpe21/ape2)
scalar U22 = @sqrt(fpe22/ape2)

smpl 89 @last

scalar rmse1 = @sqrt(@sum(sq_e1)/12)
scalar rmse2 = @sqrt(@sum(sq_e2)/12)

a.fcasteval f1 f2
Follow us on Twitter @IHSEViews

Maglin
Posts: 4
Joined: Tue Oct 09, 2018 2:22 am

Re: Forecast evaluation output

Postby Maglin » Thu Oct 11, 2018 7:20 am

Ok, thanks for this clarification. Googled TheilU2 and there seems to be different definitions, not sure which one is the orginal.


Return to “General Information and Tips and Tricks”

Who is online

Users browsing this forum: No registered users and 27 guests