Forecast evaluation statistics

For technical questions regarding estimation of single equations, systems, VARs, Factor analysis and State Space Models in EViews. General econometric questions and advice should go in the Econometric Discussions forum.

Moderators: EViews Gareth, EViews Moderator

KrilleJ
Posts: 40
Joined: Fri Feb 20, 2015 6:15 am

Forecast evaluation statistics

Postby KrilleJ » Thu Nov 17, 2016 2:29 am

Hi,

Is there a description of the forecast evaluation statistics that are included in EViews (e.g. for Thiel's U1 and U2)? For some of the statistics, there are some different versions with the same name in the litterature. It would be nice to see which versions that are implemented in EViews.

Best,
Kristian

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 13294
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation statistics

Postby EViews Gareth » Thu Nov 17, 2016 4:12 am

Follow us on Twitter @IHSEViews

KrilleJ
Posts: 40
Joined: Fri Feb 20, 2015 6:15 am

Re: Forecast evaluation statistics

Postby KrilleJ » Fri Nov 18, 2016 1:19 am

Thanks.

That one I found myself.

But in my forecast evaluations there are now additional measures e.g. SMAPE, Theil U1 and Theil U2. In the supplied link there is only one "Theil Inequality Coefficient".

/K

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 13294
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation statistics

Postby EViews Gareth » Fri Nov 18, 2016 1:27 am

We don't have documentation on the others (yet), but they're pretty standard.

https://en.wikipedia.org/wiki/Symmetric ... tage_error

I think there is only one definition of Theil's
Follow us on Twitter @IHSEViews

sergioh
Posts: 3
Joined: Tue Apr 10, 2018 11:20 am

Re: Forecast evaluation statistics

Postby sergioh » Tue Apr 10, 2018 2:34 pm

Hello.

I've just checked the most recent Eviews help files, and the definition of the "Theil U2 Coefficient" still hasn't been included.

Can anyone provide the exact formula used by Eviews? I tried to compute the forecast evaluation statistics by myself, and the only one that didn't match the results presented in Eviews is the Theil U2.

Thanks! Best,
Sergio

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 13294
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation statistics

Postby EViews Gareth » Tue Apr 10, 2018 3:10 pm

U2 = sqrt(fpe2/ape2)

fpe2 = sum (fpe*fpe)
ape2 = sum (ape*ape)

fpe = (f(t)-a(t))/a(t-1)
ape = (a(t-1)-a(t))/a(t-1)
Follow us on Twitter @IHSEViews

sergioh
Posts: 3
Joined: Tue Apr 10, 2018 11:20 am

Re: Forecast evaluation statistics

Postby sergioh » Wed Apr 11, 2018 6:59 am

Hello Gareth, thanks for your answer!

Assuming that a is the actual series, and f the forecast series, the definition you posted is the same I was using.

However, I didn't manage to replicate the U2 computed by Eviews (I'm using Eviews 10).

I've made a small program to illustrate it, it's in the end of the post.

The scalar "U2", calculated as you told me, is 0.943587.

And the "Theil U2" presented in the forecast evaluation results is 0.971317.

Can you please check if I've made some mistake?

Thanks! Best,
Sergio

Code: Select all

wfcreate m 2013M12 2014M12
series a
series f
a.fill 1.66, 1.93, 1.95, 2.12, 2.48, 2.62, 2.35, 2.16, 2.12, 1.73, 1.45, 1.24, 0.62
f.fill   NA, 1.84, 1.92, 1.96, 2.31, 3.61, 2.45, 2.01, 1.99, 1.81, 1.65, 1.21, 1.13

genr sq_fpe = ((f-a)/a(-1))^2
genr sq_ape = ((a(-1)-a)/a(-1))^2
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
scalar U2 = @sqrt(fpe2/ape2)

a.fcasteval f
show U2

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 13294
Joined: Tue Sep 16, 2008 5:38 pm

Re: Forecast evaluation statistics

Postby EViews Gareth » Wed Apr 11, 2018 7:46 am

The statistic is only calculated over periods for which both A and F are available both today and yesterday.

Code: Select all

wfcreate m 2013M12 2014M12
series a
series f
a.fill 1.66, 1.93, 1.95, 2.12, 2.48, 2.62, 2.35, 2.16, 2.12, 1.73, 1.45, 1.24, 0.62
f.fill   NA, 1.84, 1.92, 1.96, 2.31, 3.61, 2.45, 2.01, 1.99, 1.81, 1.65, 1.21, 1.13

series fpe = (f-a)/a(-1)
series ape = (a(-1)-a)/a(-1)
genr sq_fpe = ((f-a)/a(-1))^2
genr sq_ape = ((a(-1)-a)/a(-1))^2
smpl 2014m2 @last
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
scalar U2 = @sqrt(fpe2/ape2)
smpl @all
a.fcasteval f
show U2
Follow us on Twitter @IHSEViews

sergioh
Posts: 3
Joined: Tue Apr 10, 2018 11:20 am

Re: Forecast evaluation statistics

Postby sergioh » Wed Apr 11, 2018 10:50 am

Thanks Gareth!

I understand the requirement that A is available both today and yesterday, since both the definitions of fpe and ape depend on a(-1).

However, should it be mandatory that F is available both today and yesterday? No definitions depend on f(-1), and when this requirement is included, the one-step ahead forecast error will always be excluded from the computation of U2.

Thanks! Best,
Sergio


Return to “Estimation”

Who is online

Users browsing this forum: No registered users and 15 guests