Page 1 of 1

Forecast evaluation statistics

Posted: Thu Nov 17, 2016 2:29 am
by KrilleJ
Hi,

Is there a description of the forecast evaluation statistics that are included in EViews (e.g. for Thiel's U1 and U2)? For some of the statistics, there are some different versions with the same name in the litterature. It would be nice to see which versions that are implemented in EViews.

Best,
Kristian

Re: Forecast evaluation statistics

Posted: Thu Nov 17, 2016 4:12 am
by EViews Gareth

Re: Forecast evaluation statistics

Posted: Fri Nov 18, 2016 1:19 am
by KrilleJ
Thanks.

That one I found myself.

But in my forecast evaluations there are now additional measures e.g. SMAPE, Theil U1 and Theil U2. In the supplied link there is only one "Theil Inequality Coefficient".

/K

Re: Forecast evaluation statistics

Posted: Fri Nov 18, 2016 1:27 am
by EViews Gareth
We don't have documentation on the others (yet), but they're pretty standard.

https://en.wikipedia.org/wiki/Symmetric ... tage_error

I think there is only one definition of Theil's

Re: Forecast evaluation statistics

Posted: Tue Apr 10, 2018 2:34 pm
by sergioh
Hello.

I've just checked the most recent Eviews help files, and the definition of the "Theil U2 Coefficient" still hasn't been included.

Can anyone provide the exact formula used by Eviews? I tried to compute the forecast evaluation statistics by myself, and the only one that didn't match the results presented in Eviews is the Theil U2.

Thanks! Best,
Sergio

Re: Forecast evaluation statistics

Posted: Tue Apr 10, 2018 3:10 pm
by EViews Gareth
U2 = sqrt(fpe2/ape2)

fpe2 = sum (fpe*fpe)
ape2 = sum (ape*ape)

fpe = (f(t)-a(t))/a(t-1)
ape = (a(t-1)-a(t))/a(t-1)

Re: Forecast evaluation statistics

Posted: Wed Apr 11, 2018 6:59 am
by sergioh
Hello Gareth, thanks for your answer!

Assuming that a is the actual series, and f the forecast series, the definition you posted is the same I was using.

However, I didn't manage to replicate the U2 computed by Eviews (I'm using Eviews 10).

I've made a small program to illustrate it, it's in the end of the post.

The scalar "U2", calculated as you told me, is 0.943587.

And the "Theil U2" presented in the forecast evaluation results is 0.971317.

Can you please check if I've made some mistake?

Thanks! Best,
Sergio

Code: Select all

wfcreate m 2013M12 2014M12
series a
series f
a.fill 1.66, 1.93, 1.95, 2.12, 2.48, 2.62, 2.35, 2.16, 2.12, 1.73, 1.45, 1.24, 0.62
f.fill   NA, 1.84, 1.92, 1.96, 2.31, 3.61, 2.45, 2.01, 1.99, 1.81, 1.65, 1.21, 1.13

genr sq_fpe = ((f-a)/a(-1))^2
genr sq_ape = ((a(-1)-a)/a(-1))^2
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
scalar U2 = @sqrt(fpe2/ape2)

a.fcasteval f
show U2

Re: Forecast evaluation statistics

Posted: Wed Apr 11, 2018 7:46 am
by EViews Gareth
The statistic is only calculated over periods for which both A and F are available both today and yesterday.

Code: Select all

wfcreate m 2013M12 2014M12
series a
series f
a.fill 1.66, 1.93, 1.95, 2.12, 2.48, 2.62, 2.35, 2.16, 2.12, 1.73, 1.45, 1.24, 0.62
f.fill   NA, 1.84, 1.92, 1.96, 2.31, 3.61, 2.45, 2.01, 1.99, 1.81, 1.65, 1.21, 1.13

series fpe = (f-a)/a(-1)
series ape = (a(-1)-a)/a(-1)
genr sq_fpe = ((f-a)/a(-1))^2
genr sq_ape = ((a(-1)-a)/a(-1))^2
smpl 2014m2 @last
scalar fpe2 = @sum(sq_fpe)
scalar ape2 = @sum(sq_ape)
scalar U2 = @sqrt(fpe2/ape2)
smpl @all
a.fcasteval f
show U2

Re: Forecast evaluation statistics

Posted: Wed Apr 11, 2018 10:50 am
by sergioh
Thanks Gareth!

I understand the requirement that A is available both today and yesterday, since both the definitions of fpe and ape depend on a(-1).

However, should it be mandatory that F is available both today and yesterday? No definitions depend on f(-1), and when this requirement is included, the one-step ahead forecast error will always be excluded from the computation of U2.

Thanks! Best,
Sergio