EViews.com

Posted: **Fri May 07, 2010 2:12 am**

Eviews can report on a number of forecast evaluation criteria but these are only available as options to the fit and forecast equation procedures. Typically the criteria are used to compare regression-based forecasts with those obtained from other methods but these methods may also be implemented in Eviews e.g. naïve methods. The attached program ‘tricks’ the software into calculating evaluation criteria for any method of forecast. This is achieved by first running a least squares of the forecast values on themselves which always generates a (0, 1) parameter vector. The ‘independent variable’ observations are then changed to ‘actual values’ before the ‘fit’ procedure is undertaken to obtain the required evaluation. The method relies on the fact that all the evaluation statistics are symmetric with respect to what are called ‘forecasts’ and ‘actuals’. The attached code is in the form of a subroutine for pasting into an Eviews program. This permits its use on a number of alternative forecasts within a single Eviews job. Each result can be distinguished by the series names reported in the evaluation tableaux. I have verified the subroutine’s results with several examples against the approach of trubador elsewhere in this forum. All indices match to many decimal places with the exception of MAPE. For the moment I am inclined to think that this is a bug in Eviews! I have available a full Eviews program used for testing and evidence of the bug for distribution via PM.
Gerald

Code: Select all


'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
subroutine eval(series a, series f)
   series fa = f
   equation eqtemp.ls f c fa
   fa = a
   freeze eqtemp.fit(e) a
   delete fa eqtemp
endsub
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
smpl fsmplstart fsmplend ‘ ~~ the ex post forecast period
series y   ‘~~ contains the actuals series
series yf   ‘~~ contains the forecast series
call eval(y, yf)

Posted: **Fri May 07, 2010 8:46 am**

Couple of responses to this. First of all, thanks (diolch yn fawr iawn) for the contribution.

As of EViews 7, all of the forecast statistics can be calculated via functions, thus you could do:

Code: Select all


scalar mape = @mape(yf,y)
scalar mae = @mae(yf,y)
scalar theil = @theil(yf,y)

etc...

The reason you're getting MAPE wrong (and I believe it is a bug in your methodology rather than EViews), is that your assertion that

The method relies on the fact that all the evaluation statistics are symmetric with respect to what are called ‘forecasts’ and ‘actuals’.

is, of course, wrong in the case of MAPE. You're dividing by YF, when you should be dividing by Y.

Posted: **Mon May 10, 2010 2:06 am**

Gareth,
Many thanks for your correction - duly acknowledged! Ironically it was the erroneous thought embedded in the your quote of my message that was the inspiration for the method. It also indicates the simple corrections needed (code attached) which give 100% conformity with the 'trubador' algorithm.
I note with interest the v7 facilities... I now have an argument to seek finance for an upgrade!
Gerald

Code: Select all


'~~~~~~~~~~~~~~~~~~~
subroutine eval(series a, series f)
’ useful for v6 and before
   series fa = a
   equation eqtemp.ls a c fa
   fa = f
   freeze eqtemp.fit(e) f
   delete fa eqtemp
endsub
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~
smpl fsmplstart fsmplend ‘ ~~ the ex post forecast period
series y   ‘~~ contains the actuals series
series yf   ‘~~ contains the forecast series
call eval(y, yf)

Posted: **Mon May 10, 2010 8:01 am**

It is a nifty idea.

EViews.com

Forecast Evaluation Criteria #2

Forecast Evaluation Criteria #2

Re: Forecast Evaluation Criteria #2

Re: Forecast Evaluation Criteria #2

Re: Forecast Evaluation Criteria #2