## Forecast Evaluation Criteria #2

For posting your own programs to share with others

Moderators: EViews Gareth, EViews Moderator

Dataminer
Posts: 11
Joined: Thu Jul 23, 2009 2:01 am

### Forecast Evaluation Criteria #2

Eviews can report on a number of forecast evaluation criteria but these are only available as options to the fit and forecast equation procedures. Typically the criteria are used to compare regression-based forecasts with those obtained from other methods but these methods may also be implemented in Eviews e.g. naïve methods. The attached program ‘tricks’ the software into calculating evaluation criteria for any method of forecast. This is achieved by first running a least squares of the forecast values on themselves which always generates a (0, 1) parameter vector. The ‘independent variable’ observations are then changed to ‘actual values’ before the ‘fit’ procedure is undertaken to obtain the required evaluation. The method relies on the fact that all the evaluation statistics are symmetric with respect to what are called ‘forecasts’ and ‘actuals’. The attached code is in the form of a subroutine for pasting into an Eviews program. This permits its use on a number of alternative forecasts within a single Eviews job. Each result can be distinguished by the series names reported in the evaluation tableaux. I have verified the subroutine’s results with several examples against the approach of trubador elsewhere in this forum. All indices match to many decimal places with the exception of MAPE. For the moment I am inclined to think that this is a bug in Eviews! I have available a full Eviews program used for testing and evidence of the bug for distribution via PM.
Gerald

Code: Select all

`'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~subroutine eval(series a, series f)   series fa = f   equation eqtemp.ls f c fa   fa = a   freeze eqtemp.fit(e) a   delete fa eqtempendsub'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~smpl fsmplstart fsmplend ‘ ~~ the ex post forecast periodseries y   ‘~~ contains the actuals seriesseries yf   ‘~~ contains the forecast seriescall eval(y, yf)`

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11368
Joined: Tue Sep 16, 2008 5:38 pm

### Re: Forecast Evaluation Criteria #2

Couple of responses to this. First of all, thanks (diolch yn fawr iawn) for the contribution.

As of EViews 7, all of the forecast statistics can be calculated via functions, thus you could do:

Code: Select all

`scalar mape = @mape(yf,y)scalar mae = @mae(yf,y)scalar theil = @theil(yf,y)`

etc...

The reason you're getting MAPE wrong (and I believe it is a bug in your methodology rather than EViews), is that your assertion that
The method relies on the fact that all the evaluation statistics are symmetric with respect to what are called ‘forecasts’ and ‘actuals’.

is, of course, wrong in the case of MAPE. You're dividing by YF, when you should be dividing by Y.

Dataminer
Posts: 11
Joined: Thu Jul 23, 2009 2:01 am

### Re: Forecast Evaluation Criteria #2

Gareth,
Many thanks for your correction - duly acknowledged! Ironically it was the erroneous thought embedded in the your quote of my message that was the inspiration for the method. It also indicates the simple corrections needed (code attached) which give 100% conformity with the 'trubador' algorithm.
I note with interest the v7 facilities... I now have an argument to seek finance for an upgrade!
Gerald

Code: Select all

`'~~~~~~~~~~~~~~~~~~~subroutine eval(series a, series f)’ useful for v6 and before   series fa = a   equation eqtemp.ls a c fa   fa = f   freeze eqtemp.fit(e) f   delete fa eqtempendsub'~~~~~~~~~~~~~~~~~~~~~~~~~~~~smpl fsmplstart fsmplend ‘ ~~ the ex post forecast periodseries y   ‘~~ contains the actuals seriesseries yf   ‘~~ contains the forecast seriescall eval(y, yf)`

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11368
Joined: Tue Sep 16, 2008 5:38 pm

### Re: Forecast Evaluation Criteria #2

It is a nifty idea.