Tracking and Evaluating Model Forecasts

For technical support, tips and tricks, suggestions, or any other information regarding the EViews model object.

Moderators: EViews Gareth, EViews Moderator

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Tracking and Evaluating Model Forecasts

Postby gary738 » Tue Feb 22, 2011 3:11 am

Hello,

I am new to EViews and need help on the following Model problem:

Having estimated a model, I need to create for each tracked endogenous variable over each of successive n-step forecasts, where n = 1, 2, ..., 8:

1) a table (in the form of a matrix filled with blocks of data, with "rows" for variables, "columns" for forecast periods) which would contain the forecast evaluation criteria (RMSPE, MAE, MAPE, Theil)

I can do each of the steps involved by hand, but wonder if there is not a way to do so programatically, so that if the model is changed, the individual steps do not have to be repeated by hand?
This would seem to involve "looping" over the forecasts, extracting the relevant statistics (eg, @mape) and placing them in a table, but I am not sure how to do this properly

2) a graph containing the actual plus each of the (8) forecasts over the forecasting period

I can obtain the individual forecast graphs, but I have no idea how to overlay the various forecasts plus the actual onto a single graph for all (8) forecast periods

Many thanks for any help.
Gary

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12295
Joined: Tue Sep 16, 2008 5:38 pm

Re: Tracking and Evaluating Model Forecasts

Postby EViews Gareth » Tue Feb 22, 2011 8:56 am

There is nothing built in that will do it, but it shouldn't be too hard to program. As a start, to simply put MAPE in a table:

Code: Select all

table mytable
mytable(1,1) = "MAPE:"
mytable(1,2) = @mape(y, y_0)
Follow us on Twitter @IHSEViews

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Wed Feb 23, 2011 2:39 am

Thank you. I am looking at the "rollfcst" programmes and this Forum for guidance on how to do the programming, as I have no experience whatsoever with EViews
Regards
Gary

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Wed Feb 23, 2011 2:48 am

Further to this:

Is it possible when using the "scenario" command in a loop to name the newly created scenario according to a wildcard, for example

mymodel.scenario(...) "Scenario {%0}"

where the {%0} is determined within the loop?

Thanks
Gary

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12295
Joined: Tue Sep 16, 2008 5:38 pm

Re: Tracking and Evaluating Model Forecasts

Postby EViews Gareth » Wed Feb 23, 2011 9:06 am

Sure.

Code: Select all

for !i=1 to 10
   %scenname = "Scenario " + @str(!i)
    mymodel.scenario()  %scenname
next
Follow us on Twitter @IHSEViews

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Wed Feb 23, 2011 9:38 am

Perfect! Many thanks.
Gary

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Thu Feb 24, 2011 10:42 am

Yet further to this:

If I try to run a modification of your code, as follows,

for %i=1 M2 G RS
%scenname = "Scenario " + {%i}
mymodel.scenario(a=%i) %scenname
next

I get the error "String added to series in "%senname = "Scenario " + M2"."

Obviously I have not understood something (eg, is §scenname actually a series?). Can you enlighten me, please?

Thanks
Gary

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12295
Joined: Tue Sep 16, 2008 5:38 pm

Re: Tracking and Evaluating Model Forecasts

Postby EViews Gareth » Thu Feb 24, 2011 10:46 am

Code: Select all

for %i=1 M2 G RS
%scenname = "Scenario " + %i
mymodel.scenario(a=%i) %scenname
next


You didn't need the braces around the %i
Follow us on Twitter @IHSEViews

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Thu Feb 24, 2011 12:37 pm

Thanks!

But now I get the following error when I try to solve the model using

'define scenarios
for %i M2 G RS

' move fcst_sample !step obs at a time
for !j = 1 to 8 step !step

' set fcst_sample to forecast period
smpl 2001q1 2001q1+!j

' solve model for alternative scenario
%scenname = "Scenario " + %i
mymodel.scenario(a=%i) %scenname
mymodel.override {%i}
mymodel.solve(s=d,d=s,a=f)
end
end

Evidently I am still doing something very wrong (apologies for my stupidity - this is my first attempt ever at EViews programming)
Gary
Attachments
scenario_err.jpg
scenario_err.jpg (11.17 KiB) Viewed 5308 times

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12295
Joined: Tue Sep 16, 2008 5:38 pm

Re: Tracking and Evaluating Model Forecasts

Postby EViews Gareth » Thu Feb 24, 2011 12:48 pm

This is actually a little tricky...

The issue is that you cannot change a model scenario to a scenario that doesn't exist, without using the "n" option (to create a new scenario). However, equally, you cannot use the "n" option if the scenario you're setting already exists.

I believe the logical way to fix this problem is to set the scenario outside of your inner loop (since it doesn't depend upon anything inside the inner loop):

Code: Select all

'define scenarios
for %i M2 G RS

' move fcst_sample !step obs at a time
for !j = 1 to 8 step !step

%scenname = "Scenario " + %i
mymodel.scenario(a=%i, n) %scenname

' set fcst_sample to forecast period
smpl 2001q1 2001q1+!j

' solve model for alternative scenario
mymodel.override {%i}
mymodel.solve(s=d,d=s,a=f)

next
next
Follow us on Twitter @IHSEViews

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Thu Feb 24, 2011 12:59 pm

Thank you, Gareth.

In that case, shouldn't it be:

'define scenarios
for %i M2 G RS

%scenname = "Scenario " + %i
mymodel.scenario(a=%i, n) %scenname

' move fcst_sample !step obs at a time
for !j = 1 to 8 step !step

' set fcst_sample to forecast period
smpl 2001q1 2001q1+!j

' solve model for alternative scenario
mymodel.override {%i}
mymodel.solve(s=d,d=s,a=f)

next
next

Thanks again
Gary

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12295
Joined: Tue Sep 16, 2008 5:38 pm

Re: Tracking and Evaluating Model Forecasts

Postby EViews Gareth » Thu Feb 24, 2011 2:13 pm

Sorry, yes, bad copy and paste on my behalf.
Follow us on Twitter @IHSEViews

gary738
Posts: 8
Joined: Tue Feb 22, 2011 2:12 am

Re: Tracking and Evaluating Model Forecasts

Postby gary738 » Thu Feb 24, 2011 2:42 pm

Thanks again, Gareth

Tried it and it worked!!

Gary


Return to “Models”

Who is online

Users browsing this forum: No registered users and 2 guests