Page 1 of 1

Tracking and Evaluating Model Forecasts

Posted: Tue Feb 22, 2011 3:11 am
by gary738
Hello,

I am new to EViews and need help on the following Model problem:

Having estimated a model, I need to create for each tracked endogenous variable over each of successive n-step forecasts, where n = 1, 2, ..., 8:

1) a table (in the form of a matrix filled with blocks of data, with "rows" for variables, "columns" for forecast periods) which would contain the forecast evaluation criteria (RMSPE, MAE, MAPE, Theil)

I can do each of the steps involved by hand, but wonder if there is not a way to do so programatically, so that if the model is changed, the individual steps do not have to be repeated by hand?
This would seem to involve "looping" over the forecasts, extracting the relevant statistics (eg, @mape) and placing them in a table, but I am not sure how to do this properly

2) a graph containing the actual plus each of the (8) forecasts over the forecasting period

I can obtain the individual forecast graphs, but I have no idea how to overlay the various forecasts plus the actual onto a single graph for all (8) forecast periods

Many thanks for any help.
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Tue Feb 22, 2011 8:56 am
by EViews Gareth
There is nothing built in that will do it, but it shouldn't be too hard to program. As a start, to simply put MAPE in a table:

Code: Select all

table mytable mytable(1,1) = "MAPE:" mytable(1,2) = @mape(y, y_0)

Re: Tracking and Evaluating Model Forecasts

Posted: Wed Feb 23, 2011 2:39 am
by gary738
Thank you. I am looking at the "rollfcst" programmes and this Forum for guidance on how to do the programming, as I have no experience whatsoever with EViews
Regards
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Wed Feb 23, 2011 2:48 am
by gary738
Further to this:

Is it possible when using the "scenario" command in a loop to name the newly created scenario according to a wildcard, for example

mymodel.scenario(...) "Scenario {%0}"

where the {%0} is determined within the loop?

Thanks
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Wed Feb 23, 2011 9:06 am
by EViews Gareth
Sure.

Code: Select all

for !i=1 to 10 %scenname = "Scenario " + @str(!i) mymodel.scenario() %scenname next

Re: Tracking and Evaluating Model Forecasts

Posted: Wed Feb 23, 2011 9:38 am
by gary738
Perfect! Many thanks.
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 10:42 am
by gary738
Yet further to this:

If I try to run a modification of your code, as follows,

for %i=1 M2 G RS
%scenname = "Scenario " + {%i}
mymodel.scenario(a=%i) %scenname
next

I get the error "String added to series in "%senname = "Scenario " + M2"."

Obviously I have not understood something (eg, is §scenname actually a series?). Can you enlighten me, please?

Thanks
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 10:46 am
by EViews Gareth

Code: Select all

for %i=1 M2 G RS %scenname = "Scenario " + %i mymodel.scenario(a=%i) %scenname next
You didn't need the braces around the %i

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 12:37 pm
by gary738
Thanks!

But now I get the following error when I try to solve the model using

'define scenarios
for %i M2 G RS

' move fcst_sample !step obs at a time
for !j = 1 to 8 step !step

' set fcst_sample to forecast period
smpl 2001q1 2001q1+!j

' solve model for alternative scenario
%scenname = "Scenario " + %i
mymodel.scenario(a=%i) %scenname
mymodel.override {%i}
mymodel.solve(s=d,d=s,a=f)
end
end

Evidently I am still doing something very wrong (apologies for my stupidity - this is my first attempt ever at EViews programming)
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 12:48 pm
by EViews Gareth
This is actually a little tricky...

The issue is that you cannot change a model scenario to a scenario that doesn't exist, without using the "n" option (to create a new scenario). However, equally, you cannot use the "n" option if the scenario you're setting already exists.

I believe the logical way to fix this problem is to set the scenario outside of your inner loop (since it doesn't depend upon anything inside the inner loop):

Code: Select all

'define scenarios for %i M2 G RS ' move fcst_sample !step obs at a time for !j = 1 to 8 step !step %scenname = "Scenario " + %i mymodel.scenario(a=%i, n) %scenname ' set fcst_sample to forecast period smpl 2001q1 2001q1+!j ' solve model for alternative scenario mymodel.override {%i} mymodel.solve(s=d,d=s,a=f) next next

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 12:59 pm
by gary738
Thank you, Gareth.

In that case, shouldn't it be:

'define scenarios
for %i M2 G RS

%scenname = "Scenario " + %i
mymodel.scenario(a=%i, n) %scenname

' move fcst_sample !step obs at a time
for !j = 1 to 8 step !step

' set fcst_sample to forecast period
smpl 2001q1 2001q1+!j

' solve model for alternative scenario
mymodel.override {%i}
mymodel.solve(s=d,d=s,a=f)

next
next

Thanks again
Gary

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 2:13 pm
by EViews Gareth
Sorry, yes, bad copy and paste on my behalf.

Re: Tracking and Evaluating Model Forecasts

Posted: Thu Feb 24, 2011 2:42 pm
by gary738
Thanks again, Gareth

Tried it and it worked!!

Gary