EViews Blog

[gerluc]

Hello.
I am trying to replicate one result in an article posted by IHSEViews in their EViews blog on June 26, 2019 entitled "Bayesian VAR Prior Comparison". Specifically, I am trying to replicate the results reported in Table 1 of the article on the row reported for the Normal-Wishart prior. I am using the same workfile that the authors used, and followed what I think the authors were doing in the article (estimate from 1959 to 1974 and then recursively increase the sample by one quarter at a time). However, I cannot replicate the reported value of the RMSE in the table. In fact, I am getting a much lower RMSE value of 0.028 than what was reported in the table. Here is my program:


pageselect quarterly

series gdp_f

for !i=1 to 192

' run a normal-Wishart VAR on a recursive sample

smpl @first @first+63+!i

var nmvar.bvar(prior=nw, mu1=1, c1=0.1, c2=0.1, c3=4) 1 5 gdp gdpi fed

' use the estimate for an out-of-sample forecast

smpl @first+64 @first+64!i

nmvar.fit(mean, draws=100000) tmp_f

gdp_f = gdp_tmp_f

scalar rmse_gdp = @rmse(gdp, gdp_f)

next

show rmse_gdp

'===================

Is there anything missing/incorrect from the above program that I cannot replicate the RMSE value of 3.0613/3.0682 in Table 1 of the article? I just use the default priors for an NW and followed 100,000 draws as mentioned in the article. I am using EViews11.

[EViews Gareth]

Your samples are wrong, I think. It should be

Code: Select all

smpl @first @first+63+!i-1


For estimation

and

Code: Select all

smpl @first+64+!i-1 @first+64+!i-1


For forecasting

You also want to calculate the RMSE after all the loops are done, not during the loop.

[gerluc]

Thanks, Gareth.
My forecast sample was also missing a + sign between 64 and !. Anyway, when I try your suggestion, I am getting an error, which says:

Error in sample: Attempt to set sample outside of workfile range.

I think the reason for this error is because the range of my loop should be from 1 to 128 (so that the first estimation sample is from 1959Q1 to 1974Q4, and the last estimation sample from 1959Q1 to 2006Q4). Here is the snippet of my updated codes:

for !i=1 to 128
'run a normal-Wishart VAR on a recursive sample

smpl @first @first+64+!i-1
var nmvar.bvar(prior=nw, mu1=1, c1=0.1, c2=0.1, c3=4) 1 5 gdp gdpi fed
'use the estimate to run an out-of-sample forecast
smpl @first+64+!i-1 @first+64+!i-1
nmvar.fit(mean, draws=100000) tmp_f
gdp_f = gdp_tmp_f
next
scalar rmse_gdp = @rmse(gdp, gdp_f)
show rmse gdp
'================

I have to make the estimation samples to smpl @first @first+64+!i-1, so to ensure that the last estimation is from 1959Q1 to 2006Q4. This updated code now works. However, I still could not replicate the RMSE value of 3.0613/3.0682 in Table 1 of the blog by IHSEViews. With this updated code, I am getting a much lower RMSE of 0.0015. Does it mean that I have beaten the results reported in the blog?
Seriously, can it be that I am looking at the wrong mean of the draws to compute the RMSE? At which series does EViews11 store the mean of the draws? Is that the tmp_f series in the above code? Or, when I try to store each simulation in a page, there is a page created named draws which does not have the number of the simulation appended at the end, is that the one? If so, how can I access that to bring that page into the same page where my actual gdp series is located?

[EViews Gareth]

Be careful to make sure you'd calculating the RMSE over the entire forecast horizon. At the moment, with the code you posted, the RMSE is being calculated over whatever the sample is left at at the end of the loop (i.e. the very last forecast period, not the whole horizon).

[gerluc]

Yes. I now adjust for the forecast horizon with including the smpl 1975q1 2006q4:

for !i=1 to 128
'run a normal-Wishart VAR on a recursive sample

smpl @first @first+64+!i-1
var nmvar.bvar(prior=nw, mu1=1, c1=0.1, c2=0.1, c3=4) 1 5 gdp gdpi fed
'use the estimate to run an out-of-sample forecast
smpl @first+64+!i-1 @first+64+!i-1
nmvar.fit(mean, draws=100000) tmp_f
gdp_f = gdp_tmp_f
next
smpl 1975q1 2006q4
scalar rmse_gdp = @rmse(gdp, gdp_f)
show rmse gdp
'================

Still I obtain a much lower RMSE of 0.027. What do you think?

[gerluc]

Again, recalling my earlier question -- maybe the reason that I am getting a different RMSE is that I am looking at the wrong mean of the draws to compute the RMSE? At which series does EViews11 store the mean of the draws? Is that the tmp_f series in the above code?
When I try to store each simulation in a page, there is a page created named draws which does not have the number of the simulation appended at the end of the pagename, is this the mean of the draws? If so, how can I access/bring this mean to that page where my actual gdp series is located so I can compute the RMSE?

[EViews Gareth]

You are looking at the correct series.

The remaining discrepancy is because we multiplied the RMSEs by 100 in order to make them easier to read in the table :D

[gerluc]

Thanks. I see.
So I guess the difference between 3.0613% (in table 1 of the blog) as compared to what I get at 2.7% is the difference in the prior values used in the estimation?
Final questions -- if all the results reported in the columns labeled R of the tables of the blog refers to the use of raw original data (no use of dummy observations as priors?), how are these different to the results reported in the columns labeled D?

[EViews Gareth]

The ones labelled D use dummy observations.

[gerluc]

If that is the case, how are the columns D differ from the columns labeled DC? When you say uses dummy observations, you are referring to both sum of coefficients and initial observations dummy variables?

Thanks.

[EViews Gareth]

Both types.

As explained beneath the table, D uses dummies just in the final estimation, DC uses them in both the initial covariance estimate and the final estimate.

[gerluc]

Hi Gareth,

In the EViews blog, what was the maximum number of iterations used to optimize the hyperparameters in the estimation of the GLP prior?

[gerluc]

Also, how many MCMC draws were set for the independent normal-Wishart? Thanks.

[EViews Gareth]

100,000 for both.

[gerluc]

Sorry forgot to also ask. What was your burn-in percentage? Thanks.