Replication, Warning Messages, and State Space Models

[daktx2]

Hi all,

A colleague and I have been struggling to replicate results when comparing Eviews to several papers.

The first replication issue we had was trying to replicate stage 1 of the Laubach and Williams model. The workfile is attached here.

Laubach Williams Stage 1.wf1

(40.06 KiB) Downloaded 518 times

I use the actual estimates found by L-W as the initial values for the maximization, and use the same priors for the initial state that L-W did in their code.

However, when I use Eviews to do this maximization, I get the message "Failure to improve likelihood (non-zero gradients) after 62 iterations," and the estimates Eviews settles on are not that close to what L-W got. I get the same results using Eview's other options (Newton-Raphson, etc.).


The second replication issue we had was trying to replicate the much simpler Stock (1986) model for the potential output. The workfile is attached here.

Watson.WF1

(67.25 KiB) Downloaded 399 times

[For this model, using the Newton Raphson method, we're able to replicate the Stock results, at least approximately. However, the same "failed to improve log likelihood" message as above shows up, and if I look at the summary of the gradients, they're quite far from 0.

What alarms me is that I don't even get the same log likelihood values when I compare Eviews to R when evaluating it at the same point. Does anyone here have experience with these kinds of state space issues, and have any insight into what Eviews is doing with State Space models?

[EViews Glenn]

A couple of things which may help. I took a brief look at the Watson example and did some fiddling around (technical term )

First, some findings...

If I change the starting values for your model to params closer to the final Watson results

@PARAM C(1) 1.501 C(2) -.57 C(3) -5.26 C(4) .007 C(5) -5

and then run using Marquardt (for the Watson sample of 1949 through 1984) I get convergence to values that are similar to those from Watson, but as you note, not exact (more on this in a bit). If I run using BFGS from those same initials, I get slightly different results. If I run using BHHH, I get non-convergence at parameters that are similar.

Next, a few comments...

First off, I don't expect the results to match those from Watson exactly as the data don't appear to be the same. If I run the ARIMA(1,1,0) that Watson runs, I get an AR coefficient that differs a fair amount. Note that this difference persists whether I estimate the model using conditional least squares or ML.

Second, I'm convinced that the numeric derivatives for this parameterization of the problem are poor. I believe that this explains why the BHHH/Newton and BFGS results differ so much, and why you are getting the non-zero gradients. For that reason, I would trust the BFGS results more than the others.

Third, as to likelihoods, it doesn't surprise me that you are finding differences between EViews and R. Reported likelihoods in many problems are problematic as one never knows whether inessential constants are included. Further, the problem in state space formulations is that different implementations will treat the initial conditions differently. Note that you can, from what I can tell, get very close to the Watson likelihood by adding T*log(2PI)/2 to the EViews likelihood to remove the inessential constants. I'm curious as what the R implementation says it does.

Lastly, I'll note that these problems are difficult to estimate. I will note that you may require a more sequential approach in which you play with different algorithms in different parts of the estimation. And starting values appear to matter a great deal.

I'll poke around a bit more but wanted to give you some initial things to think about. I'm also going to look into whether we can improve the numeric derivatives, but as you probably know that's easier said than done.

[daktx2]

Thanks for looking into this so deeply Glenn! It really is helpful.

You hit the nail on the head with what the R code is doing with the likelihood. I think my data also is of a slightly different vintage than the most recent L-W data, so some slight discrepancies are to be expected.

These kinds of models are indeed difficult to estimate! The issues we're seeing (bad numerics, starting values mattering, etc.) are often symptoms of the data not informing some of the parameters well! We might end up needing a fully Bayesian MCMC approach for estimation. That way, we'd avoid some of the issues with maximizing the oddly shaped likelihoods that arise from state space models. Though, this comes at a not insignificant computational cost!

Let me know what you find w/r/t the numeric derivatives, and if you find that a reparameterization might help. Thanks again!

[khnaqvi]

Hi I am using state space model to estimate the income elasticity of imports and price elasticity of imports. I am attaching the file, I am having these difficulties:
1. The standard errors of the coefficients are NA.
2. The smooth state estimates for the price elasticity are constant throughout the estimated sample.
I have use the following specification: Where (rm is real imports, rer is real exchange rate and gdp is gross domestic product. While rm_trend stands for the HP filtered series for imports, gdp_trend is also HP filtered series for the GDP and rer_trend is likewise the HP filtered series for real exchange rate. Please see the estimated model "sspace_model" in the attached file.

@signal log(rm_trend) = c(1)+pi*log(gdp_trend) + el*log(rer_trend)

@state pi = pi(-1) +[var=exp(c(2))]
@state el = el(-1) + [var=exp(c(4))]

Needing your advice.

thanks
Hassan