EViews.com

Hello.
Would appreciate some help figuring out what's causing the errors.

I'm trying to put together a state-space model with all coefficients/variances already estimated, in order to use Eviews Kalman filter/smoother.
For some (but not all) combinations of my 200-300 signal variables I get "matrix near singular" errors while trying to initialize/estimate the state-space model.

I'm not sure what it's trying to estimate other than the final state vector, which I can try to estimate on my own using Kalman filter formulas and it works w/o a hitch (I do have to use @pinverse though, due to signal idiosyncratic error covariance matrix being rank-deficient).

A attach a small example workfile.
Estimating statespace 'now' on a sample of 1993m1 1993m1 yields the error.
Running the code below to calculate the final state works fine (and produces something fairly close to Eviews calculations for specifications where now.ml does not fail)
Thanks in advance to anyone who can help.

PS: my signal idiosyncratic error covariance matrix is rank-deficient in most specifications, even when now.ml goes w/o error.

All that estimate does in a model without coefficients is to run the filter to initialize internal values.

I haven't thought about the KF in a long time, but my first instinct was that you having to use @pinverse in some case was likely to be related to the issue. A quick check of the internals confirms that we are detecting singularity when computing the gain and that @transpose(H)*P1_0*H+R is the problematic term.

Thanks for a reply.
I can post a file with pretty much the SS model above, dropping just one variable, and the 'estimation' would run just fine despite the same matrix still being rank deficient. Which tells me you're using some version of @pinverse code while computing the first step of the filter, but perhaps it is coded differently from the @pinverse code, cause the latter works.

Here's a slightly different file, with a rank-deficient idiosyncratic covariance matrix where Eviews estimation works.

@inverse(@transpose(H)*P1_0*H+R) would give a 'near-singular' matrix error. The code above (with @pinverse) and the Eviews now.ml produce results differing in the 4th decimal.

I suspect the Kalman state filtering formulas use somewhat different implementation of the pseudo-inverse.

I'm afraid that I wasn't clear in the earlier post. Our implementation of the kfilter does not use a pinverse.

Is there a reference on how it works w/o it? I'm trying to understand why it works in some rank-deficient signal variance cases and fails in others, so that I could possibly prevent failures early.

We use a standard inverse. My guess is that the rank deficiencies are not numeric in the other cases.