Kalman filter state vector unmatch

For technical questions regarding estimation of single equations, systems, VARs, Factor analysis and State Space Models in EViews. General econometric questions and advice should go in the Econometric Discussions forum.

Moderators: EViews Gareth, EViews Moderator

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Kalman filter state vector unmatch

Postby watabehj » Tue Apr 12, 2011 2:19 am

Hello. First time posting.

I was trying to make migration from TSP to Eviews. However, I couldn't reproduce the result of the original TSP program using Kalman filter. I found that the state vectors calculated in TSP and Eviews are different.
Attached please find the Excel file which contains both programs.
Kalmantest.xls
Excel file concerning Kalman filter state vector unmatch
(27 KiB) Downloaded 259 times

Sheet "TSP PGM" contains TSP version, and Sheet "Eviews PGM" contains Eviews version. (Note: This is simplified version of original program, so it contains only one observation.) The state vector calculated by TSP is (-0.06, -0.02, -0.02, 0.44), whereas that of Eviews is (0.2, 0, 0, -0.1). I also calculated the state vector using formula written in wikipedia, and that result matched TSP (see Sheet "Check by Excel").

My Eviews is version 7.1. I would appreciate any information on this matter.

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Tue Apr 12, 2011 11:10 am

Hi, welcome to EViews, glad to have you join us...

Two things going on here...

First, the EViews specification doesn't quite match the specification you appear to be using in your calculation. In particular, you've set the signal variances to zero, which doesn't match the R signal variance matrix (which is a diagonal matrix with unit variances) that you used in your spreadsheet. You'll want to modify the EViews signal equations to add the variances.

Second, as noted in the manual, the EViews initial state vector and variance must be specified as P(1|0) and s(1|0), not as P(0|0) and s(0|0) as you appear to be doing. For s(0|0) this doesn't matter in your specification since applying the one-step-ahead update yields the same vector, but this difference does matter for P(1|0). Translating from P(0|0) to P(1|0) using the expression

P(1|0) = F*P(0|0)*F' + Q

and then entering those values in as the initial conditions solves the second problem. Note that the invariance of the state vector to the prediction update is not a general result and typically one would have to obtain s(1|0).

Modifying your program, we have

Code: Select all

wfcreate biwatest m 2011:02 2011:02            
series Y1            
Y1.fill(o=2011M2) 0.1            
series Y2            
Y2.fill(o=2011M2) 0.2            
smpl 2011M2 2011:02            
sspace ss1            
  ss1.append @signal Y1=sv1+sv4 + [var=1]         
  ss1.append @signal Y2=sv1 + [var=1]            
  ss1.append @state sv1= sv1(-1) + sv2(-1) + [var=1]            
  ss1.append @state sv2= sv1(-1)          
  ss1.append @state sv3= sv2(-1)            
  ss1.append @state sv4= sv4(-1) + [var=1]            
vector(4) svec0            
svec0.fill 0,0,0,1            
sym svar0 = @identity(4)            
svar0(1,1) = 3
svar0(1,3) = 1
svar0(2,1) = 1
svar0(3,1) = 1
svar0(4,4) = 2   
ss1.append @mprior svec0            
ss1.append @vprior svar0            
ss1.ml(showopts,m=1000,c=1e-9)            
ss1.makestate(t=filt) f*

which yields the desired results. I've specified each of the individual elements of the modified svar0, but you could, if you want, specify matrices for F and Q instead, and then use an expression to modify your original svar0.

Hope this answers your question.

(Incidentally, answering this question was made a lot easier by all of the information you provided in the spreadsheet. In particular, it would have taken me forever to find the missing R signal variances had you not been so explicit about what you were trying to do and had I only had the TSP code to go by... Thanks...)
Last edited by EViews Glenn on Wed Apr 13, 2011 12:22 pm, edited 1 time in total.

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Re: Kalman filter state vector unmatch

Postby watabehj » Tue Apr 12, 2011 8:59 pm

Thank you very much! It perfectly solved my problem. :D

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Re: Kalman filter state vector unmatch

Postby watabehj » Fri Jan 31, 2020 12:51 am

The above program Mr. EViews Glenn posted at Wed Apr 13, 2011 4:10 am used to yield the state vector (-0.06, -0.02, -0.02, 0.44) as intended. But when run on the newest version of EViews, it yields the state vector (-0.08, -0.06, -0.02, 0.44). I'd like to know why this change occurred. I would appreciate any information on this matter.

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Fri Jan 31, 2020 1:39 pm

Could you please repost the spreadsheet if possible?

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Re: Kalman filter state vector unmatch

Postby watabehj » Sun Feb 02, 2020 5:21 pm

Attached please find the spreadsheet I previously posted at Tue Apr 12, 2011 7:19 pm.
Attachments
Kalmantest.xls
(27 KiB) Downloaded 46 times

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Tue Feb 04, 2020 11:26 am

If you change your option to smoothing, you match the TSP results (note that your TSP command smooths).

ss1.makestate(t=smooth) f*

[edit: However, in this case the smooths and the filtered should align. All of the reported one-step ahead and smoothed look fine and all of our test cases report that there is no issue there, but I think there is an issue with our output of the filtered. I'm investigating further.]
Last edited by EViews Glenn on Tue Feb 04, 2020 7:56 pm, edited 1 time in total.

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Re: Kalman filter state vector unmatch

Postby watabehj » Tue Feb 04, 2020 7:51 pm

Thank you for your reply. I see that "makestate(t=filt)" and "makestate(t=smooth)" used to yield the same state vector for my one-period sample, but now they yield different state vectors. I presume that change occurred at November, 2019 patch, which describes "Fix for kalman filter contemporaneous updates for output of saved series." Unfortunately, we are now using Kalman filter without smoothing, so the change is fatal for us.

It seems that "makestate(t=filt)" command used to yield "the filtered (contemporaneous) state vector" noted as alpha_t in the User's Guide Chapter "State Space Models and the Kalman Filter", but now it yields "the final state vector" noted as alpha_T in the User's Guide. However, "Make State Series" section in the User's Guide says it saves alpha_t, not alpha_T. So I suspect that "makestate(t=filt)" command now does not match the description of the User's Guide. I'd like to know if there is a way to save alpha_t in the newest version of EViews.

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Tue Feb 04, 2020 7:58 pm

Thanks for the additional info. I'll get back to you as soon as possible.

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Wed Feb 05, 2020 12:02 pm

Apologies. Found the issue and we will get a fix out shortly. I'm very sorry for the inconvenience. Thanks again for your help.

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

Re: Kalman filter state vector unmatch

Postby EViews Glenn » Thu Feb 06, 2020 2:24 pm

The patch has been released.

watabehj
Posts: 20
Joined: Tue Apr 12, 2011 1:19 am

Re: Kalman filter state vector unmatch

Postby watabehj » Wed Feb 12, 2020 5:15 pm

We've downloaded the patch and it seems OK. Thank you for the quick fix.


Return to “Estimation”

Who is online

Users browsing this forum: No registered users and 4 guests