Hello. First time posting.
I was trying to make migration from TSP to Eviews. However, I couldn't reproduce the result of the original TSP program using Kalman filter. I found that the state vectors calculated in TSP and Eviews are different.
Attached please find the Excel file which contains both programs.
Sheet "TSP PGM" contains TSP version, and Sheet "Eviews PGM" contains Eviews version. (Note: This is simplified version of original program, so it contains only one observation.) The state vector calculated by TSP is (-0.06, -0.02, -0.02, 0.44), whereas that of Eviews is (0.2, 0, 0, -0.1). I also calculated the state vector using formula written in wikipedia, and that result matched TSP (see Sheet "Check by Excel").
My Eviews is version 7.1. I would appreciate any information on this matter.
Kalman filter state vector unmatch
Moderators: EViews Gareth, EViews Moderator
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
Hi, welcome to EViews, glad to have you join us...
Two things going on here...
First, the EViews specification doesn't quite match the specification you appear to be using in your calculation. In particular, you've set the signal variances to zero, which doesn't match the R signal variance matrix (which is a diagonal matrix with unit variances) that you used in your spreadsheet. You'll want to modify the EViews signal equations to add the variances.
Second, as noted in the manual, the EViews initial state vector and variance must be specified as P(1|0) and s(1|0), not as P(0|0) and s(0|0) as you appear to be doing. For s(0|0) this doesn't matter in your specification since applying the one-step-ahead update yields the same vector, but this difference does matter for P(1|0). Translating from P(0|0) to P(1|0) using the expression
P(1|0) = F*P(0|0)*F' + Q
and then entering those values in as the initial conditions solves the second problem. Note that the invariance of the state vector to the prediction update is not a general result and typically one would have to obtain s(1|0).
Modifying your program, we have
which yields the desired results. I've specified each of the individual elements of the modified svar0, but you could, if you want, specify matrices for F and Q instead, and then use an expression to modify your original svar0.
Hope this answers your question.
(Incidentally, answering this question was made a lot easier by all of the information you provided in the spreadsheet. In particular, it would have taken me forever to find the missing R signal variances had you not been so explicit about what you were trying to do and had I only had the TSP code to go by... Thanks...)
Two things going on here...
First, the EViews specification doesn't quite match the specification you appear to be using in your calculation. In particular, you've set the signal variances to zero, which doesn't match the R signal variance matrix (which is a diagonal matrix with unit variances) that you used in your spreadsheet. You'll want to modify the EViews signal equations to add the variances.
Second, as noted in the manual, the EViews initial state vector and variance must be specified as P(1|0) and s(1|0), not as P(0|0) and s(0|0) as you appear to be doing. For s(0|0) this doesn't matter in your specification since applying the one-step-ahead update yields the same vector, but this difference does matter for P(1|0). Translating from P(0|0) to P(1|0) using the expression
P(1|0) = F*P(0|0)*F' + Q
and then entering those values in as the initial conditions solves the second problem. Note that the invariance of the state vector to the prediction update is not a general result and typically one would have to obtain s(1|0).
Modifying your program, we have
Code: Select all
wfcreate biwatest m 2011:02 2011:02
series Y1
Y1.fill(o=2011M2) 0.1
series Y2
Y2.fill(o=2011M2) 0.2
smpl 2011M2 2011:02
sspace ss1
ss1.append @signal Y1=sv1+sv4 + [var=1]
ss1.append @signal Y2=sv1 + [var=1]
ss1.append @state sv1= sv1(-1) + sv2(-1) + [var=1]
ss1.append @state sv2= sv1(-1)
ss1.append @state sv3= sv2(-1)
ss1.append @state sv4= sv4(-1) + [var=1]
vector(4) svec0
svec0.fill 0,0,0,1
sym svar0 = @identity(4)
svar0(1,1) = 3
svar0(1,3) = 1
svar0(2,1) = 1
svar0(3,1) = 1
svar0(4,4) = 2
ss1.append @mprior svec0
ss1.append @vprior svar0
ss1.ml(showopts,m=1000,c=1e-9)
ss1.makestate(t=filt) f*
which yields the desired results. I've specified each of the individual elements of the modified svar0, but you could, if you want, specify matrices for F and Q instead, and then use an expression to modify your original svar0.
Hope this answers your question.
(Incidentally, answering this question was made a lot easier by all of the information you provided in the spreadsheet. In particular, it would have taken me forever to find the missing R signal variances had you not been so explicit about what you were trying to do and had I only had the TSP code to go by... Thanks...)
Last edited by EViews Glenn on Wed Apr 13, 2011 12:22 pm, edited 1 time in total.
Re: Kalman filter state vector unmatch
Thank you very much! It perfectly solved my problem.
Re: Kalman filter state vector unmatch
The above program Mr. EViews Glenn posted at Wed Apr 13, 2011 4:10 am used to yield the state vector (-0.06, -0.02, -0.02, 0.44) as intended. But when run on the newest version of EViews, it yields the state vector (-0.08, -0.06, -0.02, 0.44). I'd like to know why this change occurred. I would appreciate any information on this matter.
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
Could you please repost the spreadsheet if possible?
Re: Kalman filter state vector unmatch
Attached please find the spreadsheet I previously posted at Tue Apr 12, 2011 7:19 pm.
- Attachments
-
- Kalmantest.xls
- (27 KiB) Downloaded 351 times
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
If you change your option to smoothing, you match the TSP results (note that your TSP command smooths).
ss1.makestate(t=smooth) f*
[edit: However, in this case the smooths and the filtered should align. All of the reported one-step ahead and smoothed look fine and all of our test cases report that there is no issue there, but I think there is an issue with our output of the filtered. I'm investigating further.]
ss1.makestate(t=smooth) f*
[edit: However, in this case the smooths and the filtered should align. All of the reported one-step ahead and smoothed look fine and all of our test cases report that there is no issue there, but I think there is an issue with our output of the filtered. I'm investigating further.]
Last edited by EViews Glenn on Tue Feb 04, 2020 7:56 pm, edited 1 time in total.
Re: Kalman filter state vector unmatch
Thank you for your reply. I see that "makestate(t=filt)" and "makestate(t=smooth)" used to yield the same state vector for my one-period sample, but now they yield different state vectors. I presume that change occurred at November, 2019 patch, which describes "Fix for kalman filter contemporaneous updates for output of saved series." Unfortunately, we are now using Kalman filter without smoothing, so the change is fatal for us.
It seems that "makestate(t=filt)" command used to yield "the filtered (contemporaneous) state vector" noted as alpha_t in the User's Guide Chapter "State Space Models and the Kalman Filter", but now it yields "the final state vector" noted as alpha_T in the User's Guide. However, "Make State Series" section in the User's Guide says it saves alpha_t, not alpha_T. So I suspect that "makestate(t=filt)" command now does not match the description of the User's Guide. I'd like to know if there is a way to save alpha_t in the newest version of EViews.
It seems that "makestate(t=filt)" command used to yield "the filtered (contemporaneous) state vector" noted as alpha_t in the User's Guide Chapter "State Space Models and the Kalman Filter", but now it yields "the final state vector" noted as alpha_T in the User's Guide. However, "Make State Series" section in the User's Guide says it saves alpha_t, not alpha_T. So I suspect that "makestate(t=filt)" command now does not match the description of the User's Guide. I'd like to know if there is a way to save alpha_t in the newest version of EViews.
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
Thanks for the additional info. I'll get back to you as soon as possible.
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
Apologies. Found the issue and we will get a fix out shortly. I'm very sorry for the inconvenience. Thanks again for your help.
-
- EViews Developer
- Posts: 2672
- Joined: Wed Oct 15, 2008 9:17 am
Re: Kalman filter state vector unmatch
The patch has been released.
Re: Kalman filter state vector unmatch
We've downloaded the patch and it seems OK. Thank you for the quick fix.
Who is online
Users browsing this forum: No registered users and 36 guests