## Understanding the difference between Sspace and R's DLM

For technical questions regarding estimation of single equations, systems, VARs, Factor analysis and State Space Models in EViews. General econometric questions and advice should go in the Econometric Discussions forum.

Moderators: EViews Gareth, EViews Moderator

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Understanding the difference between Sspace and R's DLM

Hi,

I am trying to convert a model I had put together using R's DLM (Dynamic Linear Model) into Eviews Sspace framework. The model is a Nelson Siegal style latent factor model, using inflation expectations rather than bond yields. I can't seem to get estimates anywhere near what I get with R. I've provided the same starting parameters, using the same data, but I get an error "Near singular matrix in "DLM_INFE.ML" on line 525." I suspect the issue is coming in with how I am specifying my error variances, currently I am imposing non-negativity by

Code: Select all

`dlm_infe.append @evar var(g1) = (infe_dlm(18)^2) `

When I change this to the exponential

Code: Select all

`exp(infe_dlm(18))`

I don't get the error, but I do get junk results.

As you are probably aware, the state space representation of the NS model popularised by Diebold and Li, has a VAR(1) process for the state. In R, I parameterise the state VCV matrix directly by setting the matrix Q = to a 3x3 lower triangular matrix of parameters to be estimated (see code below), which then multiply by it's transpose to impose non-negativity and symmetry. So I suspect this is where I am going wrong with my Eviews model, as everything else seems to be the same, however, I am directly imposing the constraints in Eviews.

My Eviews code is below:

Code: Select all

`if @isobject("dlm_infe") =1 then   delete dlm_infeendifvector(6) mprior = 0sym(6,6) vprior = 0vprior(1,1) = 0.1vprior(2,2) = 0.1vprior(3,3) = 0.1vprior(4,4) = 0.1vprior(5,5) = 0.1vprior(6,6) = 0.1param infe_dlm(1) 0.06for !i = 2 to 10   param infe_dlm(!i) 0.5nextfor !i = 11 to 17   param infe_dlm(!i) -1.5nextfor !i = 18 to 23   param infe_dlm(!i) -1next'sspace dlm_infe dlm_infe.append @vprior vpriordlm_infe.append @mprior mprior' Measurment errorsdlm_infe.append @ename e1dlm_infe.append @ename e2dlm_infe.append @ename e3dlm_infe.append @ename e4dlm_infe.append @ename e5dlm_infe.append @ename e6dlm_infe.append @ename e7'State errorsdlm_infe.append @ename g1dlm_infe.append @ename g2dlm_infe.append @ename g3' Measurment equationsdlm_infe.append @signal infe_1 = level + ((1-exp(-infe_dlm(1)*40 ))/(infe_dlm(1)*40))*slope+(((1-exp(-infe_dlm(1)*40))/(infe_dlm(1)*40)-exp(infe_dlm(1)*40)))*curv + e1   dlm_infe.append @signal infe_2 = level + ((1-exp(-infe_dlm(1)*1))/(infe_dlm(1)*1))*slope+(((1-exp(-infe_dlm(1)*1))/(infe_dlm(1)*1)-exp(infe_dlm(1)*1)))*curv +e2dlm_infe.append @signal infe_3 = level + ((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4))*slope+(((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4)-exp(infe_dlm(1)*4)))*curv +e3dlm_infe.append @signal infe_4 = level + ((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4))*slope+(((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4)-exp(infe_dlm(1)*4)))*curv +e4dlm_infe.append @signal infe_5 = level + ((1-exp(-infe_dlm(1)*8))/(infe_dlm(1)*8))*slope+(((1-exp(-infe_dlm(1)*8))/(infe_dlm(1)*8)-exp(infe_dlm(1)*8)))*curv +e5dlm_infe.append @signal infe_6 = level + ((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4))*slope+(((1-exp(-infe_dlm(1)*4))/(infe_dlm(1)*4)-exp(infe_dlm(1)*4)))*curv +e6dlm_infe.append @signal infe_7 = level + ((1-exp(-infe_dlm(1)*8))/(infe_dlm(1)*8))*slope+(((1-exp(-infe_dlm(1)*8))/(infe_dlm(1)*8)-exp(infe_dlm(1)*8)))*curv +e7' State equationsdlm_infe.append @state level = infe_dlm(2)*level(-1)+infe_dlm(3)*slope(-1)+infe_dlm(4)*curv(-1)+(1-infe_dlm(2))*mean_level(-1)-infe_dlm(3)*mean_slope(-1)-infe_dlm(4)*mean_curv(-1) +g1 dlm_infe.append @state slope = infe_dlm(5)*level(-1)+infe_dlm(6)*slope(-1)+infe_dlm(7)*curv(-1)-infe_dlm(5)*mean_level(-1)+(1-infe_dlm(6))*mean_slope(-1)-infe_dlm(7)*mean_curv(-1) +g2   dlm_infe.append @state curv = infe_dlm(8)*level(-1)+infe_dlm(9)*slope(-1)+infe_dlm(10)*curv(-1)-infe_dlm(8)*mean_level(-1)-infe_dlm(9)*mean_slope(-1)+(1-infe_dlm(10))*mean_curv(-1) +g3dlm_infe.append @state mean_level = mean_level(-1)dlm_infe.append @state mean_slope = mean_slope(-1) dlm_infe.append @state mean_curv = mean_curv(-1) dlm_infe.append @evar var(e1) =  (infe_dlm(11)^2) 'exp(infe_dlm(11)) 'dlm_infe.append @evar var(e2) =  (infe_dlm(12)^2) 'exp(infe_dlm(12)) 'dlm_infe.append @evar var(e3) = (infe_dlm(13)^2) 'exp(infe_dlm(13)) 'dlm_infe.append @evar var(e4) =  (infe_dlm(14)^2) 'exp(infe_dlm(14))'dlm_infe.append @evar var(e5) = (infe_dlm(15)^2) 'exp(infe_dlm(15)) ' dlm_infe.append @evar var(e6) = (infe_dlm(16)^2) 'exp(infe_dlm(16)) 'dlm_infe.append @evar var(e7) = (infe_dlm(17)^2) 'exp(infe_dlm(17)) '' State error variancesdlm_infe.append @evar var(g1) = (infe_dlm(18)^2) 'exp(infe_dlm(18)) 'dlm_infe.append @evar var(g2) = (infe_dlm(19)^2) 'exp(infe_dlm(19)) 'dlm_infe.append @evar var(g3) = (infe_dlm(20)^2) 'exp(infe_dlm(20)) 'dlm_infe.append @evar cov(g1,g2) = infe_dlm(21)dlm_infe.append @evar cov(g1,g3) = infe_dlm(22)dlm_infe.append @evar cov(g2,g3) = infe_dlm(23)smpl 1989:3 2020:1dlm_infe.ml`

And my R code (relevant sections, the matrix qq and W). Note, the exph[] vector simply contains the following, c(40, 1,4,4,8,4,8) which I have hard entered above in the eviews code.

Code: Select all

`buildNSInf <- function(p){    exph <- exph    # Deibold and Li Loadings  ff <- c(1,(1-exp(-p[1]*exph[1]))/(p[1]*exph[1]),((1-exp(-p[1]*exph[1]))/(p[1]*exph[1])-exp(-p[1]*exph[1])),0,0,0,          1,(1-exp(-p[1]*exph[2]))/(p[1]*exph[2]),((1-exp(-p[1]*exph[2]))/(p[1]*exph[2])-exp(-p[1]*exph[2])),0,0,0,          1,(1-exp(-p[1]*exph[3]))/(p[1]*exph[3]),((1-exp(-p[1]*exph[3]))/(p[1]*exph[3])-exp(-p[1]*exph[3])),0,0,0,          1,(1-exp(-p[1]*exph[4]))/(p[1]*exph[4]),((1-exp(-p[1]*exph[4]))/(p[1]*exph[4])-exp(-p[1]*exph[4])),0,0,0,          1,(1-exp(-p[1]*exph[5]))/(p[1]*exph[5]),((1-exp(-p[1]*exph[5]))/(p[1]*exph[5])-exp(-p[1]*exph[5])),0,0,0,          1,(1-exp(-p[1]*exph[6]))/(p[1]*exph[6]),((1-exp(-p[1]*exph[6]))/(p[1]*exph[6])-exp(-p[1]*exph[6])),0,0,0,          1,(1-exp(-p[1]*exph[7]))/(p[1]*exph[7]),((1-exp(-p[1]*exph[7]))/(p[1]*exph[7])-exp(-p[1]*exph[7])),0,0,0            )        gg <- c(p[2],p[3],p[4],(1-p[2]),-p[3],-p[4],          p[5],p[6],p[7],-p[5],(1-p[6]),-p[7],          p[8],p[9],p[10],-p[8],-p[9],(1-p[10]),          0, 0,0,1,0,0,          0, 0,0,0,1,0,          0, 0,0,0,0,1  )    qq <- c(p[11],0,0,0,0,0,          p[12],p[13],0,0,0,0,          p[14],p[15],p[16],0,0,0,          0,0,0,0,0,0,          0,0,0,0,0,0,          0,0,0,0,0,0)    hh <- c(  p[17],0,0,0,0,0,0,            0,p[18],0,0,0,0,0,            0,0,p[19],0,0,0,0,            0,0,0,p[20],0,0,0,            0,0,0,0,p[21],0,0,            0,0,0,0,0,p[22],0,            0,0,0,0,0,0,p[23])    FF1 <- matrix(ff,7,6, byrow = TRUE)      GG1 <- matrix(gg, 6, 6, byrow = TRUE)    W1 <- matrix(qq,6,6, byrow = TRUE)    V1 <- matrix(hh,7,7, byrow = TRUE)      return(list(        m0 = c(0,0,0,0,0,0),      C0 = bdiag(diag(0.1,3),diag(0.1,3)),     FF = FF1,    GG = GG1,    W = W1%*%t(W1),    V = V1%*%t(V1)      ))      }ggp <- rep(0.5,9)wp <- rep(-1,6)vp <- rep(-1.5,7)dat1 <- matrix(as.numeric(unlist(RBA1[,-c(1:2)])),nrow=nrow(RBA1))LC <- rep(exp(-8),25)UC <- rep(exp(12),25)LC[2:10] <- -InfUC[2:10] <- InfNSml <-  dlmMLE(dat1, parm =c(lam,ggp,wp,vp) , build = buildNSInf,                lower=LC ,                upper=UC  ,                control = list(trace = 6, REPORT = 5, maxit = 500)) `

Of course, I can call R from Eviews to run this code (which, by the way is a fantastic application) but I would really like to understand why R's DLM produces something sensible and why I can't recreate this with Eviews. By sensible, my model converges and the the states look reasonable.

I'm not saying DLM is correct, the error here is no-doubt my own, I just want to understand it so I know for next time :)

Thanks

I am using Eviews 12 and have updated for the latest patch

More details on R's DLM can be found here:
https://cran.r-project.org/web/packages ... es/dlm.pdf

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Re: Understanding the difference between Sspace and R's DLM

Hi,

I've attached a workfile if it helps - any guidance would be appreciated.

Attachments
rba_infe.wf1

EViews Glenn
EViews Developer
Posts: 2659
Joined: Wed Oct 15, 2008 9:17 am

### Re: Understanding the difference between Sspace and R's DLM

Been down a few rabbit holes trying to track this down, but I just noticed that the signal equation specifications in your sspace object don't match the Diebold and Li paper and that they differ from the R specifications. I'm not sure that this is the sole reason for the issues you are seeing, but I believe that the last exponential term should have a negative sign inside the exp function.

For example, you have,

Code: Select all

`dlm_infe.append @signal infe_1 = level + ((1-exp(-infe_dlm(1)*40 ))/(infe_dlm(1)*40))*slope+(((1-exp(-infe_dlm(1)*40))/(infe_dlm(1)*40)-exp(infe_dlm(1)*40)))*curv + e1`

That specification should, I believe, be

Code: Select all

`dlm_infe.append @signal infe_1 = level + ((1-exp(-infe_dlm(1)*40 ))/(infe_dlm(1)*40))*slope+(((1-exp(-infe_dlm(1)*40))/(infe_dlm(1)*40)-exp(-infe_dlm(1)*40)))*curv + e1`

Haven't done any additional testing yet, but I thought I'd point this out before I went too much further. Note also that I haven't looked carefully at your state equation specifications yet.

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Re: Understanding the difference between Sspace and R's DLM

Thanks Glenn, indeed an error. In regards to the state equations, I have specified this model a little differently to how I have seen others on this forum specify this model. This is because I originally estimated the model using R's DLM package, which doesn't allow an intercept in the state equation, so I have moved the intercepts into the state vector - hopefully I've done this correctly.

EViews Glenn
EViews Developer
Posts: 2659
Joined: Wed Oct 15, 2008 9:17 am

### Re: Understanding the difference between Sspace and R's DLM

Took a quick look at the state specs. Just double checking that you did want to extend Diebold and Li by specifying the states as a VAR(2).

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Re: Understanding the difference between Sspace and R's DLM

I don't think that is the intention, no.

The original specification was:

Y = F*X(t) + E(t)
(X(t)-U) = G*(X(t-1)-U) + W(t)

In R, the state space model doesn't allow for an intercept in the state equation (which the above has) so to get around that I have moved the intercept into the state vector:

X(t) = U +G*X(t) -G*U +W(t)
X(t) = (I-G)*U + G*X(t-1) +W(t)

So now the state vector becomes

[level, slope, curv, U(level) , U(slope), U(curv)]

The the first three state variables follow an AR(1) process and the intercepts (what I have called U) are non-time varying state variables.

That was the intention anyway, maybe it is not the correct way to approach the problem?

EViews Glenn
EViews Developer
Posts: 2659
Joined: Wed Oct 15, 2008 9:17 am

### Re: Understanding the difference between Sspace and R's DLM

Yes, what you are doing makes perfect sense. I wasn't quite sure what the "mean_" variables were doing since I hadn't really looked at the original specification, and think I just mushed all of the variables together in my mind. Sorry to worry you.

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Re: Understanding the difference between Sspace and R's DLM

Hi,

Just wondering if there has been any further developments on this? I have been playing about with thing, trying alternative measurement equations for instance, I still get the singular matrix error.

I also noticed that if I comment out the state covariance commands, the singular matrix error doesn't appear, but unsurprisingly the model is junk.

Be good to know what is happening here.

Thanks

Elderfield.A
Posts: 59
Joined: Thu Jul 11, 2019 10:47 pm

### Re: Understanding the difference between Sspace and R's DLM

I've made a bit of progress on this, my starting parameters were the causing the issue. When I change these, I can get the model to estimate, not converge but at least estimate. So I think that is a good start!

I guess that puts this one to bed.