The (not so) Dynamic forecasting in State Space models

[jason_ll]

Hello,
I've estimated a state-space model and would like to produce out-of-sample forecasts for all the signal and state variables dynamically.

My signal equations contain lagged dependent variables.

The problem is that EViews treats those lags as exogenous variables. This makes it impossible for me to produce forecasts that are more than 1 step ahead.

For instance, let's assume "var" is a signal and "stat" is a state variable:

Code: Select all

var = c(1) + c(2)*var(-1) + c(3)*stat +e1
@state stat = c(4)*stat(-1) +e2

EViews treats var(-1) as an exogenous variable, instead of recalculating it dynamically for the out-of-sample forecast.
Is there any way I can rewrite this to fix this problem?

Question 2 (less important): when forecasting, is there any way I can ask EViews to not overwrite historical data where it exists? (in other words, treat all actuals as exogenous, and forecast dynamically when the actuals do not exist - similar to checking the "For all other endogenous - Exclude for periods where Actuals exist" box in the model object)

Thanks a lot!

[jason_ll]

No answer. Does this mean it can't be done?

[xprimexinverse]

Hi,

Take a look at the program below. I think it's possible to do it if you use the makefilter command and set-up a loop. I demonstrate this below by forecasting an AR(1) model from both the equation and sspace objects. This is a simple example, but the general idea should hold. You can do variations of this where you can re-estimate the model parameters, but I don't do that here.

You can view the program below or download it here:

dyn_sspace.prg

(3.51 KiB) Downloaded 292 times

Cheers,
Graeme

Code: Select all

' --------------------------------------------------------------------------------------------------------------------
'   DESCRIPTION:    State-Space model dynamic forecasting with lagged signal
'
'   DATE:           05 / 03 / 2017
'
'   AUTHOR:         Graeme Walsh
' --------------------------------------------------------------------------------------------------------------------

' ********************************************************************************************************************
' Pre-liminary set-up
' ********************************************************************************************************************
 
  wfcreate(wf=example) Q 1980Q1 2017Q4

' ********************************************************************************************************************
' Create Simulated Data
' ********************************************************************************************************************

  rndseed 123456

  series e1 = nrnd
  series Y = 0

  model AR2
  AR2.append Y = 3.20 + 0.22 * Y(-1) + 0.15 * Y(-2) + e1
  AR2.solveopt(s=d,d=d)
  AR2.solve

  series Y = Y_0

' ********************************************************************************************************************
' Restructure page for out-of-sample forecasting
' ********************************************************************************************************************
 
  pagestruct(end=2020Q4)

' ********************************************************************************************************************
' AR(1) Model
' ********************************************************************************************************************

' Estimate model
  smpl 1980Q1 2017Q4
  equation ARMOD.ls Y = c(1) + c(2) * Y(-1)

' Out-of-sample forecast
  smpl 2018Q1 2020Q4
  ARMOD.forecast YF1

' ********************************************************************************************************************
' State-space model
' ********************************************************************************************************************

' Create the sspace object
  sspace dyn_mod 

' Measurement equation
  dyn_mod.append @signal Y = c(1) + c(2) * Y(-1) + 0 * sv1 + eps
  dyn_mod.append @ename eps
  dyn_mod.append @evar var(eps) = exp(c(4))

' Dummy state equation (sv1)
  dyn_mod.append @state sv1 = 0 * sv1 + eps_sv1
  dyn_mod.append @ename eps_sv1
  dyn_mod.append @evar var(eps_sv1) = 0 
   
' Estimate model by maximum likelihood
  smpl 1980q1 2017q4
  dyn_mod.ml

' Make copy of model with set coefficients
  dyn_mod.makefilter new_dyn_mod

' Copy the orginal series
  series Y_COPY = Y
   
' Loop for dynamic forecasting
  for !i = 0 to 11
     smpl @first 2018Q1+!i
       new_dyn_mod.ml
       smpl 2018Q1+!i 2020Q4
       new_dyn_mod.forecast @signal *F2
       series Y = YF2
  next
 
' Restore original series and tidy up
   smpl @all
   series YF2 = Y
   series Y = Y_COPY
   delete Y_COPY
 
' ********************************************************************************************************************
' Compare Forecasts
' ********************************************************************************************************************

  show Y YF1 YF2
   
' -------------------------------------------------------------------------------------------------------------------
'   END OF PROGRAM
' -------------------------------------------------------------------------------------------------------------------


[jason_ll]

Thanks Gareme for sharing your code. While this certainly works, I feel like the EViews built-in function should be doing a better job, and giving us more options.

[xprimexinverse]

Hi Jason,

Let me try again! =)

I understand that there appears to be some limitations to the sspace object; however, the state-space form is a very flexible mathematical structure, in the sense that many models, including the AR(1) model, have more than one state-space representation. The flexibility of the theory carries over to flexibility with the software.

I think the knack is to be aware of the state-space form used by the software (covered in the official documentation - but I'm sure you're aware of it anyway!) and use that to cast your model into a state-space representation such that the software can do all of the things you want it to do.

For example, an AR(1) model can be written with a lagged endogenous (measurement) variable in the observation equation or, equivalently (see footnote), an AR(1) model can be written with an augmented state vector where the lag variable is included in the state vector and spelled out in a state equation. The details of this are given in Harvey's textbook.

If you write the AR(1) model in the augmented state vector form (i.e. include the lag variable in the state vector and not as a pre-determined variable in the measurement equation) then the dynamic forecasting will work in EViews without the need for the makefilter + loop approach that I outlined in my previous post.

See the program below for an updated demonstration of this.

You can download the program here:

dyn_sspace2.prg

(4.34 KiB) Downloaded 284 times

Cheers,
Graeme

Footnote: OK, there are some minor differences which have implications for the uncertainty of the forecasts, but this is of second-order importance. For details, refer to Harvey.

Code: Select all

' --------------------------------------------------------------------------------------------------------------------
'   DESCRIPTION:    State-Space model dynamic forecasting with (1) lagged signal (2) augmented state
'
'   DATE:           08 / 03 / 2017
'
'   AUTHOR:         Graeme Walsh
' --------------------------------------------------------------------------------------------------------------------

' ********************************************************************************************************************
' Pre-liminary set-up
' ********************************************************************************************************************
 
  wfcreate(wf=example) Q 1980Q1 2017Q4

' ********************************************************************************************************************
' Create Simulated Data
' ********************************************************************************************************************

  rndseed 123456

  series e1 = nrnd
  series Y = 0

  model AR2
  AR2.append Y = 3.20 + 0.22 * Y(-1) + 0.15 * Y(-2) + e1
  AR2.solveopt(s=d,d=d)
  AR2.solve

  series Y = Y_0

' ********************************************************************************************************************
' Restructure page for out-of-sample forecasting
' ********************************************************************************************************************
 
  pagestruct(end=2020Q4)

' ********************************************************************************************************************
' AR(1) Model
' ********************************************************************************************************************

' Estimate model
  smpl 1980Q1 2017Q4
  equation ARMOD.ls Y = c(1) + c(2) * Y(-1)

' Out-of-sample forecast
  smpl 2018Q1 2020Q4
  ARMOD.forecast YF1

' ********************************************************************************************************************
' State-space model - AR(1) version 1
' ********************************************************************************************************************

' Create the sspace object
  sspace dyn_mod 

' Measurement equation
  dyn_mod.append @signal Y = c(1) + c(2) * Y(-1) + 0 * sv1 + eps
  dyn_mod.append @ename eps
  dyn_mod.append @evar var(eps) = exp(c(4))

' Dummy state equation (sv1)
  dyn_mod.append @state sv1 = 0 * sv1 + eps_sv1
  dyn_mod.append @ename eps_sv1
  dyn_mod.append @evar var(eps_sv1) = 0 
   
' Estimate model by maximum likelihood
  smpl 1980Q1 2017Q4
  dyn_mod.ml

' Make copy of model with set coefficients
  dyn_mod.makefilter new_dyn_mod

' Copy the orginal series
  series Y_COPY = Y
   
' Loop for dynamic forecasting
  for !i = 0 to 11
     smpl @first 2018Q1+!i
       new_dyn_mod.ml
       smpl 2018Q1+!i 2020Q4
       new_dyn_mod.forecast @signal *F2
       series Y = YF2
  next
 
' Restore original series and tidy up
   smpl @all
   series YF2 = Y
   series Y = Y_COPY
   delete Y_COPY

' ********************************************************************************************************************
' State-space model - AR(1) version 2
' ********************************************************************************************************************

' Create the sspace object
  sspace dyn_mod2

' Measurement equation
  dyn_mod2.append @signal Y =  Y_STAR + eps
  dyn_mod2.append @ename eps
  dyn_mod2.append @evar var(eps) = 0

' State equation
  dyn_mod2.append @state Y_STAR = c(1) + c(2) * Y_STAR(-1) + eps_sv1
  dyn_mod2.append @ename eps_sv1
  dyn_mod2.append @evar var(eps_sv1) = exp(c(3))
   
' Estimate model by maximum likelihood
  smpl 1980Q4 2017Q4
  dyn_mod2.ml

' Out-of-sample forecast
  smpl 2018Q1 2020Q4
  dyn_mod2.forecast @signal *F3
 
' ********************************************************************************************************************
' Compare Forecasts
' ********************************************************************************************************************

  show Y YF1 YF2 YF3
   
' -------------------------------------------------------------------------------------------------------------------
'   END OF PROGRAM
' -------------------------------------------------------------------------------------------------------------------


[jason_ll]

Thanks again. However, this is a univariate case.
My problem in the first post was that I wanted to model a signal variable using its own lag as well as a contemporaneous state variable.

So back to my example:

Code: Select all

@signal var = c(1) + c(2)*var(-1) + c(3)*stat +e1
@state stat = c(4)*stat(-1) +e2

Now, sure I can redefine var as a state variable as follows:

Code: Select all

@signal var = state_var
@state_var = c(1) +c(2)*state_var(-1) +c(3)*stat +e1
@state stat = c(4)*stat(-1) +e2

However, in that case I get an error because I can't use a contemporaneous state variable in the equation of another state variable.
Even though I could use "stat(-1)" in this particular case without a problem, this is indeed problematic in the case of a bigger model where "stat" shows up in other signal equations.

Perhaps this would work... although it's very far from elegant:

Code: Select all

@signal var = state_var
@state_var = c(1) +c(2)*state_var(-1) +c(3)*(c(4)*stat(-1) +e2) +e1
@state stat = c(4)*stat(-1) +e2

If you have a more elegant solution for that, I'd be grateful. Either way, I will take your advice and look at the textbook.