Switching Regression Dependent on Coefficient Vector c?

[Ecovi]

Using the switching regression in Eviews 8 I noticed I get overflow errors on estimation when I enter certain (extreme) values in the coefficient vector c. So apparently, Eviews's initial values depend on this vector, even when "Eviews Supplied" is chosen under Start Method". The user manual makes no mention of this. It only mentions: "The EViews supplied methods employ simple least squares coefficient estimates or the specified fraction of those estimates."

I'd like to have a way to use informative non-user-supplied starting values, e.g. as the manual says it does, by doing simple OLS. I guess it actually does perform some least squares, but in any case, the starting estimates also depend on the coefficient vector.

I'm now setting c=0 before each model estimation to work around such cases where the estimation produces invalid results (extreme values of c result in overflow errors and errors regarding the logarithm of negative values). Still I'd like to know the exact method Eviews uses, especially how it depends on c.

[EViews Glenn]

It's a timing issue.

What is going on is that we set up some structures prior to performing estimation, and then, if necessary, fill the coefficients with the least squares results as described in the manual. For extreme values of the coefficients in the C vector, we error in the first step, prior to actually computing the least squares results and initializing values for the Markov evaluation. In this case, the error message is completely spurious, but it does incorrectly stop the estimation.

Let me see if I can adjust the timing of the initialization. For now, your solution should be fine and will use the least squares estimates. You should note that the estimation results that you see are invariant to the actual values that you put in C.

[edit] I've submitted a fix for the spurious error message. It should appear in the next patch of EViews 8. Sorry for the inconvenience.

[Ecovi]

Thanks, so this is actually a bug. (Though I don't quite understand how this is a timing issue. You just have to move some code around I guess.)

When I set c=0 I get the errors much less frequently, but still in some cases they appear.

Anyway, I'm happy it will be fixed.

[EViews Glenn]

You can still get numeric problems due to the estimation of the switching model itself, but those should be unrelated to what is in the coefficient vector when you start if you are using EViews starting values.

[Ecovi]

I'd like to follow up with some issues I'm having with Eviews's switching regression implementation.

1) What is the searchstds option for exactly? Is it the same as the "Random scale" input field of the graphical interface? If so, it doesn't seem to stick in that field if entered via the command line.

2) What happens when I use only one random initial estimate (or startnum=1 via the command line)? It seems pointless doing this, but Eviews accepts it nonetheless. It should accept either zero or more than one in my opinion. This is a minor point, but it may be important for the next point.

3) I'm still having issues with some overflow errors that should not be there in my opinion. Especially when having some ill-defined model specification with respect to the data, in at least 1% of the cases the estimation can trigger an overflow error, somehow caused by an unlucky chosen initial random start. For example, when estimating a complex 8-regime switching variance model repeatedly with different random starts on some white noise with a low sample size and with a low number of iterations for random starts, as follows:

Code: Select all

setmaxerrs 1000 wfcreate u 50 smpl 1 50 series y=nrnd for !i=1 to 250 c=0 equation eqn.switchreg(type=simple, nstates=8, startnum=1, startiter=0, searchnum=0, searchstds=0, heterr) y next scalar errors=@errorcount

The number of errors is especially sensitive to the sample size and the number of initial random starts (startiter), here set to zero. For higher values of startiter the errors almost disappear. Errors are also more frequent for heterogeneous variances across regimes (heterr), as in the example above. I suspect there is a division by zero somewhere in the random initial start code which is not checked. When going with data even less suitable for such a mixture model (like a random walk) the error count becomes even higher.

[EViews Glenn]

I'd like to follow up with some issues I'm having with Eviews's switching regression implementation.
1) What is the searchstds option for exactly? Is it the same as the "Random scale" input field of the graphical interface? If so, it doesn't seem to stick in that field if entered via the command line.

Sorry. Documentation error "searchsds". I'll get that fixed.

2) What happens when I use only one random initial estimate (or startnum=1 via the command line)? It seems pointless doing this, but Eviews accepts it nonetheless. It should accept either zero or more than one in my opinion. This is a minor point, but it may be important for the next point.

You get one random initial estimate. I think that "pointless" is a bit harsh. Suppose, for example, that you wish to specify a user-defined coefficient but randomly permute it. Or even that you simply want to take the basic regression results and do one draw based on them. Neither of these may be my preferred method, as I'd almost always do more draws, but they are valid requests, and I don't feel as though I'm in a position to tell a user that it's a pointless approach.

3) I'm still having issues with some overflow errors that should not be there in my opinion. Especially when having some ill-defined model specification with respect to the data, in at least 1% of the cases the estimation can trigger an overflow error, somehow caused by an unlucky chosen initial random start. For example, when estimating a complex 8-regime switching variance model repeatedly with different random starts on some white noise with a low sample size and with a low number of iterations for random starts, as follows:

I think you are asking a lot for there to be no errors in the specification that you provided. With 8 regimes one is naturally driven to specifying one or more state probabilities to be zero and possibly one state probability to 1. The EViews parameterization of the state probabilities uses a (possibly multinomial) logistic transform that has exponential weights.This naturally leads to the prospect of overflows in the exponentiation step when you are at boundary conditions. It's not in a division by zero, but rather an exponention of a very large number as the numeric algorithm tries to generate zeros. There are also some cases where the exponential weights send probabilities close to zero, at which point the variances of the state errors are not-identified and can and do blow-up. It's very tricky to try to trap these errors without altering the properties of the estimation algorithms in undesirable ways.

The bottom line, I think, is that a switching regression estimation algorithm will not work well in cases where there the DGP doesn't have switching.

[Ecovi]

Thanks for the elaborate reply. At least I understand where it is coming from now. I'm estimating a range of models among which are the switching regression types. I'm running lots of estimations and only the switching regression implementation generates errors, that's why I wanted to solve this. That said, going out of bounds during the optimisation because of some very wrong initial estimate can be checked for.

[EViews Glenn]

I'm not certain that I understand what you mean by "checked for", nor what behavior you would like to see that is not already suppported.

You are estimating a specification that for some starting value choices cannot be estimated. This generates an error which we report and which you have chosen to ignore. Others might not wish to ignore the error.

I've looked a bit closer at the program that you provided and notice a couple of things that are relevant to this discussion. Notably, you have chosen to estimate the equation inside of a loop, where you choose a single random starting value each time through the loop. Thus, each estimation is treated by EViews as a separate estimation procedure and accordingly, if there is an error, it is reported.

As I said earlier, switching models are particularly difficult to estimate as they are highly nonlinear and subject to identification issues. Because of this, we offer useful tools for allowing you to perform estimation from different starting values. For example, had you changed your options to startnum=250, startiter=100, searchsds=2 (or something along those lines) and removed your !i loop, EViews would have estimated the model with 250 randomly chosen sets of starting values, iterating 100 times for each starting value, and then used the best of these results as starting values for the final iterative estimation. Numeric errors encountered during any of the 250 starting value choices would have been ignored.

Applying this approach to your example, I can write your program as

Code: Select all

wfcreate u 50 series y=nrnd equation eqn.switchreg(type=simple, nstates=8, startnum=250, startiter=100, heterr) y

which, in a couple of quick tests, does not generate an error and produces results.

[Ecovi]

I'm not certain that I understand what you mean by "checked for", nor what behavior you would like to see that is not already suppported.

On second thought, I think that considering the algorithm Eviews uses you cannot improve on this any further and 'solve' the error. Like you said, switching models can be difficult to estimate.
Like the bug above, I thought this would also be something that could be solved, but if it occurs it just means you need more random starts.

As I said earlier, switching models are particularly difficult to estimate as they are highly nonlinear and subject to identification issues. Because of this, we offer useful tools for allowing you to perform estimation from different starting values. For example, had you changed your options to startnum=250, startiter=100, searchsds=2 (or something along those lines) and removed your !i loop, EViews would have estimated the model with 250 randomly chosen sets of starting values, iterating 100 times for each starting value, and then used the best of these results as starting values for the final iterative estimation. Numeric errors encountered during any of the 250 starting value choices would have been ignored.

Of course using a higher value for startiter you effectively eliminate the issue. The example I provided was just an extreme case to show the problem. But also in your version of the program---which uses settings that look more like those I use personally---the issue might eventually arise, if you estimate it often enough (with low sample size particularly).

One last thing that would perhaps be an option is a kind of error handling approach in the estimation, where it tries again with another set of starting values if an overflow error occurs. Although, maybe this should be done by the user, outside of the switching regression implementation, were it not for the fact that Eviews doesn't really support error handling programmatically.

[EViews Glenn]

I'm not clear on the latter request. If you simply tell EViews to draw from lots of starting values, then values which generate errors are discarded from the comparison. And if, after all of that, you still get an error in the final estimation, you can programmatically tell EViews to repeat the previous procedure using the various error count functions and routines.

I will agree that with low enough sample sizes, it may be virtually impossible to get EViews to recover from errors. But that's probably a case of the model being unestimable rather than EViews lacking flexibility.

[Ecovi]

I think we agree on all accounts. So without doing manual error handling it is always better to do more initial random starts, and any errors that occur in the initial iterations to determine starting values are caught by Eviews. That is also what I am seeing because if they were not caught, and if I would set startiter and/or startnum to very high, it would actually increase the probability of errors in a single estimation procedure. But what about the number of final estimates (the "No. of random from final" or the searchnum option)? Can those multiple final estimations trigger an error (i.e. on the first occurrence of overflow or other errors estimation fails and generates an error) or are they also handled by Eviews (i.e. it only generates an error if all of the final estimations are unsuccessful)? If these errors are not caught I guess to get the least amount of errors in a single estimation procedure you have to choose high values for startnum and startiter (to get a good less error-prone initial estimate) and choose lower values (or zero) for searchnum (to minimise the chance of getting stuck at the final estimation, with a trade-off that you may get slightly worse estimates, on average)?

[Ecovi]

it only generates an error if all of the final estimations are unsuccessful?

Could you explain my above post a little more? I still don't know if this quoted part--which is the whole point of this--is the case...

[EViews Glenn]

You should not see an error if some of the random starts fail. When you choose to use random starts, we simply discard all of those which generate an error and perform the comparison using those which do not.

As I said in my earlier post

If you simply tell EViews to draw from lots of starting values, then values which generate errors are discarded from the comparison.

[Ecovi]

Thanks a lot, I wasn't entirely sure whether you meant if any errors during initial starting estimates are handled (the amount of which is controlled by startiter) or also the errors during final estimations (controlled by startnum).

A little more information on how the randomization works might be nice. A couple of things is still unclear from the documentation.

1) How exactly are the initial random starts generated? I can specify the random generator and the seed, but how are they computed? Just a random normal added to the parameters? If so, can we control the scale with which it does this, like for the "No. of random from final" option?

2) What is the idea behind having a .3, .5 or .8 times the Eviews supplied starting value? Is it to get a numerically more stable estimate if lower values are tried?

3) How does the "No. of random from final" (startnum) work? The manual now states:

The random starting values are chosen by taking the best estimated values to date and
adding random normals with scale given by the Random scale fraction of the final
coefficient standard deviations. The estimates with the highest likelihood become the
final estimates.

But how many of the best values are used? And how often are they re-used?

[EViews Glenn]

1. random normals with a scale using the initial coefficient variance estimate as with the finals. I'm away from my computer right now, but there should be a scale option [edit: the scale option applies both to the initial and the final randomized values]

2. I don't know if I'd say "stable", but the basic idea is correct.

3. Take the final estimates obtained after doing any initial randomization. Randomly perturb using normals, the coefficient variance estimates, and the scale to find new values. Iterate to convergence and compare with previous results. Pick the best model and use it in place of the original "final" estimate.