Comparison of Logit Models

[Georgin]

Hello, everyone!
Please, help me to compare (interpret) the results of two logit models:
I am adding more variables to the second model and I need to check whether it performs better than the original one.
The only thing I can notice is rising probability for some variables (which is bad) and rising pseudo R^2 (which is good although as I know it does not put any penalty on adding more variables like R^2 adjusted in OLS). I have no idea how to interpret, say, Log likelihood or Hannan-Quinn criterion and the Eviews help file does not say whether "the higher the better" or "the lower the better" and why.

please, check the attached file. the 2 models to contrast are named "roa" and "rfr"


(P. S. This estimation was firstly discussed at: http://forums.eviews.com/viewtopic.php?f=4&t=661
The suggestion that the original model is overfitted appears to be correct, I have managed to justify the use of maximum 6 variables to solve this issue)

[trubador]

As a general rule, given that dependent variables have the same unit of measurement, the model with the smallest information criterion is selected. Because (most) information criteria are defined as -2 times the average log likelihood plus a penalty function, which this second term is the major source of difference among them.

In your case, two out of three information criteria points to RFR model. Treating ROA as the restricted model (6 vs 3 parameters excluding constant), you can also calculate LR statistic along with its significance as follows:

Code: Select all

scalar LR_d = 2*(rfr.@logl-roa.@logl)

and

Code: Select all

scalar p_d = @chisq(LR_d,6-3)

. This statistic suggest that the RFR is a better model than ROA with %96.5 confidence. Finally, you can evaluate the forecasting performance via Forecast option in the equation dialog box. You'll see that all error measures (RMSE, MAE, MAPE and Theil) are in favor of the RFR model.

I believe these comparisons are more than enough to point you in the right direction...

[Georgin]

trubador, thank You for the excellent support!!!this is exactly what I need. Since the 2nd model is better that proves my research and data collection behind all these have not been in vain.

One more thing, is it that lower RMSE, MAE, MAPE and Theil are better? Since the RFR model has all these statistics lower than the ones of the ROA model, so I can conclude the lower the better.

[trubador]

Correct, lower error measures indicate better forecasting performance.

[Georgin]

trubador, can You please clarify one more moment.
After I had entered the commands that You had posted, I obtained two files lr_d and p_d marked '#' which do not open but show the values 8.58 and 0.035 when I highlight them, correspondingly. Could You please provide some more details on the interpretation and finding the critical chi-value for the case.

[trubador]

After I had entered the commands that You had posted, I obtained two files lr_d and p_d marked '#' which do not open but show the values 8.58 and 0.035 when I highlight them, correspondingly.

This is because I preferred to store the results as scalar. If you find it more convenient, you can also assign the values into a vector:

Code: Select all

vector(2) LR LR(1) = 2*(rfr.@logl-roa.@logl) LR(2) = @chisq(LR(1),6-3)

Could You please provide some more details on the interpretation and finding the critical chi-value for the case.

LR statistic is assumed to follow a chi-square distribution with the number of restrictions being the degrees of freedom. The p-value of 0.035 associated with the LR statistic means we can reject (with 96.5% confidence) the null hypothesis, which simply asserts that there is no significant difference between the unrestricted and the restricted model. In other words, the additional regressors do have a statistically meaningful explanatory power.

[Georgin]

trubador, thank You for the invaluable help!