When I implement the unbalanced random effect model myself (Baltagi/Chang (1994) extension of Swamy/Arora (1979)), I get the same estimates as EViews. I can replicate the coefficient estimates for the hedonic housing prices data set used in Baltagi/Chang (1994), table 2, column SA.
EViews seems to match the standard errors of Baltagi/Chang (1994) where they are noted to be "approximate" and only two decimals are given.
However, I was not able to replicate EViews standard errors. When I take the "usual way", my standard errors for the model on the hedonic housing prices data set are a little off; more precisely they are proportionate by factor 0.96308.
E.g. for variable ZN I get a standard error of 0.650012 while EViews gives 0.626015 and Baltagi/Chang (1994) state 0.63.
As it is not possible to tick or untick the tickbox for a degrees of freedom correction when requesting the oridinary standard errors, I assume EViews takes the "usual way" to calculate the variancecovariance matrix, but it seems to apply some scaling by a factor of 0.96308^2 which we do not find in the paper.
I work with EViews 9.5 student version, latest patches applied.
So, my question is: What does EViews calculate and where does this factor come from? How come that Eviews' numbers match Baltagi/Chang (1994) who do not mention a scaling of the variancecovariance matrix?
oneway unbalanced random effect model (Swamy/Arora)
Moderators: EViews Gareth, EViews Moderator

 EViews Developer
 Posts: 2590
 Joined: Wed Oct 15, 2008 9:17 am
Re: oneway unbalanced random effect model (Swamy/Arora)
I'm not sure what you mean by the "usual way" in this context. Can you be more specific about the calculation that you are doing?
Re: oneway unbalanced random effect model (Swamy/Arora)
By calculation of the variancecovariance matrix in the "usual way" I meant calculating it like in OLS but on the quasidemeaned matrix (Z):
s^2 * (Z'Z)^(1).
I get the same "S.E. of regression" as EViews, but the variancecovariance matrix is different.
s^2 * (Z'Z)^(1).
I get the same "S.E. of regression" as EViews, but the variancecovariance matrix is different.

 EViews Developer
 Posts: 2590
 Joined: Wed Oct 15, 2008 9:17 am
Re: oneway unbalanced random effect model (Swamy/Arora)
Not the correct estimator. The standard GLS estimator is
var(b) = (X' Omega^1 X)^1
So for the quasidemeaned data, you'll have
var(b) = (Z'Z)^1
var(b) = (X' Omega^1 X)^1
So for the quasidemeaned data, you'll have
var(b) = (Z'Z)^1
Re: oneway unbalanced random effect model (Swamy/Arora)
I don't think we can simply drop the sigma^2 (s^2) for the estimator of the variance of the regression coefficients, see e.g. Hsiao (2014), formula 3.3.15 (p. 43).
The values without sigma^2 are way off to what is given by EViews for the standard errors. Btw: I use s^2 = 0.135242^2 for the example data.
The values without sigma^2 are way off to what is given by EViews for the standard errors. Btw: I use s^2 = 0.135242^2 for the example data.

 EViews Developer
 Posts: 2590
 Joined: Wed Oct 15, 2008 9:17 am
Re: oneway unbalanced random effect model (Swamy/Arora)
The overall scaling is built into our GLS transform.
I don't have my books handy, but it strikes me that one reason for GLS coefficient covariance differences arises because there are multiple consistent estimators.
One approach uses the (X' Omega^1 X)^1. This is a consistent estimator and probably the most commonly employed approach.
Another consistent estimator uses both a first and second stage estimator for Omega, as in
(X' Omega1^1 X)^1 (X' Omega^1 Omega2 Omega^1 X) (X' Omega1^1 X)^1
where Omega1 is the estimator obtained from the preGLS estimates, and the Omega2 are obtained using the GLS transform components and the GLS estimator residual scaling.
I haven't looked at Hsiao in ages but my guess is that the two of you are computing the latter while we are computing the former.
I don't have my books handy, but it strikes me that one reason for GLS coefficient covariance differences arises because there are multiple consistent estimators.
One approach uses the (X' Omega^1 X)^1. This is a consistent estimator and probably the most commonly employed approach.
Another consistent estimator uses both a first and second stage estimator for Omega, as in
(X' Omega1^1 X)^1 (X' Omega^1 Omega2 Omega^1 X) (X' Omega1^1 X)^1
where Omega1 is the estimator obtained from the preGLS estimates, and the Omega2 are obtained using the GLS transform components and the GLS estimator residual scaling.
I haven't looked at Hsiao in ages but my guess is that the two of you are computing the latter while we are computing the former.
Re: oneway unbalanced random effect model (Swamy/Arora)
Thank you for your answer, Glenn.
Still, I was not able to reproduce EViews' standard errors. My results agree with, e.g., the implementation in gretl (for the unbalanced option as per Baltagi/Chang as in gretl version 2017d and the R package plm version 1.66) which claims to use the same approach, the common way as in OLS with s^2 * (Z'Z)^(1).
Still, I was not able to reproduce EViews' standard errors. My results agree with, e.g., the implementation in gretl (for the unbalanced option as per Baltagi/Chang as in gretl version 2017d and the R package plm version 1.66) which claims to use the same approach, the common way as in OLS with s^2 * (Z'Z)^(1).
Re: oneway unbalanced random effect model (Swamy/Arora)
This was taken to a wider audience on cross validated:
https://stats.stackexchange.com/questio ... armyarora
https://stats.stackexchange.com/questio ... armyarora

 EViews Developer
 Posts: 2590
 Joined: Wed Oct 15, 2008 9:17 am
Re: oneway unbalanced random effect model (Swamy/Arora)
(My apologies. I am often too busy to get back here very often and missed your followup posting. I had thought that I had answered the question and I just saw the updated thread today. Please feel free to followup on the discussion with the wider audience at cross validated.)

That said, I thought that I was clear in my earlier responses on October 23 which said that EViews does not use the "common way" (your term) OLS standard error calculation on the random effects transformed data but instead uses the GLS estimator of the standard errors. As I indicated, the differences in the standard error estimators were due to to this different choice of individual variance estimate and that by rescaling using the ratio of the standard error estimates you would obtain the desired matching results.
In practical terms, note that the fixed effects residual standard error estimate is about 0.1302487477370731 while the transformed data standard error estimate that you are using is 0.1352415529193776. If you take the ratio of the former to the latter you'll find that it is roughly equal to 0.9630823140186739, which as you note is exactly the constant scaling factor difference between the two estimates. This equivalence is not an accident.

In full detail.
I just grabbed my old 1986 Hsiao off of the shelf. As I suspected (and speculated in my earlier responses), Hsiao pulls the individual error variance out of the GLS transform when solving the normal equations, which means that his moment matrix doesn't have the appropriate scale. As a result his estimator of the coefficient standard errors is not based on the standard GLS estimator but instead uses the pseudoweighted least squares residual variance to obtain properly scaled standard errors.
Let's be more specific. Because he is focusing on the moment conditions, Hsiao employs a transformation that isn't the actual GLS transform, V^{1/2}, but rather a pseudoGLS transformation S^{1/2} = sigma_v * V^{1/2} that removes the individual scale. The Hsiao form of the RE estimator is (using the Z = S^{1/2} X notation we used in our earlier discussion):
b_pgls = (Z'Z)^{1} Z'y
= (X' S^{1/2} S^{1/2} X)^{1} X' S^{1/2} S^{1/2} y
= (X' V^{1/2} sigma_v^2 V^{1/2} X)^{1} X' V^{1/2} sigma_v^2 V^{1/2} Y
= (X' V^{1/2} V^{1/2} X)^{1} X' V^{1/2} V^{1/2} Y
= (X' V^{1} X)^{1} X' V^{1} Y
= b_gls
Note that for estimation of the coefficients, the use of the pseudoGLS estimator yields estimates that are the same as the standard GLS estimator b_gps as we can simply include or exclude the common sigma_v from both terms since it factors out and cancels.
Now let's consider the b_pgls estimator of the variance. The "common way" OLS method you use is given by Hsiao as:
sigma_v^2 * ( Z'Z)^1 = sigma_v^2 * (X' S^{1/2} S^{1/2} X)^1
= sigma_v^2 * (X' V^{1/2} sigma_v^2 V^{1/2} X)^{1}
which, if we assume that the two sigma_v's in the second line are identical, simplifies to the standard GLS estimator
= (X' V^{1/2) V^{1/2} X)^{1}
= (X' V^{1} X)^{1}
Replacing sigma_v^2 in the first line with the variance from the pseudoGLS transformed data (let's call this sigma_v(2)^2) yields your estimator. This is a perfectly reasonable estimator that follows from the pseudoGLS transformation of the data.
I reiterate, however, that this is not the calculation that EViews is performing. For estimation, EViews uses the full GLS estimator:
b_gls = (X' V(1)^{1/2} V(1)^{1/2} X)^1 X' V(1)^{1/2} V(1)^{1/2} Y.
where I am using V(1) to indicate that there is a firststage estimator of the individual idiosyncratic variance embedded in this GLS transformation.Then the variance estimator that we use is the standard GLS estimator
(X' V(1)^{1} X)^{1} = (X' V(1)^{1/2} V(1)^{1/2} X)^{1}
= (Z*'Z*)^{1}
but where, as I wrote earlier, for our Z* transform "(t)he overall scaling is built into our GLS transform".
Substituting in the definition of the pseudoGLS transform form, we have
= (X' S^{1/2} 1/sigma_v(1)^2 S^{1/2} X)^{1}
= sigma_v(1)^2 (X' S^{1/2} S^{1/2} X)^{1}
= sigma_v(1)^2 (Z'Z)^{1}
where sigma_v(1)^2 is the variance estimate embedded in the GLS transformation.
The bottom line is that the difference between the two relevant estimators of the coefficient standard error scomes down to whether you wish to use the initial GLS estimator of the individual standard error, sigma_v(1), or the pseudoGLS estimator of the individual standard error, sigma_v(2). The GLS estimate of the individual variance is obtained from the fixed effects model while the pseudoGLS estimate comes from the weighted regression. EViews does the former, you are doing the latter. Both are consistent, but as you have seen can differ in finite sample.
One final note. EViews follows Baltagi's formulation for the random effects estimator, which is why we use the full GLS transformation, and which is why the EViews standard errors are computed as simple GLS standard errors. In our view (and this is not meant pejoratively), the Hsiao formulation relies on a calculational trick that applies only in a narrow range of cases where there is a variance scale that can be pulled out of the GLS transformation [Edit: though as noted below, this trick does have a nice property]. That approach is not wrong, but the EViews GLS calculation was consciously chosen so that conceptually, EViews is performing the same form of GLS variance calculation for RE as for estimation in panel models with various GLS crosssection or period weights.
Parenthetically, as noted in the discussion above, a third method that could be employed is
While I rather prefer this approach, it is not generally employed in GLS models.
[Edit: Though to be fair, in this setting this latter approach yields the Hsiao estimator.]

That said, I thought that I was clear in my earlier responses on October 23 which said that EViews does not use the "common way" (your term) OLS standard error calculation on the random effects transformed data but instead uses the GLS estimator of the standard errors. As I indicated, the differences in the standard error estimators were due to to this different choice of individual variance estimate and that by rescaling using the ratio of the standard error estimates you would obtain the desired matching results.
In practical terms, note that the fixed effects residual standard error estimate is about 0.1302487477370731 while the transformed data standard error estimate that you are using is 0.1352415529193776. If you take the ratio of the former to the latter you'll find that it is roughly equal to 0.9630823140186739, which as you note is exactly the constant scaling factor difference between the two estimates. This equivalence is not an accident.

In full detail.
I just grabbed my old 1986 Hsiao off of the shelf. As I suspected (and speculated in my earlier responses), Hsiao pulls the individual error variance out of the GLS transform when solving the normal equations, which means that his moment matrix doesn't have the appropriate scale. As a result his estimator of the coefficient standard errors is not based on the standard GLS estimator but instead uses the pseudoweighted least squares residual variance to obtain properly scaled standard errors.
Let's be more specific. Because he is focusing on the moment conditions, Hsiao employs a transformation that isn't the actual GLS transform, V^{1/2}, but rather a pseudoGLS transformation S^{1/2} = sigma_v * V^{1/2} that removes the individual scale. The Hsiao form of the RE estimator is (using the Z = S^{1/2} X notation we used in our earlier discussion):
b_pgls = (Z'Z)^{1} Z'y
= (X' S^{1/2} S^{1/2} X)^{1} X' S^{1/2} S^{1/2} y
= (X' V^{1/2} sigma_v^2 V^{1/2} X)^{1} X' V^{1/2} sigma_v^2 V^{1/2} Y
= (X' V^{1/2} V^{1/2} X)^{1} X' V^{1/2} V^{1/2} Y
= (X' V^{1} X)^{1} X' V^{1} Y
= b_gls
Note that for estimation of the coefficients, the use of the pseudoGLS estimator yields estimates that are the same as the standard GLS estimator b_gps as we can simply include or exclude the common sigma_v from both terms since it factors out and cancels.
Now let's consider the b_pgls estimator of the variance. The "common way" OLS method you use is given by Hsiao as:
sigma_v^2 * ( Z'Z)^1 = sigma_v^2 * (X' S^{1/2} S^{1/2} X)^1
= sigma_v^2 * (X' V^{1/2} sigma_v^2 V^{1/2} X)^{1}
which, if we assume that the two sigma_v's in the second line are identical, simplifies to the standard GLS estimator
= (X' V^{1/2) V^{1/2} X)^{1}
= (X' V^{1} X)^{1}
Replacing sigma_v^2 in the first line with the variance from the pseudoGLS transformed data (let's call this sigma_v(2)^2) yields your estimator. This is a perfectly reasonable estimator that follows from the pseudoGLS transformation of the data.
I reiterate, however, that this is not the calculation that EViews is performing. For estimation, EViews uses the full GLS estimator:
b_gls = (X' V(1)^{1/2} V(1)^{1/2} X)^1 X' V(1)^{1/2} V(1)^{1/2} Y.
where I am using V(1) to indicate that there is a firststage estimator of the individual idiosyncratic variance embedded in this GLS transformation.Then the variance estimator that we use is the standard GLS estimator
(X' V(1)^{1} X)^{1} = (X' V(1)^{1/2} V(1)^{1/2} X)^{1}
= (Z*'Z*)^{1}
but where, as I wrote earlier, for our Z* transform "(t)he overall scaling is built into our GLS transform".
Substituting in the definition of the pseudoGLS transform form, we have
= (X' S^{1/2} 1/sigma_v(1)^2 S^{1/2} X)^{1}
= sigma_v(1)^2 (X' S^{1/2} S^{1/2} X)^{1}
= sigma_v(1)^2 (Z'Z)^{1}
where sigma_v(1)^2 is the variance estimate embedded in the GLS transformation.
The bottom line is that the difference between the two relevant estimators of the coefficient standard error scomes down to whether you wish to use the initial GLS estimator of the individual standard error, sigma_v(1), or the pseudoGLS estimator of the individual standard error, sigma_v(2). The GLS estimate of the individual variance is obtained from the fixed effects model while the pseudoGLS estimate comes from the weighted regression. EViews does the former, you are doing the latter. Both are consistent, but as you have seen can differ in finite sample.
One final note. EViews follows Baltagi's formulation for the random effects estimator, which is why we use the full GLS transformation, and which is why the EViews standard errors are computed as simple GLS standard errors. In our view (and this is not meant pejoratively), the Hsiao formulation relies on a calculational trick that applies only in a narrow range of cases where there is a variance scale that can be pulled out of the GLS transformation [Edit: though as noted below, this trick does have a nice property]. That approach is not wrong, but the EViews GLS calculation was consciously chosen so that conceptually, EViews is performing the same form of GLS variance calculation for RE as for estimation in panel models with various GLS crosssection or period weights.
Parenthetically, as noted in the discussion above, a third method that could be employed is
Another consistent estimator uses both a first and second stage estimator for Omega, as in
(X' Omega1^1 X)^1 (X' Omega^1 Omega2 Omega^1 X) (X' Omega1^1 X)^1
where Omega1 is the estimator obtained from the preGLS estimates, and the Omega2 are obtained using the GLS transform components and the GLS estimator residual scaling.
While I rather prefer this approach, it is not generally employed in GLS models.
[Edit: Though to be fair, in this setting this latter approach yields the Hsiao estimator.]
Who is online
Users browsing this forum: Bing [Bot] and 7 guests