Page 1 of 1

Regression assumptions

Posted: Tue Mar 05, 2013 7:42 am
by martingale
Hello to everybody

Berry (1993) states a number of assumptions to be satisfied in regression analysis(underneath). My concern is assumption 8: the residuals are normally distributed. In many cases they are not.

My questions are:
A) What is the remedy to non-normal distribution of the residuals in EViews?
B) If a remedy is not necessary, what is the theoretical justification?


Regression assumptions:
1.“All independent variables (X1, X2… Xk) are quantitative or dichotomous and the dependent variable, Y, is quantitative, continuous, and unbounded. Moreover, all variables are measured without error.”
2. “All independent variables have nonzero variance (i.e., each independent variable has some variation in value.”
3. “There is not perfect multicollinearity (i.e., there is no exact linear relationship between two or more of the independent variables)”.
4. “At each set of values for the k independent variables, (X1j, X2j... Xkj), E (ɛj|X1j, X2j… Xkj) = 0 (i.e. the mean value of the error term is zero).”
5. “For each Xi, COV(Xij, ɛj) = 0 (i.e., each independent variable is uncorrelated with the error term).”
6. “For each set of values for the k independent variables, (X1j, X2j… Xkj), VAR (ɛj|X1j, X2j… Xkj) = σ2, where σ2 is a constant (i.e., the conditional variance of the error term is constant); this is known as the assumption of homoscedasticity.”
7. "For any two observations, (X1j, X2j,…, Xkj) and (X1h, X2h,…, Xkh), COV(ɛj, ɛh) = 0 (i.e., error terms for different observations are uncorrelated); this assumption is known as lack of autocorrelation.”
8. “At each set of values for the k independent variables, ɛj is normally distributed.”

Re: Regression assumptions

Posted: Tue Mar 05, 2013 7:47 am
by startz
Normality of the errors is required for the coefficients to have normal distributions in small samples. Without this assumption one has to rely on asymptotic properties. Unless the sample is small or the errors are extremely non-normal, the assumption isn't very important.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 7:52 am
by martingale
Thanks a lot for prompt ansewr! I need some explaining:

C) What is considered to be a small sample? Can it be quantified?
D) What is considered to be extremely non-normal? Can it be quantified?

And I still need a justification on why it is not important unless C or D come into place.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 8:13 am
by startz
There's no generic answer, but a Monte Carlo can be used to see what happens in any particular case.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:06 am
by martingale
Is sounds much easier to apply a remedy. Does EView has one?

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:24 am
by startz
There is no generic remedy (nothing to do with EViews), and as a practical matter nonnormality is rarely a problem.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:31 am
by martingale
OK. But in order to justify this assertion to other people we need to elaborate on theoretical grounds, right? Any suggestions?

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:33 am
by startz
It's just (some version of) the central limit theorem.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:45 am
by martingale
I'd appreciate a reference. Need it for a master thesis. Thanx a lot for your ansewrs so far.

Re: Regression assumptions

Posted: Tue Mar 05, 2013 9:52 am
by EViews Gareth

Re: Regression assumptions

Posted: Tue Mar 05, 2013 10:06 am
by martingale
that's rather useless

Re: Regression assumptions

Posted: Thu Mar 07, 2013 2:39 pm
by martingale
A violation of the normality assumption of the residuals is not as serious as heteroscedasticity and autocorrelation. A moderate departure from normality does not impair the conclusion when the data set is large (Bhattacharyya and Johnson 1997 p. 359). Greene (2012 pp. 64-67) states that a normal distribution of the error term is not necessary for establishing results that allow statistical inference. These results are based on the law of large numbers which concerns consistency and the central limit theorem which concerns the asymptotic distribution of the estimator.