Regarding Assumptions

[jiya]

Hi everybody,..plz help me to fix the following problem..........

What are the differences between the assumptions required to run Ordinary least square (OLS) and Maximum Liklihood Estimation (MLE) methods? Are they same?

[mectricsdonk]

OLS basically minimizes the squared residuals of the dataset, hence the name least squares.
you minimize them with regard to the ols estimator b, so you take the derivative, equal it to zero then solve for b.
for OLS to be best linear unbiased estimator, the Gauss Markov conditions must be valid:
1)expectation of disturbances = 0
2)disturbances and regressors are independent
3)variance of the disturbance is constant (homoskedasticity)
4)covariance of the disturbances = 0 (no serial correlation)
it can be shown that under weaker conditions ols is still consistent, but I refer you to the many many beautiful econometric books out there, like modern econometrics by verbeek 2008 or econometric methods with applications in business heij et al. 2004.

with maximum likelihood you assume the data set follows a probability distribution and then estimate the parameters using maximum likelihood method,
where you try to find the maximum of a function by setting the score (first derivative of the likelihood function) to zero. Notice that we don't have to calculate the second derivative
if the assumed distribution is normal, t, etc. because these functions are concave so we know it's a maximum.
assumptions needed for ML are weaker than for OLS, which is why ML is used often in estimation. It can be shown that even if you assume the wrong distribution to the data
that the estimators are still consistent, then you will be using quasi-maximum likelihood. The only condition for QML is that the first two moments are correct.

hope this helped a bit

hope this helps

[jiya]

Thank you.....but can we still use MLE if one or more of the THE GAUSS MARKOV CONDITIONS are violated? For instance, can we use MLE if DISTURBANCES AND REGRESSORS ARE correlated?

[mectricsdonk]

I wrote it down for you:



Uploaded with ImageShack.us

so if E(xe) isn't zero then the ML is biased, so there x's should be exogenous and uncorrelated with the disturbances.

[jiya]

Thank you very very much….I completely agree with you…..but i m bit confused with “partial lag model” (A model where some x’s r not exogenous and hence correlated with the error term). They say that this kind of model can be estimated by 2SLS or MLE. The use of 2SLS is quite understandable…but how can they suggest MLE?

[mectricsdonk]

I'm not exactly sure what you mean, could you be more specific? do you mean distributed lag models?
You could say (imo) that LS is a special case of the MLE, as you saw above

[jiya]

I m extremely sorry for that typographical error……….. It is “spatial lag model”. They say, the spatial lag is a stochastic regressor always correlated with ε through the spatial multiplier, which makes OLS estimates biased and imposes the use of more suitable Maximum Likelihood estimators.

[mectricsdonk]

Hmmmm I'm not sure, I've to think about that.. but interesting question:)

[jiya]

is it?....please do think on this issue.....i am waiting for your reply.....bye d way do they actually mean Limited Information Maximum Likelihood Estimator??

[mectricsdonk]

Hi! I think something like, if the lagged dependent variables are correlated with the disturbances and you want to use GMM then you will have to find a suitable instrument,
which can be quite challenging. Easier is to use QML which gives consistent estimators under the (weaker) assumption that the first two moments are correct.

and LIML heard of it, but never read about it, think has something to do with TSLS

good luck! :eviews6:

[jiya]

thank you very very much....may i get your email id.....would love to communicate with you in future too.

[startz]

OLS is a specific estimator. MLE is a method for deriving estimators. In the usual linear regression model with iid normal errors, the two are the same.

If the right hand side variables are correlated with the errors, then neither OLS nor the equivalent ML estimator is unbiased or consistent. In some cases, there is a MLE that is consistent...but it isn't the usual one.

For example, Limited Information Maximum Likelihood is an MLE that uses instruments to untangle correlation between RHS variables and errors. LIML is similar to 2SLS.

As another example, if there is a lagged dependent variable with serial correlation then OLS is biased. In EViews, if you include an AR(1) term then EViews uses a maximum likelihood estimator that is consistent.

[jiya]

fine.....but how would you justify use of MLE technique in "spatial lag model" ??

[startz]

You'll probably need to write down specifically the model you wish to estimate, including the statistical assumptions about the error terms and how they are related to the RHS variables.

[jiya]

..well..... find that in the attachment here