Scatter plot with fitted regression line (not based on OLS)

[Kavorka]

In EViews, it is easy to estimate e.g. a median regression between x and y. However, I have not figured out how to make a scatter plot of the x and y observations, where the fitted values from the median regression line is plotted on top of these scattered observations.

I am only interested in a syntax solution (with no use of menus).

Thus in summary, what I want is to make a scatter plot between x and y where I can plot any line (I can figure out how to construct the fitted values).

I would be perfectly happy with a (not so very elegant) programming solution to the problem if there are no canned routines for this problem. I think that this is a relevant problem since scatter plots can often provide more information than many misspecification tests, especially when compared to the estimated regression line.

/Par

[EViews Glenn]

The only way to do what you want to do is to create a new series with the fitted values for the median regression corresponding to the values of the exogeneous variable series, and do a scatterplot changing the symbols for the median plot to line only.

First, to remind you of what you know...

Suppose, for example we have the dependent series Y, and explanatory variable X. You can display the scatter of Y against X by creating the group A

Code: Select all

group a x y

and displaying the scatter for A.

Code: Select all

a.scat

What we want to do is to modify this basic idea to display a new series containing your fitted values.

There are two approaches. The first is to, for each value of the existing X, create the fitted value YHAT from your median fit. Then you can create the group and display the scatterplot for this group

Code: Select all

group b x y yhat b.scat

If you want a line for the median regression fit, however, we need to do more work.

If the values of X are ordered, then all you need to do is to change the display for the second scatter to show a line.

Code: Select all

freeze(mygraph) b.scat mygraph.setelem(2) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(3) legend("Fitted Y")

Note that line/symbol 1 is for the first scatter and line/symbol 2 is for the second scatter so that we are changing the second scatter to a connected line with no symbols. We are also changing the legend assignment. Note that this is kind of strange for scatters. Legend element 1 is for the X axis, legend 2 is for the first scatter, and legend 3 is for the second.

If, however, the X are not ordered in the workfile, then the line is going to connect sequential points which is not what you want. You can, if feasible, solve for this by first sorting the workfile by X, then freezing MYGRAPH and redoing the steps above.

If sorting the workfile isn't practical (e.g., it's dated), then you can make a new series XORDERED and a new series FITORDERED which contain ordered values for which you wish to compute the median fit and the corresponding fit values. Then create the group containing the four series

Code: Select all

group bstar x y xordered fitordered

and display the pairwise scatter for these series

Code: Select all

freeze(mygraph) bstar.scatpair mygraph.addtext(b) "X" mygraph.setelem(1) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(2) legend("") mygraph.setelem(3) legend("Fitted Y") mygraph.setelem(4) legend("")

Notice that I had to fuss a bit with the legend and text to get things to look right. The key here is that you avoid the whole sort the workfile thing by just making a new series with sorted values, computing the fit for those values, and then doing the pairwise plot.

[Kavorka]

Thanks for your answer! However, I haven't figured it out anyway. I have pasted the code for the two various alternatives. As you can see the graphs are not straight lines. Could you please help me to point out what I am doing wrong. I realize that in alternative 1 I cannot use sort(a) two times since this will change the entire work file on both occasions. I have attached a wf1-file too (

temptestdata.WF1

(103.74 KiB) Downloaded 417 times

). I am using EViews 7.1. Thanks for your help!

Code: Select all

'ALTERNATIVE 1 wfopen "C:\XXXXXXXXXX\temptestdata.wf1" EQUATION EQ1.qreg x01 c y01 EQ1.fit yhat sort(a) yhat series fitordered = yhat sort(a) x01 series xordered = x01 group bstar x01 y01 xordered fitordered freeze(mygraph) bstar.scatpair mygraph.addtext(b) "X" mygraph.setelem(1) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(2) legend("") mygraph.setelem(3) legend("Fitted Y") mygraph.setelem(4) legend("") show mygraph 'ALTERNATIVE 2 wfopen "C:\XXXXXXXXXX\temptestdata.wf1" EQUATION EQ2.qreg x01 c y01 EQ2.fit yhat sort(a) x01 group groupB x01 y01 yhat 'groupB.scat freeze(mygraph) groupB.scat mygraph.setelem(2) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(3) legend("Fitted Y") show mygraph

[EViews Glenn]

Is X01 the dependent variable or is Y01 the dependent? As you have it, you have X01 the dependent. Change the first line accordingly (I've simplified things a bit below).

Code: Select all

EQUATION EQ1.qreg y01 c x01 EQ1.fit yhat sort(a) x01 group bstar x01 y01 x01 yhat freeze(mygraph) bstar.scatpair mygraph.addtext(b) "X" mygraph.setelem(1) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(2) legend("") mygraph.setelem(3) legend("Fitted Y") mygraph.setelem(4) legend("") show mygraph

[Kavorka]

Thank you very much for your quick reply, I appreciate it!

However, from what I can see it looks like there are small dents in the regression line even when using the data in the previous example (but I may be wrong). My impression is that it does not completely look like a exactly straight regression line (correct me if I am wrong). First of all, it was only a mistake when I swapped the position of the independent and the dependent variable when I cleaned up my program so that someone else would be able to read it (however, it was only in the version I sent to you that I made this mistake, but anyway sorry for making you confused). I just made a new program so that it would be easier for you to read, but unfortunately I swapped x and y. The program that I actually used did the same thing as the last one you sent.

This time, instead of making a new program that is easier for you to read I send parts of my real program to you so that there is less likelihood that I will make a sloppy mistake.

Here it is (however, there are some parts that I have cut out to make it easier to read, but still there are some variables that are declared which are not used in this truncated program). I use the same data as before. Look at the graphs. I cannot make the regression lines be completely straight.

I very much appreciate your help on this matter!

Best wishes,
P
(I use version 7.2)

temp prg to eviews.prg

(1.49 KiB) Downloaded 438 times

temptestdata.wf1

(103.74 KiB) Downloaded 445 times

[EViews Glenn]

You have more than one right-hand side variable. There is no guarantee that the fitted values from the regression will lie in a line for any single dimension unless there is only a single varying variable...

[Kavorka]

Sorry, I should have clearly pointed out that one of the independent variables is a slope dummy (Dx{%b}(-{!OLSlags}))*(x{%j}(-{!OLSlags})) is just a slope dummy. Sorry for not making this clear! Best wishes! /P

[Kavorka]

Thanks Glenn for your answer, however I still think that the problem is not solved (even if I first thought so). One of the independent variables is a dummy slope dummy variable so it should not be a problem to plot the regression line with a change of the slope in the middle of the graph. This dummy variable was created with the recode command which is included in the submitted program, but I understand that you at a first glance did not see this (therefore, I should have informed you about this).

However, even when choosing a model with only one independent variable (no dummy variable), the regression line does not seem like a straight line. To make this as clear as possible I send over a new program and some fictitious data again. Again what I want is a regression line from a quantile regression (or any method besides OLS with an intercept) that should be plotted in a scatter plot. Thanks for your help (if possible please help me before the weekend)! If you like you can choose the previous example with a knot restriction (slope dummy). You can choose by yourself what suits you best! It is best is you use the dummy case, but I think that this will be a general solution based on just plotting yhat in a scatter plot. I have downloaded the latest version of EViews 7.2.

to eviews.prg

(1.36 KiB) Downloaded 428 times

temptestdata.wf1

(103.74 KiB) Downloaded 435 times

[EViews Glenn]

The problem you are having is that the sorting of the workfile is changing lag structures for you. (Your original example didn't have lags in it which changes things dramatically...)

Let's see why this isn't working...then we can go from there...

You are estimating the equation.

Code: Select all

equation eqtest.qreg y0003M x003M(-1)

and computing a forecast. That's fine. We then sort by x003m(-1). Which is fine for ordering the workfile by the original dependent variable. But then you are plotting x003m(-1) against y003m which is where the problem is

The problem is that x003m(-1) for a given value of y003m isn't the same x003m(-1) that you used to have (when you did the forecast) since you've sorted the workfile. (Note the lag is processed using the current workfile structure not the original). Note that this problem is more serious than you might have thought that since your later estimations won't have the lag data in the right place.

So we have to try a slightly different approach. First, let's estimate and forecast

Code: Select all

equation eqtest.qreg y0003M x003M(-1) eqtest.fit y003mf

Now, let's create a group with the desired values

Code: Select all

group g1 x003m(-1) y003m x003m(-1) y003mf

There's nothing there that is any different that what you've done before except we haven't sorted. Now create a matrix out of the group, extract the ranks of the explanatory variable, and reorder the rows of the matrix

Code: Select all

stom(g1, gmat1) gmat1=@capplyranks(gmat1, @ranks(@columnextract(gmat1, 1), "a", "i")) freeze(mygraph) gmat1.scatpair mygraph.addtext(b) "X" mygraph.setelem(1) legend("Actual Y") mygraph.setelem(2) symbol(none) linepattern(solid) mygraph.setelem(2) legend("") mygraph.setelem(3) legend("Fitted Y") mygraph.setelem(4) legend("")

Only the first two lines are different here. This is a better solution that my tossed-off original since it doesn't sort the workfile and should be safer in a number of contexts.

Since it looks as though you are going to be doing this a lot, I would highly recommend taking a few minutes to put the above code in an add-in. Then you can add the entry to your equation menu and in a single click get the graph...or in a single command produce the graph.

[Kavorka]

This problem was a little bit more complicated than what I first thought due to some unforeseen complications from the type of data and the model it was applied on. Despite this you solved it in a general and an impressive way. Good work!

This can be used for large data sets, and I will also utilize these plots when teaching! Plotting data with regression lines are very underestimated since this is an important complement to detect problems (and often better than just to solely apply one-dimensional robustness tests).
/P