## ROC curve

For technical questions regarding estimation of single equations, systems, VARs, Factor analysis and State Space Models in EViews. General econometric questions and advice should go in the Econometric Discussions forum.

Moderators: EViews Gareth, EViews Moderator

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### ROC curve

Hello,

I'd like to know how a ROC curve can be performed under Eviews7 ?

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 12420
Joined: Tue Sep 16, 2008 5:38 pm

### Re: ROC curve

Never heard of them. Perhaps you could describe what they are.

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

For a logit model with Defaulted firm (1) or not (0) as dependent variable :

ROC curve illustrates the performance of a binary classifier system as its discrimination treshold is varied. It plots the fraction of true positive out of positives against the fraction of false positives out of the negatives. The curve thus shows the ability for the model to classify failing firms in the failing group in comparison with firms forecasted to fail but who have not. The null model has a diagonal ROC Curve, corresponding to an AUC of 0.5.

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

Anyone?

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

Up

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

### Re: ROC curve

Sorry missed this one.

It's not built-in to EViews right now. You could create one with a bit of work, either by looping over the existing fit table routine with different input values and grabbing the results from there, or, as I'd probably recommend, just doing the computations directly and plotting the result. Have you written EViews programs before?

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

No never.. I guess it's gonna be a bit tough for me to compute it then? Or can you help me?

Thanks

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

### Re: ROC curve

I did a quick bit of coding for you.The following code produces a ROC curve from a binary equation. It's not really production quality code, but should hold you over for now. Open the workfile containing the equation of interest. Open a program window by selecting File/New/Program. Paste the code below into the program window

Code: Select all

`%dep = "grade"%eq = "eq_probit"' find valid observations{%eq}.makeresid _xxe' get number of 0 and 1smpl if _xxe<>na and {%dep} = 0!n0 = @obs({%dep})smpl if _xxe<>na and {%dep} = 1!n1 = @obs({%dep})' get fittted valuessmpl if _xxe<>na{%eq}.forecast _xxyfgroup a {%dep} _xxyfstom(a, b)stom(_xxyf, _xxprob)' sort in descending order so we have threshold of infinity which is "off"vector  _xxpranks = @ranks(_xxprob, "d", "i") matrix _xxsdata = @capplyranks(b, _xxpranks)' get unique fitted valuesvector _xxuniq = @uniquevals(pranks)!nunique = @rows(_xxuniq)delete _xxuniq' create output ROC matrixmatrix(!nunique+2, 4) _xxroc' we start at threshold of infinity which is "off"!ntype1 = 0 ' number of correct 1 predictions!ntype0 = !n0 ' number of correct 0 predictions!j = 1!i = 1!n = @rows(_xxsdata)roc(1, 1) = 1 - (!ntype0 / !n0) ' 1 - specificityroc(1, 2) = !ntype1 / !n1 ' sensitivityroc(1, 3) = 0roc(1, 4) = 0while !j <= !nunique   !target = _xxsdata(!i, 2)   !val = !target   while !val = !target      !response = sdata(!i, 1)      if (!response = 0) then         !ntype0 = !ntype0 - 1      endif      if (!response = 1) then         !ntype1 = !ntype1 + 1      endif      !i = !i+1      if (!i > !n) then          !val = 0      else         !val = sdata(!i, 2)      endif   wend   _xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)   _xxroc(!j+1, 2) = !ntype1 / !n1   _xxroc(!j+1, 3) = !j / !nunique   _xxroc(!j+1, 4) = !j / !nunique   !j = !j + 1wend_xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)_xxroc(!j+1, 2) = !ntype1 / !n1delete(noerr) roc_graphfreeze(roc_graph) _xxroc.xypairroc_graph.options linepat -legend size(4,4)roc_graph.setelem(1) linewidth(1.15)roc_graph.setelem(2) linepattern(8)%title = "ROC graph for"%title = %title + " " + @upper(%eq)roc_graph.addtext(t, font("Ariel", 12)) %titleroc_graph.addtext(l) "True Positive Rate"roc_graph.addtext(b) "False Positive Rate" show roc_graphdelete _xx*`

I assume in the program that you don't have anything named ROC_GRAPH in the workfile and that you don't have variables that begin with the characters "__XX".

Next edit the program to replace GRADE and EQ_PROBIT with the variable and equation of interest. Click on run.

The results will be in the graph object ROC_GRAPH. If you want the numbers, we'll have to add a line to the program to rename the results matrix before I cleanup the temps.

I've done some quick checks of the results against Fawcett's (2006) paper, but obviously haven't done extensive testing, especially for cases where there are repeats in the classifier series.

Let me know how it works for you.

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

I did what you told me to do, and I get an error message after having clicked on "run". When I want to click on OK with the following window (that asks me to run the program with different choices of runtime erros) I get this : @UNIQUEVALS is an illegal or reserved name in "VECTOR_XXUNIQ = @UNIQUEVQLS(PRANKS)

Moreover, that would be awesome if I could get the figures related to the graph.

Thanks again

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

### Re: ROC curve

Oh yes. Its an EViews 8 function. I'm away from a computer right now but let me see what I can do later.

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

Ok, look forward to hearing from you.
Thanks

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

### Re: ROC curve

Rewritten for EViews 7 and fixed some other things that I introduced when I moved to temp matrices and series.

Code: Select all

`%dep = "grade"%eq = "eq_probit"' find valid observations{%eq}.makeresid _xxe' get number of 0 and 1smpl if _xxe<>na and {%dep} = 0!n0 = @obs({%dep})smpl if _xxe<>na and {%dep} = 1!n1 = @obs({%dep})' get fittted valuessmpl if _xxe<>na{%eq}.forecast _xxyfgroup _xxa {%dep} _xxyfstom(_xxa, _xxb)stom(_xxyf, _xxprob)' sort in descending order so we have threshold of infinity which is "off"vector  _xxpranks = @ranks(_xxprob, "d", "i") matrix _xxsdata = @capplyranks(_xxb, _xxpranks)' get unique fitted values!nunique  = 1for !i=2 to @rows(_xxsdata)if _xxsdata(!i, 2)  < _xxsdata(!i-1, 2) then   !nunique = !nunique+1endifnext' create output ROC matrixmatrix(!nunique+2, 4) _xxroc' we start at threshold of infinity which is "off"!ntype1 = 0 ' number of correct 1 predictions!ntype0 = !n0 ' number of correct 0 predictions!j = 1!i = 1!n = @rows(_xxsdata)_xxroc(1, 1) = 1 - (!ntype0 / !n0) ' 1 - specificity_xxroc(1, 2) = !ntype1 / !n1 ' sensitivity_xxroc(1, 3) = 0_xxroc(1, 4) = 0while !j <= !nunique   !target = _xxsdata(!i, 2)   !val = !target   while !val = !target      !response = _xxsdata(!i, 1)      if (!response = 0) then         !ntype0 = !ntype0 - 1      endif      if (!response = 1) then         !ntype1 = !ntype1 + 1      endif      !i = !i+1      if (!i > !n) then          !val = 0      else         !val = _xxsdata(!i, 2)      endif   wend   _xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)   _xxroc(!j+1, 2) = !ntype1 / !n1   _xxroc(!j+1, 3) = !j / !nunique   _xxroc(!j+1, 4) = !j / !nunique   !j = !j + 1wend_xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)_xxroc(!j+1, 2) = !ntype1 / !n1delete(noerr) roc_graphfreeze(roc_graph) _xxroc.xypairroc_graph.options linepat -legend size(4,4)roc_graph.setelem(1) linewidth(1.15)roc_graph.setelem(2) linepattern(8)%title = "ROC graph for"%title = %title + " " + @upper(%eq)roc_graph.addtext(t, font("Ariel", 12)) %titleroc_graph.addtext(l) "True Positive Rate"roc_graph.addtext(b) "False Positive Rate" show roc_graphdelete _xx*`

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

Wonderful! Thank you very much!
One last thing, is it possible to compute the aera under the curve ? it is supposed to be a number bounded between 0 and 1, to know how accurate the model is for predicting

EViews Glenn
EViews Developer
Posts: 2627
Joined: Wed Oct 15, 2008 9:17 am

### Re: ROC curve

Rather than me writing a program from scracth for this, you could grab the nonparametric version of the AUC from the Mann-Whitney statistic and manipulate the statistic to get the desired value.
1. Produce a forecast from the binary equation for the estimation sample and save it in a series, say Y_F.
2. Open Y_F and select View/Equality Tests by Classification...
3. Put the original variable in the Variable to Classify edit field, and select Median test. Click on OK to compute the Mann-Whitney statistic.
4. Find the value of the Wilcoxon/Mann-Whitney statistic in the table and the number of 0 and 1 values. Make a note of these values.

The following assumes that the ordinary and tie-adjusted values are the same.

Code: Select all

`!n1 = 21!n2 = 11!unorm = 3.531162!mu = !n1*!n2/2!sigma = sqr(!n1*!n2*(!n1+!n2+1)/12)!u = !unorm*!sigma + !mudelete(noerr) rocarea scalar rocarea = !u/(!n1*!n2)`

The program snippet above computes the AUC. Replace !n1, !n2, and !unorm, with the numbers of 0 and 1 values, and the value of the Mann-Whitney statistic. The scalar ROCAREA should give you the value.

As with the earlier code, this hasn't been tested much (almost at all). I did a quick calculation for the BINARY workfile that EViews provides and it seems to work, but use at your own risk.

eddylle
Posts: 23
Joined: Wed Apr 09, 2014 4:28 am

### Re: ROC curve

The answer seems to be consistent with what I expected.

Thank you very much for all your explanations.