ROC curve

For technical questions regarding estimation of single equations, systems, VARs, Factor analysis and State Space Models in EViews. General econometric questions and advice should go in the Econometric Discussions forum.

Moderators: EViews Gareth, EViews Moderator

Gulgun
Posts: 1
Joined: Fri May 22, 2015 1:06 pm

Re: ROC curve

Postby Gulgun » Fri May 22, 2015 2:50 pm

Hi,

Can we use the same codings also in eviews 8?

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 13307
Joined: Tue Sep 16, 2008 5:38 pm

Re: ROC curve

Postby EViews Gareth » Fri May 22, 2015 2:51 pm

Yes., that code was written for EViews 8.
Follow us on Twitter @IHSEViews

WerMaximus
Posts: 4
Joined: Sat May 23, 2015 10:52 am

Re: ROC curve

Postby WerMaximus » Thu Jun 04, 2015 12:28 am

EViews Glenn wrote: The following assumes that the ordinary and tie-adjusted values are the same.

Code: Select all

!n1 = 21
!n2 = 11
!unorm = 3.531162

!mu = !n1*!n2/2
!sigma = sqr(!n1*!n2*(!n1+!n2+1)/12)
!u = !unorm*!sigma + !mu
delete(noerr) rocarea
scalar rocarea = !u/(!n1*!n2)



Hi Gareth or anyone who might know the answer,

I computed the AUC the way you described. When I run the regression with only one variable, I get different results for ordinary and tie-adjusted Wilcoxon/Mann-Whitney statistic. In those cases, what am I supposed to do? Can I just choose the ordinary one or is the test unusable for that? I am using Eviews 7.1

Best regards and many thanks in advance.

mikela
Posts: 14
Joined: Tue Jan 11, 2011 8:24 am

Re: ROC curve

Postby mikela » Fri Nov 06, 2015 7:37 am

Hi
is it possible to automate step 2. Ie, execute the test for equality by classification in the program and saving the test results rather than writing it down by hand?
I can't find the command to run the test.

Many thanks,
Mikela

EViews Glenn
EViews Developer
Posts: 2671
Joined: Wed Oct 15, 2008 9:17 am

Re: ROC curve

Postby EViews Glenn » Fri Nov 06, 2015 8:06 am


diggetybo
Posts: 152
Joined: Mon Jun 23, 2014 12:04 am

Re: ROC curve

Postby diggetybo » Thu Mar 24, 2016 10:16 am

I tried the above code (both eviews 8 and 7 versions) in Eviews 9 and it threw an error to the effect '65 is not a valid index of _____.'

So it seems to have issued too many elements to the vector or matrix, because my data set only contains 64 observations. I don't know what it's really doing though, just trying to be helpful.

I'm not sure how high this is on the priority list, but having a roc program would be really useful. I'm hoping this is revisited.

I dont know why my post is showing up here, please refer to my target thread:

viewtopic.php?f=4&t=10614

diggetybo
Posts: 152
Joined: Mon Jun 23, 2014 12:04 am

Re: ROC curve

Postby diggetybo » Sat Jul 09, 2016 5:11 am

Months later,

It dawns on me that its actually the same thread, just the latest page. I didn't notice it had multiple pages and I was confused for a while. So disregard the comment on whether the thread was right or not. That was my bad.

Nonetheless, I still have that same error for this ROC curve program. Can someone walk through trouble shooting that with me, or advise me on what else I can consider using given that I'm running Eviews 9?

Thank you

diggetybo
Posts: 152
Joined: Mon Jun 23, 2014 12:04 am

Re: ROC curve

Postby diggetybo » Sat Jul 30, 2016 1:29 am

After more experimenting, I got the program to complete. Although it produced a graph that was not correct (my false positive rate x axis went from 0 to 2.5) I'm not sure why it doesn't span simply 0 to 1.

The issue before was the !i and !j were increasing beyond the rows of my matrix. I simply added an if statement to make sure they were less than the number of rows before proceeding with the !i = !i+1. I then added an else exitloop.

It appears these changes weren't needed for earlier versions of Eviews, but when I tried it with Eviews 9, it would not work. This slightly modified program will at least produce a graph, but there are some more bugs to be worked out. Can someone spot the remaining problems with the code? I have neared my wits end with it, but I'm not giving up. Have a look:

Code: Select all

%dep = "inlf"
%eq = "eq01"

' find valid observations
{%eq}.makeresid _xxe

' get number of 0 and 1
smpl if _xxe<>na and {%dep} = 0
!n0 = @obs({%dep})
smpl if _xxe<>na and {%dep} = 1
!n1 = @obs({%dep})

' get fittted values
smpl if _xxe<>na
{%eq}.forecast _xxyf
group _xxa {%dep} _xxyf
stom(_xxa, _xxb)
stom(_xxyf, _xxprob)
' sort in descending order so we have threshold of infinity which is "off"
vector  _xxpranks = @ranks(_xxprob, "d", "i")
matrix _xxsdata = @capplyranks(_xxb, _xxpranks)

' get unique fitted values
!nunique  = 1
for !i=2 to @rows(_xxsdata)
if _xxsdata(!i, 2)  < _xxsdata(!i-1, 2) then
   !nunique = !nunique+1
endif
next

!nunique=!nunique+2
' create output ROC matrix
matrix(!nunique, 4) _xxroc

' we start at threshold of infinity which is "off"
!ntype1 = 0 ' number of correct 1 predictions
!ntype0 = !n0 ' number of correct 0 predictions

!j = 1
!i = 1
!n = @rows(_xxsdata)
_xxroc(1, 1) = 1 - (!ntype0 / !n0) ' 1 - specificity
_xxroc(1, 2) = !ntype1 / !n1 ' sensitivity
_xxroc(1, 3) = 0
_xxroc(1, 4) = 0
while !j <= !nunique
   !target = _xxsdata(!i, 2)
   !val = !target
   scalar val_1 = !val
   while !val = !target
      !response = _xxsdata(!i, 1)
      if (!response = 0) then
         !ntype0 = !ntype0 - 1
      endif
      if (!response = 1) then
         !ntype1 = !ntype1 + 1
      endif
    if !i < !n then
      !i = !i+1
    else
      exitloop
      endif
   wend
   _xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)
   _xxroc(!j+1, 2) = !ntype1 / !n1
   _xxroc(!j+1, 3) = !j / !nunique
   _xxroc(!j+1, 4) = !j / !nunique
   if !j < !n-1 then
   !j = !j + 1
   else
   exitloop
   endif
wend
_xxroc(!j+1, 1) = 1 - (!ntype0 / !n0)
_xxroc(!j+1, 2) = !ntype1 / !n1

delete(noerr) roc_graph
freeze(roc_graph) _xxroc.xypair
roc_graph.options linepat -legend size(4,4)
roc_graph.setelem(1) linewidth(1.15)
roc_graph.setelem(2) linepattern(8)
%title = "ROC graph for"
%title = %title + " " + @upper(%eq)
roc_graph.addtext(t, font("Ariel", 12)) %title
roc_graph.addtext(l) "True Positive Rate"
roc_graph.addtext(b) "False Positive Rate"
show roc_graph

delete _xx*


tho_mi
Posts: 61
Joined: Sat May 12, 2012 2:41 am

Re: ROC curve

Postby tho_mi » Tue Mar 21, 2017 1:05 pm

This might be a bit late, but I also ran into problems using the code (even though it worked perfectly fine using another sample). After experimenting for about two hours I found out that the problems was the last row of "_XXSDATA". Both entries were zero and in the code there is the while-loop with the condition "!val = !target".

In theory !i should never go beyond the last row of "_XXSDATA", but since "!val" is set to 0 in case !i goes beyond the last row, it stays in the loop and then an error occured. Setting "!val" to -1 in this case solved the problem.

I'm not sure what your problem was. Did you manage to solve it?


Return to “Estimation”

Who is online

Users browsing this forum: Google [Bot] and 30 guests