DMtest - Diebold-Mariano Forecast Evaluation Test

For questions about EViews Add-ins available from the EViews Add-ins webpage. Note each add-in available on our webpage will have its own individual thread.

Moderators: EViews Gareth, EViews Moderator, EViews Esther

tvpvar
Posts: 4
Joined: Thu Sep 04, 2014 7:17 am

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby tvpvar » Fri Sep 05, 2014 2:22 pm

The DM test is supposed to be a model-free test for comparing 2 forecast paths. It would be better if the menu allows the option to select a forecast path instead of an equation. That way, a model-based forecast can be compared to a forecast coming from surveys.

ayca79
Posts: 16
Joined: Thu Mar 05, 2015 4:20 am

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby ayca79 » Fri Apr 17, 2015 4:34 am

Hello, I have a question about the equation of this add in. What kind of equation should it be? I think that it should not be a time series regression equation because DM test calculates whether the difference is significant between two series which are the series of deviation between the realized and estimated values. So the equation must be like

for equation 1
diff1=est_1-realized

and for equation 2
diff2=est_2-realized

equation 1 and equation 2 asked in the window opened must be something like the equations defined above. However, when I created a new equation object in eviews it only creates regression objects. Isn't it? I am not a frequent user of eviews but it this way as I know.

My question is:

I have two series of estimated values and one serie of realized values. I need to know whether two estimated series produced significantly different values. In other saying, I need to know whether two series of estimated values are significantly different? In order to calculate this by DMtest add-in,
What kind of equations should I create and what should I give as a forecast sample?

Thank you for any help,

CharlieEVIEWS
Posts: 202
Joined: Tue Jul 17, 2012 9:47 am

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby CharlieEVIEWS » Wed May 27, 2015 8:55 am

Out of interest, was there a conclusion as to why this add-in was programmed to take in equation objects and not just forecasted series and dependent variables? seems a slightly strange choice to me...


edit: it also seems strange that this version of the addin makes no HAC correction, when doing so is fairly easy surely - to estimate loss differentials on a constant with the NW option.

Minato
Posts: 1
Joined: Fri Jun 05, 2015 12:40 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Minato » Fri Jun 05, 2015 1:42 pm

HELP !!!Emergency !!!

Hi guys, I am writing my thesis now, and I am using EViews 9 DM-test add-in to do the test, I have a questions here. After I run the DM-test add-in, I am asked to write two names of equations. What on earth should I write ?? Let's say, I have the series of original data, the dynamic forecast data using dynamic method and static methods. Can someone tell me ??

Thank you guys in advance!!!

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11369
Joined: Tue Sep 16, 2008 5:38 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby EViews Gareth » Sat Jun 06, 2015 7:56 am

I don't think the author of this add-in is active any more.

I think for those of you who want to perform the test on an arbitrary series, rather than an equation object, you'll have to modify the source code to do that.
Follow us on Twitter @IHSEViews

luiz.murat
Posts: 1
Joined: Thu Sep 17, 2015 1:51 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby luiz.murat » Thu Sep 17, 2015 2:00 pm

Hi all,

Based on what I understood from the add-in, it seems it only works when I am comparing two different equations. Given this is a model free test, instead of using the equations I was wondering if it is possible to use the forecasts directly. If I have 2 series of forecasts + the "realized" data, I am wondering how to run DM test.

Could someone urgently help me. By the way, I dont know how to modify the code to do that.

Thanks for your support.

Luiz

Econoforecast
Posts: 25
Joined: Sun Feb 19, 2017 1:25 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Econoforecast » Mon Mar 20, 2017 4:18 pm

EViews Gareth wrote:I don't think the author of this add-in is active any more.

I think for those of you who want to perform the test on an arbitrary series, rather than an equation object, you'll have to modify the source code to do that.

Hi Gareth, I have managed to download the add in and extract the script that was used to make this add-in. How can I modify it effectively so at perform the test on two series rather than two equation objects. Please find the code below. Thanks.

Code: Select all

'Diebold-Mariano forecast evaluation test (JBES 1995);
'inputs:  two equation objects, and a forecast sample.
'Output is a spool with a results table and graphs of the forecast errors

'step 1:
'forecast from each equation over the sample period.
!selection = 1
%eq1 = ""
%eq2 = ""
%smpl = ""
@uidialog("edit", %eq1, "Enter name of equation 1", "edit", %eq2, "Enter name of equation 2", "edit", %smpl, "Enter a forecast sample", "radio",!selection,"Please select a loss function:", "Squared Absolute")

'error check for object existance and type
if @isobject(%eq1) = 0 then 
   %error = %eq1 + " is not an equation in your workfile"
   @uiprompt(%error)
   stop
   else
      if @neqna({%eq1}.@type,"EQUATION") = 1 then
         %error = %eq1 + " is not an equation in your workfile"
         @uiprompt(%error)
         stop
      endif
endif

'error check for object existance and type
if @isobject(%eq2) = 0 then 
   %error = %eq2+ " is not an equation in your workfile"
   @uiprompt(%error)
   stop
   else
      if @neqna({%eq2}.@type,"EQUATION") = 1 then
         %error = %eq2 + " is not an equation in your workfile"
         @uiprompt(%error)
         stop
      endif
endif

'check to make sure the actual values of the dependent variable exist and the equations have the same dependent variable
%vars1 = {%eq1}.@varlist
%vars2 = {%eq2}.@varlist
%dep1 = @word(%vars1,1)  'get dependent variable
%dep2 = @word(%vars2,1)  'get dependent variable

'save sample
%origsmpl = @pagesmpl

'set sample
smpl {%smpl}

%NDEP = @getnextname("NDEP")
series {%NDEP} = {%dep1}

'checking for the existence of actual values inside the dependent variable over the forecast period
if @obssmpl <> {%NDEP}.@obs then
   %msg = "Actual values of dependent variable must exist over the forecast period"
   @uiprompt(%msg)
   delete {%ndep}
   smpl {%origsmpl}
   stop
endif

delete {%ndep}

If %dep1 <> %dep2 then
%msg = "Equations must have the same dependent variable"
@uiprompt(%msg)
stop
endif

'Checking to make sure only simple samples are being used
if @wcount(%smpl) > 2 then
   %msg = "Only simple forecast samples are supported"
   @uiprompt(%msg)
   stop
endif

'perform the forecasts - creating two series, forecast1 and forecast2
%forecast1 = @getnextname("forecast1")
%forecast2 = @getnextname("forecast2")

{%eq1}.forecast {%forecast1}
{%eq2}.forecast {%forecast2}

'generating forecast errors
%FE1 = @getnextname("FE1")
%FE2 = @getnextname("FE2")

series {%FE1} = {%dep1} - {%forecast1}
series {%FE2} = {%dep2} - {%forecast2}

%Ld = @getnextname("LD")

'Step 2:
'calculate Ld, find mean and varialce
if !selection = 1 then
   series {%Ld} = {%FE1}^2-{%FE2}^2
else
   series {%Ld} = @abs({%FE1}) - @abs({%FE2})
endif
!Ldiffm = @mean({%Ld})
!Ldiffv = @var({%Ld})


'Step 3:
'Calculate covariances

'calculate number of observations in forecast sample
!t = @obssmpl

!DMstat = !Ldiffm / @sqrt((1/!T)*!Ldiffv )

!pval = 2*@cnorm(-@abs(!DMstat))

'd(noerr) a
'table a
'a(1,1) = !dmstat

if !selection = 1 then
   %lossfunc = "Squared"
else
   %lossfunc = "Absolute"
endif

'Results
'Presented in a Table
%tn = @getnextname("DMtest")
Table(9,2) {%tn}
{%tn}.setwidth(1) 18
{%tn}.setwidth(2) 10
{%tn}.setjust(a1:a9) left
!rowcounter=1
{%tn}(!rowcounter,1) = "Diebold-Mariano Forecast Eval. Test"
!rowcounter = !rowcounter + 1
%datestring = "Date: " + @datestr(@now,"MM/DD/YY") + "   Time: " + @datestr(@now,"HH:MM")
{%tn}(!rowcounter,1) = %datestring
!rowcounter=!rowcounter+1
%samplestr = "Equation 1: "  + @upper(%eq1)
{%tn}(!rowcounter,1) = %samplestr
!rowcounter=!rowcounter+1
%samplestr = "Equation 2: " + @upper(%eq2)
{%tn}(!rowcounter,1) = %samplestr
!rowcounter=!rowcounter+1
%samplestr = "Forecast Sample: "  + @upper(%smpl)
{%tn}(!rowcounter,1) = %samplestr
!rowcounter=!rowcounter+1
%samplestr = "Loss function: "  + %lossfunc
{%tn}(!rowcounter,1) = %samplestr
!rowcounter=!rowcounter+1
{%tn}.setlines(!rowcounter) +d
!rowcounter = !rowcounter + 1
'---------------------------
{%tn}(!rowcounter,1) = "DM Test Statistic: "
{%tn}(!rowcounter,2) = !DMstat
!rowcounter = !rowcounter + 1
{%tn}(!rowcounter,1) = "P-value: "
{%tn}(!rowcounter,2) = !pval
!rowcounter = !rowcounter + 1
{%tn}.setlines(!rowcounter) +d
!rowcounter = !rowcounter + 1

'creating a spool with table and graphs
%sp = @getnextname("DMResults")
spool {%sp}
{%sp}.append {%tn}
group g1 {%forecast1} {%forecast2}
{%sp}.append g1.graph
{%sp}.name untitled01 DMtable
{%sp}.name untitled02 ForecastGraphs
show {%sp}

delete {%forecast1} {%forecast2} {%ld} {%tn} g1 {%fe1} {%fe2}
smpl {%origsmpl}
Last edited by Econoforecast on Mon Mar 20, 2017 4:25 pm, edited 1 time in total.

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11369
Joined: Tue Sep 16, 2008 5:38 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby EViews Gareth » Mon Mar 20, 2017 4:25 pm

File->Open->Program in Add-ins Folder.


All add-ins are just EViews programs.
Follow us on Twitter @IHSEViews

Econoforecast
Posts: 25
Joined: Sun Feb 19, 2017 1:25 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Econoforecast » Mon Mar 20, 2017 4:26 pm

EViews Gareth wrote:File->Open->Program in Add-ins Folder.


All add-ins are just EViews programs.

Yep, just found it, thanks, see above post please, have edited.

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11369
Joined: Tue Sep 16, 2008 5:38 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby EViews Gareth » Mon Mar 20, 2017 5:25 pm

It doesn't make sense to run it on two series.
Follow us on Twitter @IHSEViews

Econoforecast
Posts: 25
Joined: Sun Feb 19, 2017 1:25 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Econoforecast » Mon Mar 20, 2017 6:56 pm

EViews Gareth wrote:It doesn't make sense to run it on two series.

Ok. Before I proceed any further, what is the difference between this add-in and the DM test in the following link? Scroll to bottom. I was unaware it was already inbuilt by IHS. Can you please clarify. Thanks.

http://www.eviews.com/help/helpintro.ht ... stics.html

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11369
Joined: Tue Sep 16, 2008 5:38 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby EViews Gareth » Mon Mar 20, 2017 7:47 pm

The add-in was available before the built in. There is no reason to use the add-in now.
Follow us on Twitter @IHSEViews

Econoforecast
Posts: 25
Joined: Sun Feb 19, 2017 1:25 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Econoforecast » Mon Mar 20, 2017 8:00 pm

EViews Gareth wrote:The add-in was available before the built in. There is no reason to use the add-in now.

I see. That clears it up. So the add-in and built in function by IHS are essentially the same.

Thanks.

Econoforecast
Posts: 25
Joined: Sun Feb 19, 2017 1:25 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby Econoforecast » Tue Apr 04, 2017 12:38 pm

EViews Gareth wrote:The add-in was available before the built in. There is no reason to use the add-in now.

Just a quick question again, what do <>prob, >prog and <prob actually stand for in this case? How does one interpret the test?

I am aware how the test works, how to interpret the t-test, just wondering what the above are for.
Attachments
17758316_10154823523108141_4740627384852321956_o.jpg
17758316_10154823523108141_4740627384852321956_o.jpg (105.25 KiB) Viewed 940 times

EViews Gareth
Fe ddaethom, fe welon, fe amcangyfrifon
Posts: 11369
Joined: Tue Sep 16, 2008 5:38 pm

Re: DMtest - Diebold-Mariano Forecast Evaluation Test

Postby EViews Gareth » Tue Apr 04, 2017 1:00 pm

Probabilities of the alternative hypotheses (that forecast 1 differs in accuracy from forecast 2, forecast 1 is less accurate, forecast 2 is more accurate).
Follow us on Twitter @IHSEViews


Return to “Add-in Support”

Who is online

Users browsing this forum: No registered users and 2 guests