Outlier treatment in a double loop

[heer0]

Hi guys,

I am using EViews 10 (Univ. ed.) and seek to implement a specific procedure for outlier treatment. In particular, I want to define outliers as observations with deviations from the median exceeding six times the interquartile range (in absolute terms). These outliers should then be replaced by the median value of the five preceding observations.

So far, my code looks like this:

Code: Select all

'Create a blank output table to store median and interquartile range of all series in the group "xt_all" table (3, xt_all.@count+1) Xt_stat setcell(Xt_stat, 1, 1, "Statistic / Series", "l") setcell(Xt_stat, 2, 1, "Median", "l") setcell(Xt_stat, 3, 1, "Interquartile range", "l") 'Loop through all series in the group "xt_all" for !i=1 to xt_all.@count 'Enforce the correct series names %name = xt_all.@seriesname(!i) 'Obtain and store median and interquartile range for each series in "Xt_all" Xt_stat(1,1+!i)=%name Xt_stat(2, 1+!i)= @median({%name}) Xt_stat(3, 1+!i)= @quantile({%name}, 0.75) - @quantile({%name}, 0.25) next 'Loop through the number of observations of the current series for !i=1 to xt_all.@count and !j=1 to @obssmpl 'Enforce the correct series name %name = xt_all.@seriesname(!i) 'Consider each observation one by one !observation = !j 'Create a temporary vector of the five observations preceding the current observation vector temp = @fill(@elem({%name}, {!observation}-1), @elem({%name}, {!observation}-2), @elem({%name}, {!observation}-3), @elem({%name}, {!observation}-4), @elem({%name}, {!observation}-5)) 'Define rule for identifying outliers if @abs(@elem({%name}, {!observation})) - @abs(Xt_stat(2, 1+!i)) > 6*@abs(Xt_stat(3, 1+!i)) then 'Replace outliers by the median of the previous five observations @elem({%name}, {!observation}) = @median(temp) 'Delete vector "temp" before starting next iteration delete temp next

However, executing the code results in an error message since I try to loop over two different numerical variables at once.

Do you have any advice on how to proceed? Your help will be much appreciated!

[EViews Gareth]

You want a nested loop:

Code: Select all

for !i=1 to xt_all.@count for !j=1 to @obssmpl next next

However, looping through observations is generally a bad idea, and is not required in EViews. I think all you need to do is something like:

Code: Select all

group prev5 x(-1 to -5) x = @recode((x-@median(x))> 6*@abs(@quantile(x,0.75)-@quantile(x,0.25)), @rmedian(prev), x)

i.e. create a group with the 5 lagged values of X. Then use the recode function to change any values of x that are greater than your condition to the median of that group.

[heer0]

You want a nested loop:

Code: Select all

for !i=1 to xt_all.@count for !j=1 to @obssmpl next next

However, looping through observations is generally a bad idea, and is not required in EViews. I think all you need to do is something like:

Code: Select all

group prev5 x(-1 to -5) x = @recode((x-@median(x))> 6*@abs(@quantile(x,0.75)-@quantile(x,0.25)), @rmedian(prev), x)

i.e. create a group with the 5 lagged values of X. Then use the recode function to change any values of x that are greater than your condition to the median of that group.

Thanks SO MUCH for your help Gareth. I tweaked your last bit of code a little:

Code: Select all

'Loop through all series in the group "xt_out" for !i=1 to xt_out.@count 'Enforce the correct series names %name = xt_out.@seriesname(!i) 'Define a temporary group for each series with its five lags group prev5_{%name} {%name}(-1 to -5) 'Replace outliers {%name} = @recode(@abs({%name}-@median({%name}))> 6*@abs(@quantile({%name},0.75)-@quantile({%name},0.25)), @rmedian(prev5_{%name}), {%name}) 'Delete the current temp group from work page delete prev5_{%name} next

Now it works as intended. Thank you again for pointing me into the right direction! You are the best. ;)