contract a page with uniquevals?

[CharlieEVIEWS]

Can somebody help me speed up the following code snippet? I want to pagecontract to leave only unique values. However, I cant figure out how to use @uniquevals effectively, and this loop is dragging as there are millions of observations in the workfile. The intention is to drop all observations which dont have unique entries in the alpha 'name' (only the first observed instance of each name string are kept, sequential ones are dropped: ordering unimportant however):

Code: Select all

genr uniquedum = 1
for !a = 2 to @obsrange
   for !b = !a-1 to 1 step -1
      if name(!a) = name(!b) then
         uniquedum(!a) = 0
         !b = 1
      endif
   next
   !c = (!a/@obsrange)*100
   statusline !c %  filtering of unique values
next
pagecontract if uniquedum = 1

Help appreciated as always! Best wishes to all.

Charlie




**************EDIT**************

This is faster, but I feel that it can be faster still....:

Code: Select all

svector unique_svector=@uniquevals(name)
for !a = 1 to @rows(unique_svector)
   for !b = 1 to @obsrange
      if unique_svector(!a)=name(!b) then
         uniquedum(!b) = 0
         !b = @obsrange
      endif
   next
   !c = (!a/@rows(unique_svector))*100
   statusline !c %  filtering of unique values
next
pagecontract if uniquedum = 1

[mamo]

Hi,
what about sorting the workpage?
Create an example workpage.

Code: Select all

'Generate example workpage
wfcreate u 100
!lo=1
!hi=10
series num=@round(!lo+(!hi-!lo)*rnd)
alpha name=@chr(64+num)
'' Generate index series
genr index=@obsnum
'' Sort page
sort name


1. To keep all cases with the first occurrences of the unique values of name:

Code: Select all

''Generate indicator series
smpl 1 1
genr selector=1
To keep
smpl 2 @last
selector = (name<>name(-1))
smpl @all
pagecontract if selector
''Get the original ordering of the observations selected
'sort index


2. To keep all cases with the last occurrences of the unique values of name:

Code: Select all

''Generate indicator series
smpl @last @last
genr selector=1
smpl @first @last-1
genr(r) selector = (name<>name(1))
smpl @all
pagecontract if selector
''Get the original ordering of the observations selected
'sort index


[CharlieEVIEWS]

This is an outstanding solution. Thank you ever so much. It works like a charm.

[mamo]

Even shorter (for 2nd case of taking last observations)

Code: Select all

wfcreate u 100
!lo=1
!hi=10
series num=@round(!lo+(!hi-!lo)*rnd)
alpha name=@chr(64+num)
smpl @all
genr selector=0
svector unique_svector=@uniquevals(name)
for !i = 1 to @rows(unique_svector)
   %s=unique_svector(!i)
    genr idx=(name=%s)*@obsnum
    selector=selector+(@obsnum=@max(idx))
next
pagecontract if selector


Best,
mamo