contract a page with uniquevals?

[CharlieEVIEWS]

Can somebody help me speed up the following code snippet? I want to pagecontract to leave only unique values. However, I cant figure out how to use @uniquevals effectively, and this loop is dragging as there are millions of observations in the workfile. The intention is to drop all observations which dont have unique entries in the alpha 'name' (only the first observed instance of each name string are kept, sequential ones are dropped: ordering unimportant however):

Code: Select all

genr uniquedum = 1 for !a = 2 to @obsrange for !b = !a-1 to 1 step -1 if name(!a) = name(!b) then uniquedum(!a) = 0 !b = 1 endif next !c = (!a/@obsrange)*100 statusline !c % filtering of unique values next pagecontract if uniquedum = 1

Help appreciated as always! Best wishes to all.

Charlie




**************EDIT**************

This is faster, but I feel that it can be faster still....:

Code: Select all

svector unique_svector=@uniquevals(name) for !a = 1 to @rows(unique_svector) for !b = 1 to @obsrange if unique_svector(!a)=name(!b) then uniquedum(!b) = 0 !b = @obsrange endif next !c = (!a/@rows(unique_svector))*100 statusline !c % filtering of unique values next pagecontract if uniquedum = 1

[mamo]

Hi,
what about sorting the workpage?
Create an example workpage.

Code: Select all

'Generate example workpage wfcreate u 100 !lo=1 !hi=10 series num=@round(!lo+(!hi-!lo)*rnd) alpha name=@chr(64+num) '' Generate index series genr index=@obsnum '' Sort page sort name

1. To keep all cases with the first occurrences of the unique values of name:

Code: Select all

''Generate indicator series smpl 1 1 genr selector=1 To keep smpl 2 @last selector = (name<>name(-1)) smpl @all pagecontract if selector ''Get the original ordering of the observations selected 'sort index

2. To keep all cases with the last occurrences of the unique values of name:

Code: Select all

''Generate indicator series smpl @last @last genr selector=1 smpl @first @last-1 genr(r) selector = (name<>name(1)) smpl @all pagecontract if selector ''Get the original ordering of the observations selected 'sort index

[CharlieEVIEWS]

This is an outstanding solution. Thank you ever so much. It works like a charm.

[mamo]

Even shorter (for 2nd case of taking last observations)

Code: Select all

wfcreate u 100 !lo=1 !hi=10 series num=@round(!lo+(!hi-!lo)*rnd) alpha name=@chr(64+num) smpl @all genr selector=0 svector unique_svector=@uniquevals(name) for !i = 1 to @rows(unique_svector) %s=unique_svector(!i) genr idx=(name=%s)*@obsnum selector=selector+(@obsnum=@max(idx)) next pagecontract if selector

Best,
mamo