Re: Coding: performance in large databases
Posted: Thu May 22, 2014 4:09 pm
Here's some code with an example of the block calculation. It does so by computing sets of 2000 x 2000 correlations. The 7000 x 7000 results matrix is declared. In each block we pull off 1000 series and compute the correlation matrix for the 1000 plus a unique set of 1000. I'll let you do the placement of the individual sets into the results matrix. Alternately, you could skip the declaration of the full matrix, and do your analysis using results for the individual computations.
Code: Select all
wfcreate test u 1 360
for !i=1 to 7000
%s = "y" + @str(!i, "g04")
series {%s} = nrnd
next
group a y*
sym(7000) results
!nser = 1000
!nblocks = 7000/!nser
!blksize = 7000/!nblocks
for !j=1 to !nblocks
%s{!j} = ""
for !k=1 to !blksize-1
!m = (!j-1)*!blksize + !k
%s{!j} = %s{!j} + " " + a.@seriesname(!m)
next
string s{!j} = %s{!j}
next
for !j=1 to !nblocks
for !k=!j+1 to !nblocks
group grp_{!j}_{!k} {%s{!j}} {%s{!k}}
do grp_{!j}_{!k}.cor(out=aa, pairwise) corr
next
next