Large table performance issues
Posted: Wed Nov 23, 2016 4:01 am
Hi there,
I was not sure if posting this either here or in the data manipulation section, but anyways.
I am experiencing serious perfomance issues when creating a long table (93860 rows and 3 columns). I am creating this table because I need to extract 1805 series (52 datapoints each) from a workfile into "relational" table format to an excel spreadsheet. I am not aware of any predefined funtion of Eviews that could do this. As an example, from Eviews I can take the series in columns (or rows) like this:
date ser1 ser2
1999 1 2
2000 2 3
2001 3 4
But I need the info to be shown like this:
Date Name Obs
1999 ser1 1
2000 ser1 2
2001 ser1 3
1999 ser2 2
2000 ser2 3
2001 ser2 4
My (not very elegant code is attached with the associated workfile). Basically what is does it to generate small auxiliary tables and then it merges everything into a big output table. My idea was to then export such a big table into a csv file that I could read with Excel. I know I could do this with a VBA macro but it usually crashes and it would be much better for many reasons to have the whole process controlled and programmed in Eviews.
What is the problem? I measure that it takes 14 sec to write the first 10000 rows, 37 seconds to write the following 10000 rows, 1:03 min the following 10000 and then it becomes even slower. Can you tell me please how I could do this in a better way?
Many thanks!
Fede
I was not sure if posting this either here or in the data manipulation section, but anyways.
I am experiencing serious perfomance issues when creating a long table (93860 rows and 3 columns). I am creating this table because I need to extract 1805 series (52 datapoints each) from a workfile into "relational" table format to an excel spreadsheet. I am not aware of any predefined funtion of Eviews that could do this. As an example, from Eviews I can take the series in columns (or rows) like this:
date ser1 ser2
1999 1 2
2000 2 3
2001 3 4
But I need the info to be shown like this:
Date Name Obs
1999 ser1 1
2000 ser1 2
2001 ser1 3
1999 ser2 2
2000 ser2 3
2001 ser2 4
My (not very elegant code is attached with the associated workfile). Basically what is does it to generate small auxiliary tables and then it merges everything into a big output table. My idea was to then export such a big table into a csv file that I could read with Excel. I know I could do this with a VBA macro but it usually crashes and it would be much better for many reasons to have the whole process controlled and programmed in Eviews.
What is the problem? I measure that it takes 14 sec to write the first 10000 rows, 37 seconds to write the following 10000 rows, 1:03 min the following 10000 and then it becomes even slower. Can you tell me please how I could do this in a better way?
Many thanks!
Fede