Copy from another workfile slower in Eviews 13

[econmarc]

Hi,

Since I started using Eviews 13 instead of Eviews 12, I realize that most program are a lot slower and it seems to be linked to copying series from another workfile. As an exemple, I run a small test on both eviews 12 and 13 and the time was 11 secondes longer with Eviews 13. This can add up quickly for some of my program.

tic
copy(c=l) "link\wfile.wf1"::D5\serie1 D5\
copy(c=l) "link\wfile.wf1"::D5\serie2 D5\
scalar time = @toc

Eviews 12: time = 2.567
Eviews 13: time = 13.351

Is there a reason for Eviews 13 to be that much slower, and is there something I can do to make it go as fast as Eviews 12 was?

Best,
Marc

[EViews Gareth]

Is your copy of EViews 13 up to date? (Help->About EViews will give you the build date)

[econmarc]

Hi Gareth,

My coworker has the most updated version of Eviews 13 and he has the same result as me.

[EViews Gareth]

Could you provide a copy of the workfile (or create a similar sized workfile that has the same issue and provide that)?

[econmarc]

Unfortunately I can't since the workfile I use is large. I did more test and it doesn't seems to be workfile related. I obtain similar result when I try to copy from smaller workfile. My latest test was on a 24MB workfile and it took 0.3 sec on eviews 12 to copy two series when it took 1.3 sec on eviews 13. On a very large workfile (4 GB) it took 1min30 to copy 1 serie in eviews 13 but only 14 sec to do the same thing in eviews 12.

One thing I observe is that in eviews 13 the time lost seems to come from loading the page. Even when using mode quiet, it will display at the bottom left that it is loading the page. When I use eviews 12 there is no message about loading page.

[EViews Gareth]

If you created the test workfile in a program, could you share the program?

[econmarc]

I'm pretty sure you could use any workfile you have to test this. I wrote you a small program that create a workfile with 2000 random series. You need to replace the "" of %link with the path of the folder where you will save the wfile1.

Those are the times I get when running this code:
Eviews 13: time = 3.8
Eviews 12: time = 0.16

Code:
wfcreate(wf=wfile1, page=D5)D5 1970 @now
for !x=1 to 2000
genr serie{!x} = rnd
next
%link = ""
wfsave %link
wfclose wfile1

wfcreate(page=d5)d5 1970 @now
%copy_link = @addquotes(%link)
tic
copy(c=l) {%copy_link}::D5\serie1 D5\
scalar time = @toc

[EViews Gareth]

For me:
EViews 12: 0.016
EViews 13: 0.012

Are you saving to a network drive or anything?

[econmarc]

Yes, I am saving to a network drive.

I just try the same code with saving on my local drive. I still get a faster time with Eviews 12, but it is much much closer. It does seems to be link to the use of network drive. Is there a way to make eviews 13 be as fast as 12 on network drive?

[EViews Gareth]

EV12 and 13 same machine, presumably?

[econmarc]

Same machine yes

[EViews Steve]

Just tested this. It has something to do with copying a series from a workfile that is not already open. If you open the source workfile before performing the COPY operations, you should see negligible performance difference. And honestly, this is probably preferable to do in your program otherwise each COPY operation will result in opening and closing the same workfile for each call.

As to why 13 is taking longer to open workfiles, I'll have to take a closer look...

Steve

[econmarc]

Hi Steve,

You are right that in some case it would be better to open the workfile before copying, but in many case it was way faster to use copy instead of opening the workfile (at least on Eviews 12).

As an exemple, using Eviews 12, if I want to copy only one serie from a large workfile (4 GB) to another workfile it take 13 sec if I use copy while the workfile is close and 1min30 if I open the workfile first (since the workfile take a long time to open). If I repeat the same test on Eviews 13, both way takes 1min30.

This drop in performance have a huge effect on some of our program that we have been runing for years on multiple previous version of Eviews.

[EViews Steve]

Understood.

I took a closer look at the changes in 13 that was causing this, and it appears that our logic behind the COPY command (when the source file isn't already open) changed last year. This was because someone noticed that the COPY operation (in this scenario) didn't actually get all series property values, such as the Label property.

The original code was using a faster (but more limited) set of functions that could find and load a single series object from a large workfile (WF1 only) WITHOUT loading everything into memory. For the sake of completeness, and also because of the complicated nature of these special read methods, we decided to change the COPY operation to instead load the entire workfile into memory and then performed the COPY as usual. Thus the slowness in your COPY operations in 13.

To support your scenario, and as long as you don't mind not copying all series properties, I've gone ahead and added a new LEGACY option to our COPY command, so that you can continue to use the older set of functions to read data without having to load the entire workfile. To use this, you'll have to add this option to all of your COPY commands in your programs. This new COPY option will be ignored by 12 without error. This will be available in the next 13 patch.

Code: Select all

copy(c=l, LEGACY) {%copy_link}::D5\serie1 D5\

As a side note, I'll mention that your use of and unloaded source workfile during a COPY is not ideal. Our COPY command actually doesn't explicitly support this scenario (at least in our documentation) and a few versions ago would always error in this scenario saying the source workfile could not be found. Also, instead of using a workfile to store your series objects in this manner, it would make more sense to use an EViews Database (.EDB) as it was designed for this exact purpose. STOREing and FETCHing objects from an EViews database is designed to be done without having to rewrite or load the entire EDB file into memory. This is normally NOT true with our workfiles.

Steve

[econmarc]

Thank you for taking the time to develop a patch for our problem, it is really appreciated. Do you know when the next patch is plan to go live?