Runtime Efficiency

[CharlieEVIEWS]

Dear All,

I'm generating/transforming/estimating an extremely large number of objects, and the program is looking to take a very, very long time, potentially a number of weeks. Where possible, I've split into different workfiles which close upon completion. All loops and conditions are nested in what I believe is the most efficient fashion. My question is (other than: are there any general tips for increasing run-time efficiency?) - if there are several millions of objects (64-bit EViews , how much is the program going to be slowed by my failure to adequately delete out-dated (no longer required) objects? Is the 'sorting' through the workfile to place new objects alphabetically what is causing the lengthy runtime? Will more ram increase efficiency in this regard, or, alternatively permit more objects per workfile and make the program more stable?

Best wishes,

Charlie

[EViews Gareth]

Are you running in quiet mode?

How much memory does EViews use during the program? (You can use Windows Task Manager to keep track)

[CharlieEVIEWS]

So far ive only run trials with a fraction of the data, but yes, always in quiet-mode and eviews seldom gets past 80% of RAM in taskmanager, albeit sometimes it can reach 90% and up. I have no idea why it doesnt run at a constant percentage of the RAM when no other applications on the machine are running.

[EViews Gareth]

Task manager should be able to tell you exactly how much memory (in MB) EViews is using, rather than just an overall percentage.

[CharlieEVIEWS]

Yeah, at present its about 300mb on this machine...?

[EViews Gareth]

And does it go up much above that during running?

[CharlieEVIEWS]

No, what is con-straining it from using more? Is it an inability to write the program in a way to utilize all available system resources? Do you have any kind of general tips in how to structure the workfiles or nest the loops/if's, and is deletion a strategy to reduce the time taken to run?

[EViews Gareth]

Well there's two causes of extreme slowness. The first is when you're handling so much data that you run out of RAM. It looks like you're not even close to doing that, from the sounds of it.
The second is just from the way the program is written. There aren't many tips I can give without seeing the program. One thing to look out for is to make sure you're not calculating things too many times. EViews is great in that it lets you use auto-series (i.e. an expression of a series - e.g. log(x)) pretty much everywhere, which increases the ease-of-use, but it can cause slow downs. If you use log(x) 100 times in your program, then EViews re-calculates log(x) 100 times. If you create a new series logx=log(X) at the top, and then use logx throughout your program, EViews only needs to calculate it once.

[CharlieEVIEWS]

Interesting, I hadnt even considered that operators like that would be slowing me down, but they undoutedly are. Thanks Gareth!

[CharlieEVIEWS]

Coming back to this post to avoid starting a new one - I think that the program I have written for a specific project is actually going to take several weeks to finish running, maybe longer. I am seeking a way to avoid this, if at all possible. All estimations appear to run relatively quickly but when it comes to calling an add-in (.fcast), the program is really dragging its heels (the addin is called a large number of times.)

I was wondering if anybody had any suggestions on how I might improve the efficiency of such a function in particular?

Loosely, the program appears to be doing things similar to the recent papers of Todd Clark, but it appears he is using RATS, and although I'm fairly familiar with RATs, i much prefer EViews for this kind of work.

Best wishes as always, and all comments are of course greatly appreciated.

Charlie

[EViews Gareth]

VAR forecasting with the .fcast add-in? If you're using standard errors too, it will take a long time. The VAR forecast add-in does standard errors through simulation. Those simulations will take a long time.

[CharlieEVIEWS]

no standard errors are being computed, no. Do you have any suggestions? I am considering splitting it into as many chunks as possible and just opening the program in several versions of EViews to maximize cpu utilization....?

[EViews Gareth]

You could do that.

If it is really the .fcast that's taking all the time, you're probably better off not using it, and doing the forecast manually.

If you open the add-in up, you'll see it is pretty simple - it just adds the VAR to a model object and then solves the model object over the forecast period.

If your VARs are all independent, you could add them all to the same model object and then just do one solve, which would be faster than creating a whole bunch of different ones.

[CharlieEVIEWS]

Wow, that was surprisingly easy to extract the necessary code from the add-in. A quick benchmark exercise takes 686.288 seconds for the add-in call procedure, and only 399.467 seconds for the manual solve. Outstanding! For anyone interested, I am using the extracted lines of code:

Code: Select all

rename *__v *______temp %mname = @getnextname("vmod") {%vname}.makemodel({%mname}) {%mname}.scenario(a=__v,n) "vscen" {%mname}.solve(d=d) rename *__v *{%fname} rename *_____temp *__v rename vvvvvvvtemp* vtemp* delete vmod01

Thanks Gareth - outstanding support as always!

[EViews Gareth]

You can probably get rid of some of the renames and deletes to speed it up even further