Aremos Databank Backup command

[mmiller2]

We have been transitioning from Aremos to EViews for a number of years now, and there is really only one thing we still need Aremos to accomplish - compressing databanks. We have tried, unsuccessfully, to implement this feature in EViews by opening an existing databank and copying all of the objects in it (i.e., just series and lists) to a newly created databank.

Although that process is slower and less efficient than doing it in Aremos, it does "work." However, the problem with that method is that the "Last Update" field, which is stored internally in the databank for each object, gets updated to the current date. Would it be possible to create an option to the EViews copy command to not update the "Last Update" field when bulk copying from one databank to another?

I believe that created and last updated fields are also stored internally in the databank itself (in the very first 256 byte block), as they are both displayed when a databank is opened and don't necessarily correspond to what the operating system says (e.g., Created on 16 JUN 2017 Last updated on 13 APR 2021, while OS says created ‎Wednesday, ‎March ‎24, ‎2021, ‏‎9:52:26 AM).

I could probably put together some poorly written code to do this myself in Java or Python, but I figured why not ask the professionals.

Backup.png (84.84 KiB) Viewed 24656 times

[EViews Gareth]

What's the end goal?

In Aremos how does creating a copy, compress the database?

[mmiller2]

Internally, the databank is a forest of B-trees with each forest based on the version (e.g., .a, .q, .m, .null, etc.) of each object. To improve runtime performance (at the expense of space), Aremos (and EVIews) does not actually remove objects in databanks from the trees on deletes or relocations (if more space is needed), it just marks things as deleted and leaves them in place. Over time, this leads to an accumulation of "deleted "space in the databank that is never reclaimed. A normal copy does not remove this space, but a backup command in Aremos achieves compression proportional to the amount of "deleted" space. Additionally, the backup command does not modify the metadata (i.e., Last Update) for objects. There is currently no way to accomplish this functionality in EViews.

[EViews Gareth]

Doesn't a repack do that?

[mmiller2]

Doesn't a repack do that?

I'm not aware of a repack command, but the dbpack command appears not to be implemented for databanks:

PackDatabase.png (2.86 KiB) Viewed 24639 times

[EViews Gareth]

Ah, I didn't understand you wanted EViews to do this on external databases, sorry.

Its unlikely we'll add features to support a dead external database type, but we'll make a note nevertheless.

[mmiller2]

Thanks for making a note of it, Gareth. Referring to the Aremos databank format as a dead external database type reminded me of that scene from Monty Python and the Holy Grail: https://www.youtube.com/watch?v=QcbR1J_4ICg