Possible issue with rounding third decimal place

[aarnon]

I have a value with three decimal places, say 5.225, that I would like display with just two decimal places. But when I enter the command

Code: Select all

_this.setformat(@all) f.2

my 5.225 rounds to 5.22 rather than 5.23.

Whether a 5 rounds up or down seems to depend on the value. For example, 10.225 rounds to 10.23. If I change the numbers slightly to 5.325 and 10.325, it goes the other way: I get 5.33 and 10.32...

As far as I can tell, this only happens with 5's in the third decimal place. If have 5.25 and set format to f.1, I get 5.3. If I have 5.2225 and set format to f.3 I get 5.223.

Bug? Or are the rules of rounding way more complex than I think?

[startz]

Is it possible that what you are seeing as 5.225 is actually 5.2249?

[EViews Gareth]

Almost certainly due to machine precision. A computer doesn't store decimals exactly, but stores them at, in EViews case, double precision.

It is probably the case that 5.225 is actually stored as 5.2249999999999999999, which then rounds down.

[EViews Gareth]

Is it possible that what you are seeing as 5.225 is actually 5.2249?

Code: Select all

create u 5 series y=5.225 series x=10.225 group g x y y.setformat f.2 x.setformat f.2 show g

[aarnon]

Sorry, should have been clearer: those are just numbers I made up. When I put in

Code: Select all

show 5.225 10.225 _this.setformat(@all) f.2

I see 5.22 and 10.23.

Also nearly forgot: EViews 8 Enterprise 64-bit, Jul 29 2014 build.

[EViews Gareth]

Of course this is a display issue only (still an issue, but not as serious as a calculational one).

[aarnon]

So there's no fix, other than say adding 1/100000000000 or something? Like you say, it's just display so not a big deal.

[EViews Chris]

Here's a bit more detail as to what is going on.

Most decimals aren't exactly representable in the format that is used to hold numbers within EViews (IEEE double precision floating point). This is because IEEE floating point uses a power of two for the exponent of the number rather than a power of 10.

The number 5.225 (5225 * 10^-3) is actually stored in IEEE double precision as:

5882827013252710 * 2^-50

However, this representation is not exact - it is just the nearest number to 5.225 available in IEEE double precision format.

if we convert 5882827013252710 * 2^-50 back into a 17-digit decimal number we get:

5.2249999999999996

which is slightly less than 5.225.

When we round this to 2 decimal places for output, the number is closer to 5.22 than 5.23, so we end up outputting 5.22.

It's not ideal but it's not easy to provide a perfect solution. One approach would be to hold decimals as true base ten numbers rather than numbers using powers of two, but there would be a substantial performance penalty in this approach because the CPU of your computer actually works with base two numbers, so moving back and forth would slow things down quite a lot.

At the end of the day, your solution of adding a small positive offset onto the number to avoid rounding down in cases where you know the series contains numbers to a fixed number of decimal places is probably a reasonable workaround.

[startz]

Chris,

Suppose rather than comparing the relevant digit to 5, you created the full comparison in IEEE. In other words, when asked to round y=5.225, generate xIEEE=5.225 and then round up if y>=xIEEE. Would this work as the rounding "error" is identical on both sides of the comparison?