[Bug]: The precisionRounding: 14 causes the =200.05 - 200 equation to print 0.0500000000000114 instead of 0.05

[AMBudnik]

Description

As in the title, if we use precisionRounding: 14 for the 200.05 - 200 calculation we get 0.0500000000000114 instead of 0.05 value. However, it all works well if we setup the precisionRounding to 10.

Video or screenshots

No response

Demo

https://codesandbox.io/s/black-lake-wdy2d3?file=/src/hyperformulaConfig.js:123-144

HyperFormula version

2.3.1

Your framework

none

Your environment

Chrome 116 / macOS Ventura

[sequba]

It is caused by the (im)precision of the JavaScript floating-point arithmetics. In most programming languages, calculations that involve fractional numbers are never 100% accurate, e.g. the famous case with 0.1 + 0.2 ≠ 0.3.

HyperFormula uses the standard JavaScript implementation of arithmetic operations, which says that 200.05 - 200 = 0.05000000000001137. To make this result more human-friendly, HyperFormula applies rounding according to precisionRounding parameter:

precisionRounding	Rounded result
10	0.05
11	0.05
12	0.05000000000001
13	0.050000000000011
14	0.0500000000000114
15	0.05000000000001137

Currently, our documentation says:

We recommend setting precisionRounding to a value between 10 and 14.

with 14 being the default value.

What can we do?

1. Change the default value of precisionRounding to 10 or 11 (breaking change)
2. Change the recommended values of precisionRounding
3. Explain the rounding issue better in our documentation:

• Add Setting precisionRounding too high will expose the floating-point calculation errors (for example, with precisionRounding set to 15, 0.1 + 0.2 results in 0.3000000000000001). to the description of precisionRounding
• Describe the imprecision of the floating-point arithmetics in our Specifications and limits guide.

[sequba]

After discussing it with @evanSe we decided to apply the documentation changes (points 2 and 3) in the scope of this task. Changing the default value of the config parameter (point 1) will be scheduled for the next major release.

[evanSe]

Changed my opinion on this, changing precisionRounding default value to something less than 14 causes more issues than it solves.

[sequba]

Instead of changing the default value, we will expand the reference to describe the js arithmetic issues better and recommend precisionRounding: 10 for most day-to-day use cases.