You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
units.parse has rounding errors when calculating 0.5 during conversion with the milli- suffix (lowercase m) and Kilo- (k/K). All other prefixes seem to calculate properly.
System Details:
OPA version: >= v0.41.0 (checked against both 0.41.0 and 0.42.0)
@tmos22 Thanks for reporting this! I was able to reproduce the issue locally from your test cases, and I have a pretty good idea of how to fix the issue. Internally, it's caused by floating-point rounding issues, even though OPA uses big.Float internally for the units parsing builtin.
The solution is probably going to involve normalizing the path that incoming units quantities go through, so that they experience identical rounding behavior, regardless of whether we're dealing with milli or Kilo units.
Previously, we used big.Float for units parsing, but this resulted in
occasional rounding issues, due to values like 0.001 not being perfectly
representable in floating-point format.
This issue was fixed by switching units parsing to use big.Rat, since
Rationals can generally handle such quantities with perfect precision.
Fixesopen-policy-agent#4856.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Previously, we used big.Float for units parsing, but this resulted in
occasional rounding issues, due to values like 0.001 not being perfectly
representable in floating-point format.
This issue was fixed by switching units parsing to use big.Rat, since
Rationals can generally handle such quantities with perfect precision.
Fixes#4856.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Short description
units.parse
has rounding errors when calculating0.5
during conversion with the milli- suffix (lowercasem
) and Kilo- (k
/K
). All other prefixes seem to calculate properly.System Details:
Steps To Reproduce
opa run
units.parse("500m")
0.5000000000000000104
units.parse("0.0005K")
0.49999999999999999997
Expected behavior
All units that should be parsed to the same value should equal the same value (all 0.5s should be equal)
The text was updated successfully, but these errors were encountered: