Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decimals lose precision when decoding. #116

Closed
Sekenre opened this issue Jul 20, 2021 · 0 comments
Closed

Decimals lose precision when decoding. #116

Sekenre opened this issue Jul 20, 2021 · 0 comments

Comments

@Sekenre
Copy link
Collaborator

Sekenre commented Jul 20, 2021

If you have a decimal fraction with more than the default 28 significant figures, it will be truncated on decode:

>>> import cbor2
>>> from decimal import Decimal
>>>
>>> example = Decimal('9.7703426561852468194804075821069770622934E-38')
>>> cbor2.loads(cbor2.dumps(example)) == example
False
>>> from decimal import localcontext
>>> with localcontext() as ctx:
...     ctx.prec = 60
...     cbor2.loads(cbor2.dumps(example)) == example
...
True

2 options to fix this, either set the context based on math.log10(significand) or build the decimal.Decimal object by unpacking the significand into a tuple of decimal digits.

Found using Hypothesis!
Sekenre added a commit to Sekenre/cbor2 that referenced this issue Jul 21, 2021
@Sekenre
Problem: agronholm#116 Decimals are losing precision on decode
0b94096 
Solution: Instead of construct the Decimal arithmetically, build an
intermediate Decimal containing the sign and digits, then build the
final result with a tuple of `(sign, digits, exponent)`. Since there is no
calculation going on, the default precision in the Decimal context does
not take effect.

Unfortunately this does not work with bigfloat because the exponent in
that case is a power of 2 instead of a power of 10.
Sekenre added a commit that referenced this issue Jul 23, 2021
@Sekenre
Problem: #116 Decimals are losing precision on decode
b21e20b 
Solution: Instead of construct the Decimal arithmetically, build an
intermediate Decimal containing the sign and digits, then build the
final result with a tuple of `(sign, digits, exponent)`. Since there is no
calculation going on, the default precision in the Decimal context does
not take effect.

Unfortunately this does not work with bigfloat because the exponent in
that case is a power of 2 instead of a power of 10.
@Sekenre Sekenre closed this as completed Jul 23, 2021
vamega pushed a commit to vamega/cbor2 that referenced this issue Nov 15, 2023
@Sekenre
Problem: agronholm#116 Decimals are losing precision on decode
66287fd 
Solution: Instead of construct the Decimal arithmetically, build an
intermediate Decimal containing the sign and digits, then build the
final result with a tuple of `(sign, digits, exponent)`. Since there is no
calculation going on, the default precision in the Decimal context does
not take effect.

Unfortunately this does not work with bigfloat because the exponent in
that case is a power of 2 instead of a power of 10.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Sekenre