Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add default division precision WARNING in README #338

Open
kai-thompson opened this issue Dec 27, 2023 · 2 comments
Open

Add default division precision WARNING in README #338

kai-thompson opened this issue Dec 27, 2023 · 2 comments

Comments

@kai-thompson
Copy link

The readme states that this library can handle up to 31 precision after the decimal. This is fine until a division operation is performed.
@mwoss
Copy link
Member

mwoss commented Dec 30, 2023

DivisionPrecision variable specifies the precision of the Div operation, by default it's set to 16 for performance reasons as most users do not need more precise calculations. The variable is documented and exposed to the end users.

Do you think we should add this information to README?

@kai-thompson
Copy link
Author

@mwoss I think it would be helpful to add in the README. I may have overlooked the documentation, but I only found this after explicitly looking for a hard-coded 16 in the source since I noticed numbers rounding to 16 decimal places.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mwoss @kai-thompson