Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: store small decimal values as int #768

Open
wangrunji0408 opened this issue Jan 10, 2023 · 1 comment
Open

perf: store small decimal values as int #768

wangrunji0408 opened this issue Jan 10, 2023 · 1 comment

Comments

@wangrunji0408
Copy link
Member

RisingLight now uses rust_decimal to represent decimal values. However, computations on this type are relatively slow due to its large precision (96 bits) and variable scale. For small decimals, e.g. DECIMAL(15,2) in TPC-H tables, we could store and compute them as i64. This would bring a huge performance boost (considering SIMD). A potential challenge of this change is that we can no longer tell the exact value from array without its datatype. 🤔

Reference: https://duckdb.org/docs/sql/data_types/numeric#fixed-point-decimals

@unconsolable
Copy link
Member

IMHO another challenge is converting arrays from different WIDTH/SCALE to perform arithmetic, comparison, etc... Seems that DuckDB puts some efforts on it, while arrow-rs don't consider it.

ref. apache/arrow-rs#2637, duckdb/duckdb#858

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants