Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ColumnWriterImpl::write_batch_with_statistics can write incorrect page statistics #2015

Closed
tustvold opened this issue Jul 6, 2022 · 0 comments · Fixed by #2022
Closed

ColumnWriterImpl::write_batch_with_statistics can write incorrect page statistics #2015

tustvold opened this issue Jul 6, 2022 · 0 comments · Fixed by #2022
Labels
bug parquet Changes to the parquet crate

Comments

@tustvold
Copy link
Contributor

tustvold commented Jul 6, 2022

Describe the bug

If calls to write_batch_with_statistics are mixed with calls to write_batch it can lead to invalid page statistics, as they will only be computed for the data passed to write_batch.

To Reproduce

Inspect code

Expected behavior

These methods should behave consistently

Additional context
Add any other context about the problem here.
@tustvold tustvold added the bug label Jul 6, 2022
This was referenced Jul 6, 2022
Closed
Merged
@alamb alamb changed the title ColumnWriterImpl::write_batch_with_statistics incorrect page statistics ColumnWriterImpl::write_batch_with_statistics can write incorrect page statistics Jul 7, 2022
@alamb alamb added the parquet Changes to the parquet crate label Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
2 participants
@alamb @tustvold