Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IPC] Lower level API to access requirement_alignment and other extensions #5588

Open
alamb opened this issue Apr 4, 2024 · 2 comments
Open
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@alamb
Copy link
Contributor

alamb commented Apr 4, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

As part of #5554 @hzuo added the ability to enforce alignment

We could add a with_require_alignment to the FileDecoder

However, given the way the read_dictionary is structured, there was no way to add this option without breaking the API, as @tustvold noted https://github.com/apache/arrow-rs/pull/5554/files#r1549239776

Thus the require_alignment argument is now not exposed

Describe the solution you'd like

It would be nice to have a way to add new options in a way that did not break API compatibility going forward, and expose the requirement_alignment feature

Describe alternatives you've considered

@tustvold suggests https://github.com/apache/arrow-rs/pull/5554/files#r1550178314

possibly some sort of Decoder struct with a member function on it...

Ideally it would be designed so that it could be shared across things like StreamDecoder and FileDecoder

Additional context

@alamb alamb added arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog labels Apr 4, 2024
@hzuo
Copy link
Contributor

hzuo commented Apr 4, 2024

Small clarification: The Decoders (FileDecoder and StreamDecoder) do already expose with_require_alignment as part of #5554.

However, read_record_batch and read_dictionary don't expose the functionality right now - that's the subject of this issue.

Thanks for filing @alamb !

@alamb alamb changed the title [IPC] Decoder / lower level API to access requirement_alignment and other extensions [IPC] Lower level API to access requirement_alignment and other extensions Apr 4, 2024
@alamb
Copy link
Contributor Author

alamb commented Apr 4, 2024

Thanks -- updated the title

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

2 participants