Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Row Format: API to check if datatypes are supported #3077

Closed
crepererum opened this issue Nov 10, 2022 · 1 comment · Fixed by #3080
Closed

Row Format: API to check if datatypes are supported #3077

crepererum opened this issue Nov 10, 2022 · 1 comment · Fixed by #3080
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@crepererum
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'm trying to use the arrow row format in DataFusion. For that it would be helpful to know early (read: before piping any actual payload through physical execution) if a certain data type is supported by the row format or not.

Describe the solution you'd like
RowConverter::new should check supported types and return a Result<> if types are not supported. RowConverter::convert_{columns,rows} should never bail out due to unsupported types.

Describe alternatives you've considered
Piping some one-row payload through the converter to check if the types would be supported. This seems to be a hack though.

Additional context
-
@crepererum crepererum added the enhancement Any new improvement worthy of a entry in the changelog label Nov 10, 2022
crepererum added a commit to crepererum/arrow-rs that referenced this issue Nov 10, 2022
@crepererum
refactor: check supported types when creating RowConverter
cff7a4a 
Check supported row format types when creating the converter instead of
during conversion. Also add an additional method
`RowConverter::supports_fields` to check types w/o relying on an error.

Closes apache#3077.
@crepererum crepererum mentioned this issue Nov 10, 2022
Merged
tustvold added a commit that referenced this issue Nov 10, 2022
@crepererum @tustvold
early type checks in RowConverter (#3080)
8d364fe 
* refactor: remove duplicate code

Decimal types are already handled by `downcast_primitive`.

* refactor: check supported types when creating `RowConverter`

Check supported row format types when creating the converter instead of
during conversion. Also add an additional method
`RowConverter::supports_fields` to check types w/o relying on an error.

Closes #3077.

* Simplify

Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
@alamb
Copy link
Contributor

alamb commented Nov 11, 2022

label_issue.py automatically added labels {'arrow'} from #3080

@alamb alamb added the arrow Changes to the arrow crate label Nov 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

early type checks in RowConverter crepererum/arrow-rs
2 participants
@alamb @crepererum