Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the constant data type constructors for ListArray #2311

Closed
HaoYang670 opened this issue Aug 4, 2022 · 1 comment · Fixed by #2327
Closed

Add the constant data type constructors for ListArray #2311

HaoYang670 opened this issue Aug 4, 2022 · 1 comment · Fixed by #2327
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@HaoYang670
Copy link
Contributor

HaoYang670 commented Aug 4, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
There are lots of places in our code when we need to build the data type for a list array:

let data_type = if OffsetSize::IS_LARGE {
    DataType::LargeList(field)
} else {
    DataType::List(field)
};

There are 2 shortcomings:

  1. We judge the OffsetSize at runtime (I am not sure whether the LLVM could do constant folding on it), however, this computation could be done at compile time.
  2. We have to copy this logic everywhere, which leads to repeated code and it is easy to make mistakes.

Describe the solution you'd like
Add a constant value for List array:

const DATA_TYPE_CONSTRUCTOR: fn(Box<Field>) -> DataType = 
    if OffsetSize::IS_LARGE { DataType::LargeList} else {DataType::List}

Then we can simplify the code to get the data type at runtime:

let data_type = Self::DATA_TYPE_CONSTRUCTOR(field);

Describe alternatives you've considered
We could not do this.

Additional context
Related to #2274.

@HaoYang670 HaoYang670 added the enhancement Any new improvement worthy of a entry in the changelog label Aug 4, 2022
@HaoYang670
Copy link
Contributor Author

I will work on this.

@alamb alamb added the arrow Changes to the arrow crate label Aug 5, 2022
@alamb alamb changed the title Add the constant data type constructor for ListArray Add the constant data type constructors for ListArray Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
2 participants