Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is possible to implement GenericListArray::from_iter ? #3702

Closed
suxiaogang223 opened this issue Feb 12, 2023 · 2 comments
Closed

Is possible to implement GenericListArray::from_iter ? #3702

suxiaogang223 opened this issue Feb 12, 2023 · 2 comments
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@suxiaogang223
Copy link
Contributor

Hi,I'm new to Rust and I really like the Rust language and the project. This is what came to mind when I read the code for this project.

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The existed method GenericListArray::from_iter_primitive can only create Array of List<T>,it can't create Array of List<List<T>>and other more nested List.

Describe the solution you'd like
Is possible to implement GenericListArray::from_iter so we can create GenericListArray like this:

    // create ListArray [[[1,2],null],[[3]]]
    let data = vec![
        Some(vec![Some(vec![Some(1), Some(2)]), None]),
        Some(vec![Some(vec![Some(3)])]),
    ];

    let list_array = GenericListArray::<i32>::from_iter::<Int32Type,..?>(data);

Describe alternatives you've considered
Is there a way to support arbitrary nesting depth of list handlers via generic implementation?
I'm curious if Rust's metaprogramming is powerful enough to implement this feature.

Additional context
https://docs.rs/arrow-array/32.0.0/arrow_array/array/struct.GenericListArray.html#method.from_iter_primitive

@suxiaogang223 suxiaogang223 added the enhancement Any new improvement worthy of a entry in the changelog label Feb 12, 2023
@tustvold
Copy link
Contributor

tustvold commented Feb 12, 2023

I suspect FromIterator will run into issues with trait specialization, we run into this also with supporting both optional and non-optional iterators, but I can't say I've tried to make this work.

That being said you can write something like this, which may be sufficient for your needs. The reason this works is you "help" the compiler by fully specifying the nested type in the form of the builder.

let mut builder = ListBuilder::new(ListBuilder::new(Int32Builder::new()));
builder.extend(vec![
    Some(vec![Some(vec![Some(1), None, Some(2)]), None]),
    Some(vec![]),
    Some(vec![None]),
    None,
]);
let built = builder.finish();

@suxiaogang223
Copy link
Contributor Author

thanks, I read the implementation of Extend and it was very enlightening for me.

@tustvold tustvold added the arrow Changes to the arrow crate label Feb 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

2 participants