Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Utility to carefully determine Iterable-like inputs for as_column #8796

Closed
galipremsagar opened this issue Jul 20, 2021 · 2 comments · Fixed by #15739
Closed

[FEA] Utility to carefully determine Iterable-like inputs for as_column #8796

galipremsagar opened this issue Jul 20, 2021 · 2 comments · Fixed by #15739
Labels
cuDF (Python) Affects Python cuDF API. feature request New feature or request
Projects
Feature Planning
Milestone
cuDF Python Refac...

Comments

@galipremsagar
Copy link
Contributor

Is your feature request related to a problem? Please describe.
We have a bunch of pa.array calls that happen in our Column constructor i.e., as_column. Most of the Iterable objects get sent to pa.array calls and thus get exhausted giving no opportunity to retrieve back the previous elements and this is problematic as we do try/except around this logic. Hence we would need a proper utility function to determine if the input to as_column is an Iterator that is not a sequence.

cc: @shwina
@galipremsagar galipremsagar added feature request New feature or request Needs Triage Need team to review and classify cuDF (Python) Affects Python cuDF API. labels Jul 20, 2021
@github-actions github-actions bot added this to Needs prioritizing in Feature Planning Jul 20, 2021
@beckernick beckernick removed the Needs Triage Need team to review and classify label Jul 23, 2021
@beckernick beckernick added this to the cuDF Python Refactoring milestone Jul 23, 2021
@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@vyasr
Copy link
Contributor

vyasr commented May 13, 2024

@mroeschke @brandon-b-miller this issue is something to keep in mind with the ongoing refactors to as_column and adding column factories for pylibcudf.

@mroeschke mroeschke mentioned this issue May 13, 2024
Merged
3 tasks
rapids-bot bot pushed a commit that referenced this issue May 14, 2024
@mroeschke
Validate and materialize iterators earlier in as_column (#15739)
4069c82 
closes #8796

I left a `TODO` in `as_column` to validate earlier that `arbitrary` is an iterable or sequence like if it wasn't a recognized array like (e.g. numpy array, pandas object, etc). Additionally, ensure we materialize iterators since there are some checks that would exhaust the object

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #15739
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cuDF (Python) Affects Python cuDF API. feature request New feature or request
Projects
No open projects
Feature Planning
Needs prioritizing
Milestone
cuDF Python Refactoring
Development

Successfully merging a pull request may close this issue.

Validate and materialize iterators earlier in as_column mroeschke/cudf
4 participants
@vyasr @beckernick @galipremsagar @GregoryKimball