Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: fix parquet test_pandas_timestamp_overflow_pyarrow test #8733

Merged
merged 5 commits into from Feb 22, 2022

Conversation

jorisvandenbossche
Copy link
Member

This test started failing on arrow master (noticed thanks to our nightly builds testing dask). The reason for the failure is that we started using the types_mapper keyword in pa.Array.to_pandas (previously this was ignored for arrays), but in this test it is passed a wrong values (it should be a callable, not a dict). Now, I think the argument can also just be removed.

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jorisvandenbossche! Would you mind pushing an empty commit with test-upstream in it? That will trigger a CI build which uses the nightly version of pyarrow here

@jorisvandenbossche
Copy link
Member Author

The current upstream build is not actually using pyarrow dev (for some reason mamba is not selecting the nightly version, I looked at this before in the past (#8281), but not fully sure what's the reason for that)

@jorisvandenbossche
Copy link
Member Author

Hmm, it seems we only have python 3.7 and python 3.10 packages in the nightlies, that is something to fix on the pyarrow side.

@jsignell
Copy link
Member

We are also planning on adding Python 3.10 support to dask pretty soon, so at that point we will be able to update the upstream tests to use that. In the meantime I am going to merge this since it seems right from your explanation @jorisvandenbossche

@jsignell jsignell merged commit abd145e into dask:main Feb 22, 2022
@jorisvandenbossche jorisvandenbossche deleted the arrow-test branch February 23, 2022 07:25
@jorisvandenbossche
Copy link
Member Author

Thanks. I am also working on getting the nightly packages back up for all python versions (apache/arrow#12492), once that is ready I will open a PR here to ensure we are using the latest pyarrow nightly in the upstream build.

@jrbourbeau
Copy link
Member

Awesome, thanks as always @jorisvandenbossche

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants