Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RLS: pandas 1.5.3 #49857

Closed
2 tasks done
datapythonista opened this issue Nov 23, 2022 · 25 comments
Closed
2 tasks done

RLS: pandas 1.5.3 #49857

datapythonista opened this issue Nov 23, 2022 · 25 comments
Labels

Comments

@datapythonista
Copy link
Member

datapythonista commented Nov 23, 2022

Will be releasing when the blockers are addressed:

@datapythonista
Copy link
Member Author

Seems like there is not much added to the 1.5.3 milestone, even if we skipped the December release.

Maybe we can release 1.5.3 after the dev call next week as usual, with what's merged, and anything you want to see in 1.5, and plan the release of 2.0 RC for February?

CC: @pandas-dev/pandas-core

@MarcoGorelli
Copy link
Member

sounds good!

@Dr-Irv
Copy link
Contributor

Dr-Irv commented Jan 3, 2023

seems like there is not much added to the 1.5.3 milestone, even if we skipped the December release.

Not much? If I go to https://github.com/pandas-dev/pandas/milestone/100 it shows 17 open issues and 55 closed ones!

@datapythonista
Copy link
Member Author

Not much? If I go to https://github.com/pandas-dev/pandas/milestone/100 it shows 17 open issues and 55 closed ones!

My bad, you're right. I guess I had some extra filter when I checked, that's not what I saw. Probably worth continuing to 1.5.4 and possibly 1.5.5 for now then, and aim to releasing 2.0 around March or April.

@phofl
Copy link
Member

phofl commented Jan 4, 2023

I‘d prefer doing the rc for 2.0 in February anyway (if we get everything in by then). We can still work on 1.5.4 if necessary

@MarcoGorelli
Copy link
Member

Yeah TBH I wouldn't block 2.0, it's been 2 years already and none of the regressions look that crucial to me (though I may be wrong), if they don't get fixed by 2.0 they'll get fixed later

@datapythonista
Copy link
Member Author

We always released sequentially until now. I think the releases in the home page, and not sure if something else, will need to be changed to show the latest version as 2.x instead of 1.5.x if 1.5.x is released later. But probably worth doing it anyway, as I guess we'll eventually have a patch release after a major or minor.

Let's release independently then. I'll be releasing 1.5.3 after the dev call, as we've been doing. And we can release 2.0 as soon as it's ready. I see we've got things in the milestone that are probably not blockers. I guess the main things are enforcing deprecations and the copy-on-write stuff, no? Maybe we can add to #46776 the inventory of blocker issues and PRs, so we can better track the status. Or maybe remove from the milestone everything that is not a blocker?

@datapythonista
Copy link
Member Author

I added a list of blockers for 2.0 in #46776. Seems like there is consensus to not delay it until the 1.5.x series is over, and release it once the blockers are addressed.

For 1.5.3, is there any blocker? Assuming there will be at least a 1.5.4? I'm happy to release it early next week if there is nothing in particular to wait for. And then we can focus on 2.0 in the dev call.

@Dr-Irv
Copy link
Contributor

Dr-Irv commented Jan 6, 2023

Not sure if #50558 is considered a blocker or not since we're no longer compatible with the latest SQLAlchemy
Also #50342 reveals an issue with numpy 1.24.0. Again, not sure if this is a blocker or not.

@cdcadman
Copy link
Contributor

cdcadman commented Jan 8, 2023

I don't know if it's possible or not, but I'd be delighted if pandas 1.5.3 could support sqlalchemy 2.0. I have an open PR to get the main branch working with sqlalchemy 2.0 (#48576).

@phofl
Copy link
Member

phofl commented Jan 8, 2023

Hm we are only back porting regression fixes in general. The Alchemy changes are too substantial to backport I think

@MarcoGorelli
Copy link
Member

MarcoGorelli commented Jan 8, 2023

if you need this, you should be able to install the nightly pandas wheel, see https://anaconda.org/scipy-wheels-nightly/pandas (and that'd generally be really useful to help pandas catch bugs before the final 2.0 release)

@lithomas1
Copy link
Member

I'm +0 on backporting.
Since pandas 2.0 is breaking, some users might be stuck on older pandas (not necessarily by their choice, since some downstream libraries may pin to less than 2.0) for a while, so I would be fine backporting as long as the API is not changed.

@phofl
Copy link
Member

phofl commented Jan 8, 2023

I am more weary of introducing new bugs for users that can’t upgrade. Alchemy is a major breaking change with 2.0, so you have to put a lot of effort in to upgrade it as well

@jreback
Copy link
Contributor

jreback commented Jan 8, 2023

yeah i wouldn't put any effort to backport in sqlalchemy support
the upgrade path is generally going to be long because sqlalchemy is changing a lot

pandas 2 will be out for quite a long time before this is actually a problem of supporting sqlalchemy 2.0; we r breaking a relatively small amount of things

so -1 on backporting as this introduces a large risk

@Dr-Irv
Copy link
Contributor

Dr-Irv commented Jan 9, 2023

pandas 2 will be out for quite a long time before this is actually a problem of supporting sqlalchemy 2.0; we r breaking a relatively small amount of things

so -1 on backporting as this introduces a large risk

Agree, but we may still want to address #50558 because SQLAlchemy 1.4.46 issues a deprecation warning, whereas 1.4.45 doesn't. Not sure if there are a minimal set of changes to avoid that warning.

@mroeschke
Copy link
Member

mroeschke commented Jan 13, 2023

As discussed on the 2023-01-11 dev call. these were the items blocking the release 1.5.3

Anything else? cc @pandas-dev/pandas-core

@rhshadrach
Copy link
Member

Would be good to get #50627 in, but certainly not a blocker.

@datapythonista
Copy link
Member Author

Working on a fix for the CI problems with the website github api quota (which seem to be even more frequent now). Will start the release after that.

@datapythonista
Copy link
Member Author

I created the issue (#50814) and milestone for 1.5.4 and rolled everything there, except the backport of the clean up of the whatsnew notes (#50813), and the fix to the github quota issues (#50811), which I'd like to get in before the release. We've got the sdist build failing, but seems like a problem in conda, and it's not immediate to know what's causing it, I think we can go ahead anyway. We can also pin, or try the fix I propose in #50779 (comment).

If there are any comments please let me know asap, otherwise I'll be merging the two pending PRs when CI is ready, and I'll move forward with the release.

@pandas-dev/pandas-core please do not merge anything on the 1.5.4 milestone before the release without letting me know first. I think it's unlikely there is anything to merge, but just in case.

@datapythonista
Copy link
Member Author

Tagging the release.

@datapythonista
Copy link
Member Author

Created:

Will merge both PRs and finish the release once their CI is green.

@datapythonista
Copy link
Member Author

Wheels built and uploaded to PyPI. pip install pandas is now installing 1.5.3, and seems to work fine. Docs have been updated to the latest version.

Conda-forge is taking a long time to open the automatic PR. I'll wait a bit more, and if it's still not generated I'll open the PR manually. Once the conda-forge packages are ready, only announcing the new release is missing.

@datapythonista
Copy link
Member Author

Opened the PR for conda-forge manually: conda-forge/pandas-feedstock#153

@datapythonista
Copy link
Member Author

Release complete. Some of the conda-forge builds take 3 hours, so it's 6 hours to check if the CI is green and get the packages. But 1.5.3 is now available in conda-forge, I tested the packages and seem to be working well (at least in my platform).

I have problems logging into twitter and mastodon, @MarcoGorelli will be posting about the release there. It's been announced everywhere else. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

9 participants