Plan for 10.0

[ghost]

It's clear that PEP 518 support won't be implemented anytime soon. In addition, some maintainers mentioned that master was not in a releasable state. In order to allow a release, I propose that PEP 518 support be reverted completely and both PEP 518 and 517 be deferred indefinitely. I can submit a PR to remove PEP 518 if needed.

[pfmoore]

Strong -1 on this. If pip isn't going to implement PEP 517/518, then those PEPs are essentially dead for the foreseeable future. I don't think that is helpful. It also doesn't send a particularly good message to the users and developers of flit, that interoperability with pip is going to be possible in a reasonable timescale (and personally, I very strongly want to work towards a situation where pip and flit work together as well as pip and setuptools do currently).

I don't think it's likely that a new PR to remove PEP 518 is going to be helpful. Even if that were the route we wanted to take (and I don't think it is) It's not going to be merged unless it's clear that all it does is revert the changes made to implement PEP 518 - and who's going to be able to review it to confirm that? Personally, I'd rather accept that PEP 518 support is incomplete for now and go with what we have - but we've not really managed to achieve any consensus on that.

Github issues haven't proved particularly helpful as a way of co-ordinating the work needed to handle PEPs 517 and 518. It seems like it's too easy for discussions to fragment into multiple issues/PRs. I don't have a solution for this.

One thing that might be worth considering is that the big issue here is not actually PEP 517/518. Rather, it's the need to add a new concept into pip - that of having a "build environment" that provides all of the prerequisites to build a package, but which is different from the installation environment where the package will be installed (in the sense that if building needs xxx and it's not currently installed, we need to temporarily make xxx available for the install without permanently adding it to the user's environment). It's actually that refactoring of pip's design/internals that is the hard part here. But it needs to be done. Rather than trying to make a choice between a "big bang" change to get everything right in one huge PR, or doing nothing, I still think (as I've been saying from the start) that we need to focus on smaller, incremental PRs that can be reviewed and merged one at a time. That's where we are now - the initial PEP 518 work implements partial "build isolation" - we need to incrementally improve that rather than declaring it a failure and ripping it out in favour of some purely hypothetical (and likely never going to happen) rewrite that implements the perfect solution in one go.

I can't help feeling that the PEP process is a better way of thrashing out the sorts of design issues we're facing here than github issue comments. But I don't know how we'd capture the current problem in a PEP-able format. Nor do I know if distutils-sig has any appetite for another PEP to add to the queue of unimplemented ideas :-(

[pfmoore]

By the way - the only actual constraint on releasing pip 10 is that we have addressed any release blockers. Currently, the only release blockers for pip 10.0 are these: #4647 and #4803

One is simply saying we need to document our PEP 518 support (and any limitations it may have) and the other is reporting that the current code tries to run setup.py egg_info before installing the requirements defined in pyproject.toml. We could simply document that at the moment, pip only guarantees that PEP 518 requirements are available when setup.py bdist_wheel gets run, but not when setup.py egg_info is run. It may not meet the spirit of PEP 518, and it may mean that we need to describe what we have as "first steps towards PEP 518 support" rather than actual support - but if the goal is to release pip 10, that's fine. There's no need to hold a pip 10 release hostage to a perfect solution to the PEP 518 issues.

Heck, we could simply downgrade those two issues to no longer be release blockers.

[ghost]

This comment is completely divorced from reality. The existing PEPs fully and adequately specify the behavior that pip must implement, which it currently does not. In addition, as I have previously shown, the support that pip does implement is not helpful to SciPy, scikit-build, or pretty much all of the other scientific projects. The remarks about the difficulty of implementation are not really an adequate description of the situation either: I am able to write a pip that does support these fully, but the cost of a fork is high -- especially with pip, and I have no interest in constantly having to merge changes from upstream.

[ghost]

Currently, the only release blockers for pip 10.0 are these: #4647 and #4803

Are you absolutely sure that a situation where pip can fork-bomb a system is not a release blocker?

[pradyunsg]

TBH, I've had this idea of removing PEP 518 code (pradyunsg@6ff51ba) but I agree with Paul here that the ramifications of doing that are almost equivalent to killing PEP 517/518.

Are you absolutely sure that a situation where pip can fork-bomb a system is not a release blocker?

Could you make a new issue for it, with an example/description of how to cause the current master to do this? If nothing else, the only place it has been mentioned was in the depths of another issue and I personally still don't really understand the edge case well enough.

It's actually that refactoring of pip's design/internals that is the hard part here.

+1

I have an idea here that helps out with even the dependency resolution situation but I'd rather not digress right now.

There's no need to hold a pip 10 release hostage to a perfect solution to the PEP 518 issues.

+1

and who's going to be able to review it to confirm that?

I can, but I genuinely don't think that's enough. :)

cost of a fork is high

Yes. Please, don't do that.

Likely, a better approach is to break up #4799 into smaller bite-sized chunks that can be processed in the trickle of dev-time that's available. I believe @xoviat has stated that he has completed a wheel-build-dep only variant of PEP 518 there (or something like that).

If we somehow don't resolve this until April, I'll probably have the bandwidth to be able to take this on.

can't help feeling that the PEP process is a better way.

I actually have a draft that I'll put in a Gist when I have the time to finish it, regarding what can be done for some of the refactoring work. If someone wants, I'll pull it higher in my queue.

---

All that said, the closest I could come up with some acceptable strategy, assuming removing PEP 518 code now, would be:

• remove PEP 518 stuff
• release 10.0
• refactor refactor refactor
• implement PEP 518 + 517
• release 11.0

I won't mind this if somehow we can ensure that 3 and 4 happen in a reasonable time frame or someone shows that what we have now is so bad that redoing it would be less work.

[ghost]

Likely, a better approach is to break up #4799 into smaller bite-sized chunks that can be processed in the trickle of dev-time that's available. I believe @xoviat has stated that he has completed a wheel-build-dep only variant of PEP 518 there (or something like that).

I do not have the bandwidth to constantly fight the maintainers to merge my changes. I still have not heard that there would be a good-faith commitment to review and merge them ("agree with my changes at a high level").

[pradyunsg]

Lacking a better place to say this: I really think we should have a beta release, whenever we start the process.

[pfmoore]

I'm OK with that. I'm still quite happy to do the release - but I'm not going to try to arbitrate over what goes in, what doesn't and how we resolve the differences.

What I'd like is:

1. Confirmation from @pypa/pip-committers that they feel we're in a position to go ahead with a release.
2. No open release blocker issues. I'll deal with Document the extent of our PEP 518 support #4803, but someone needs to sort out PEP 518 support: setup.py is called before build requirements have been installed #4647. Either remove the release blocker tag, or resolve it.
3. A lack of complaints from the general community that "this is not in a state that we can release". Normally, we wouldn't insist on this, but for this specific release only, and in relation to what we have in place for PEP 518 specifically, I'd like to ensure we're not missing any significant concerns from users. If concerns do arise, I'd expect them to be expressed in the form of issues with reproducible test cases, that are severe enough to warrant a release blocker tag, so this basically is a subset of (2). Thanks to @xoviat and @pradyunsg for raising an issue for the forkbomb problem, and producing an acceptable (short-term) fix. Any remaining issues like this?

The reasons I'd like to see pip 10 out of the door:

1. There are significant encoding issues on Windows that are hitting our users in non-ASCII locales. We've had a fix in master for a while now, and it's hurting our users not to have it released. I gather that the issue causes upgrading to fail, so such users are stuck. Ideally, we should get a version with the fix in released soon enough to go into the next releases of Python 2.7 and 3.6 (and 3.7).
2. I'd like to get the switch to the _internal namespace done with. In my view, anyone who is going to move away from using pip's internal functionality as a result of our pre-announcement will probably have done so. We need to get on with the release to maintain the credibility of that announcement.

(1) is important enough that if pip 10 is significantly delayed, I think we should consider backporting the fix to a 9.0.x patch release. I don't really want to do that myself, though, as our release process isn't set up for patch releases and I don't want to be the one to design that new process.

[ghost]

No open release blocker issues. I'll deal with #4803, but someone needs to sort out #4647. Either remove the release blocker tag, or resolve it.

The next step is approving gh-4990 and gh-4991. The speed at which this is resolved depends only on how quickly PRs are approved.

[ghost]

Also, as soon as those PRs plus gh-4987 is merged, I can rebase my branch and it should resolve the issue completely. We can then proceed directly to PEP 517.

[pfmoore]

The next step is approving gh-4990 and gh-4991.

Both approved (by me, at least). I'll leave it to @pradyunsg to decide if he's OK merging the 3 PRs you mention, or if he wants additional reviews.

I can rebase my branch and it should resolve the issue completely.

Maybe, but is your branch now small and self-contained enough to review? Maybe @pradyunsg is OK with reviewing and merging it. I don't have a feel for whether I could review it, it's some time since I looked at it. I'm willing to take another look, but I reserve the right to still ask for clarification (if you can't explain things to the extent I'm need, I'm fine with simply deferring to others - but I'd prefer to help review as I know we have few committers available to work on this).

We can then proceed directly to PEP 517.

I'm not interested in PEP 517 for pip 10. In fact, I'd prefer to ignore it completely to be sure we just get PEP 518 dealt with, without confusion and side issues. If your branch fixes #4647 at the cost of opening up a potential source of new issues in PEP 517 support (I've no idea how extensive the tests are in your branch), then I'd honestly rather look for an alternative short-term fix (even if that's simply downgrading #4647 to not a release blocker and fixing it via your branch early in the pip 11 cycle).

[ghost]

Maybe, but is your branch now small and self-contained enough to review?

It will be once those are merged. IIRC, the only blocker was that you objected to gh-4800. Perhaps the diff is not equivalent, but gh-4987 is functionally gh-4800, only that PR was blocked.

[ghost]

If your branch fixes #4647 at the cost of opening up a potential source of new issues in PEP 517 support

gh-4799 will, when rebased, resolve problems that are only related to PEP 518. Once that is merged, I will rebase gh-4589, which will partially implement PEP 517.

[ghost]

Okay, I've rebased gh-4799 on top of gh-4987 and will rebase again on master once gh-4987 is merged; then it should be ready to review. Note that we may need a regression test to demonstrate that gh-4799 fixes gh-4647.

[ghost]

After gh-4799 is merged, we should IMO have a release candidate and then start trying to find bugs.

[alex]

Is there a plan for what the process from RC to release is?

(My particular interest is in the urllib3 improvements to use SecureTransport, which helps resolve the need to have all PyPI consumers use TLS1.2 by June)

[ghost]

With NumPy and SciPy, we release, tell people to test it out, and then if no bugs are found in a couple weeks, that's the release that goes out. If bugs are found, the process repeats.

[pfmoore]

Basically what @xoviat says. I'd call it a beta rather than a RC, but that's just terminology. In the past, we've had limited success getting people to test pre-releases, But frankly, there's not much else we can do, so we'll have to work with what we've got.

The big question is how well we can support freezing master for the testing period. That'll probably be what dictates the length of that period. I'm not keen on managing bugfixes across a release and a development branch for the first time on this release. It's probably something we should consider introducing longer-term, but likely not until we're a bit less critically low on resources :-)

(Disclaimer: This is only an "official" plan to the extent that I've offered to do the release. If one of the other @pypa/pip-committers has other ideas and wants to help out or take over, that's fine with me).

[pradyunsg]

I also think 2 week period should be fine. I imagine we can hold-off any non-bugfix PRs for that long. If it gets longer, then we'll have to take a look at things.

@pfmoore I'd also appreciate it if you could take the time to review #4799 (did I get the number right?).

I'm happy to help out with the release. =)

[pfmoore]

@pradyunsg Sorry, I can't really review #4799 in the form it's currently in, and I don't want to cause any additional frustration to @xoviat (or myself ;-)) by pushing for yet more explanations of what's going on.

If I get time to research the underlying code to a point where I follow the PR, I'll come back to it, but until then I'll defer to the people who know this part of the code.

[pfmoore]

@xoviat appears to have closed #4799. Where does that leave us with PEP 518 support and a pip 10 release? Are the parties with concerns about the current state of support for PEP 518 willing to accept a release with what we currently have in master? If not, how do we unblock things?

Options seem to be:

1. Reopen Install build requirements before calling setup.py #4799 (@pradyunsg had promised to merge it, I don't know where that promise stands now).
2. Accept master as fit for release as is.
3. Revert PEP 518 support completely.

(1) relies on @xoviat and @pradyunsg working together to resurrect #4799. (2) relies on @benoit-pierre and the other people concerned about current support for PEP 518 confirming that they are OK with accepting it. (3) needs a PR.

The final option, which I'm reluctant to take, is for PyPA to unilaterally declare we just release master as pip 10. Before doing that, I'd want explicit OKs from @dstufft, @pradyunsg and @xavfernandez, but if they agree, I'm willing to do the release.

(Note for the future - we need to seriously consider having a maintenance branch. But I don't know if that's even possible with our current level of manpower :-()

[pradyunsg]

I'll be happy to resurrect #4799 on my own. I'm just a little surprised that it was closed. I'd marked 1st March as the day to merge that PR in my calendar, so what's happened is funny to me at some level. I think reverting is reasonable, mostly because I'm not aware of discussions about how to introduce build isolation to pip and implementation of the same. I'm strongly in favour of a maintenance branch and for keeping things proper with that.

…

On Thu, 1 Mar 2018, 04:27 Paul Moore, ***@***.***> wrote: @xoviat <https://github.com/xoviat> appears to have closed #4799 <#4799>. Where does that leave us with PEP 518 support and a pip 10 release? Are the parties with concerns about the current state of support for PEP 518 willing to accept a release with what we currently have in master? If not, how do we unblock things? Options seem to be: 1. Reopen #4799 <#4799> ***@***.*** <https://github.com/pradyunsg> had promised to merge it, I don't know where that promise stands now). 2. Accept master as fit for release as is. 3. Revert PEP 518 support completely. (1) relies on @xoviat <https://github.com/xoviat> and @pradyunsg <https://github.com/pradyunsg> working together to resurrect #4799 <#4799>. (2) relies on @benoit-pierre <https://github.com/benoit-pierre> and the other people concerned about current support for PEP 518 confirming that they are OK with accepting it. (3) needs a PR. The final option, which I'm reluctant to take, is for PyPA to unilaterally declare we just release master as pip 10. Before doing that, I'd want explicit OKs from @dstufft <https://github.com/dstufft>, @pradyunsg <https://github.com/pradyunsg> and @xavfernandez <https://github.com/xavfernandez>, but if they agree, I'm willing to do the release. (Note for the future - we need to seriously consider having a maintenance branch. But I don't know if that's even possible with our current level of manpower :-() — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7SX_SX_DSOUPMadv2XYXyjb-e52hOks5tZdnagaJpZM4Rlmgm> .

[pradyunsg]

Cool, so #4799 is good to go IMO. I'll merge it once CI is happy.

The roadmap for pip 10.0, as I understand, would be:

• Triage Issues/PRs -- I'll take out some time and do this in the coming exam week. :P
• Important PRs: Improve message when setuptools is missing (PEP 518) #4989, Enable vendor C libraries on non-Windows platforms #4835
• Important Issues: Document the extent of our PEP 518 support #4803
• Nice-to-have PRs: Add a verbosity attribute to basecommand #4977, Reduce action at distance #4978, Move dependency info and install order into Resolver #4925, Warn about broken dependencies during pip install #5000, Bugfix: Fixes #3763, #3830, and #4453 #5010, Vendor Update - stdout and stderr of shell should be decoded with system encoding #4932, Add auto completion of short options #4956, Use log level info when ignoring packages due to environment markers #4877
• A beta release
  • Publicity saying -- lots of changes made, please try it out?
• 2 week feedback period
• Main Release

[pfmoore]

Nice! Thanks for putting that list together. I'll try to find some time over the next week or so to review them, and I'll get up to speed on the release process.

[pradyunsg]

I'll add all linked issues to the 10.0 milestone.

[ghost]

gh-5033 is a new release blocker that I recently discovered. conda-forge will be broken without it.

[pradyunsg]

I think we're super close to beta now. IMO, the state of everything in the milestone for 10.0 is:

• Waiting on CI (Warn about broken dependencies during pip install #5000, Cleanup some author names #5118)
• Waiting on review, can defer till 10.0 final (Enable vendor C libraries on non-Windows platforms #4835)
• Waiting on author, can defer till 10.0 final (Fixes 4893 - error handling for malformed configurations #5026)
• Deferred until after 10.0 final (Require minimum pip version? #4145, uninstall -nspkg.pth files installed with setup.py develop #4176, DistInfoDistribution._compute_dependencies fails if the package has extras_require that are PEP440 versions. #4356)

@pfmoore Do you agree?

[pfmoore]

I'll be cutting the beta this weekend, so yeah, we're that close.

• Warn about broken dependencies during pip install #5000 and Cleanup some author names #5118 Agreed, if you're happy to merge once CI has passed that's fine.
• Enable vendor C libraries on non-Windows platforms #4835 and Fixes 4893 - error handling for malformed configurations #5026 Sorry, but if these don't make the beta, I'd rather defer them till after 10.0. To be clear, I want to have a change freeze between 10.0 beta and 10.0 final - we've put enough effort into asking people to test the beta that I don't feel it's fair to then add functional changes once the beta has been released.
• Specifically Enable vendor C libraries on non-Windows platforms #4835 I won't have time to review (especially as it's non-Windows) so I'm expecting that to have to wait till post-10.0 now.
• If you're really keen on merging Fixes 4893 - error handling for malformed configurations #5026 after the author responds, you have 24 hours left. But honestly, I feel like it's being rushed to hit the deadline, and I'd be inclined to defer it to post-10.0.
• The deferred ones, I agree.

Basically, we should be slowing down now - I'm seeing a rush to get the "last few things in". While I can understand that if we were to have a long delay after pip 10 before the next release, I'm hoping (and willing to commit effort to ensuring) that we can get back to a better release pace after pip 10 (3-6 months between releases). So the rush is somewhat artificial.

PS Sorry, I did want to put up a reminder that I was intending to freeze after the beta release (it has been mentioned elsewhere). But I couldn't think of a good place to post it - I'd forgotten we had this issue which would have been ideal.

[pradyunsg]

Sounds good to me. Deferring those to after 10.0 is also cool with me. :)

…

On Fri, 30 Mar 2018, 14:39 Paul Moore, ***@***.***> wrote: I'll be cutting the beta this weekend, so yeah, we're that close. - #5000 <#5000> and #5118 <#5118> Agreed, if you're happy to merge once CI has passed that's fine. - #4835 <#4835> and #5026 <#5026> Sorry, but if these don't make the beta, I'd rather defer them till *after* 10.0. To be clear, I want to have a change freeze between 10.0 beta and 10.0 final - we've put enough effort into asking people to test the beta that I don't feel it's fair to then add functional changes once the beta has been released. - Specifically #4835 <#4835> I won't have time to review (especially as it's non-Windows) so I'm expecting that to have to wait till post-10.0 now. - If you're really keen on merging #5026 <#5026> after the author responds, you have 24 hours left. But honestly, I feel like it's being rushed to hit the deadline, and I'd be inclined to defer it to post-10.0. - The deferred ones, I agree. Basically, we should be slowing down now - I'm seeing a rush to get the "last few things in". While I can understand that if we were to have a long delay after pip 10 before the next release, I'm hoping (and willing to commit effort to ensuring) that we can get back to a better release pace after pip 10 (3-6 months between releases). So the rush is somewhat artificial. PS Sorry, I did want to put up a reminder that I was intending to freeze after the beta release (it has been mentioned elsewhere). But I couldn't think of a good place to post it - I'd forgotten we had this issue which would have been ideal. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7Sb4Ri-qHTZAe0VbB5Eh8gDY4NH6Pks5tjfZTgaJpZM4Rlmgm> .

[dstufft]

@pfmoore We haven't done beta releases for awhile, but feel free to make a release branch for 10 for now if you want, once final is out we can merge it back into master and delete it.

[pfmoore]

Thanks, I'll do that - although my git skills may not be up to merging it back so I may have to ask for assistance later over that :-)

[pfmoore]

One other thing that I wasn't sure about - is there anything we need to do to promote the current docs from "latest" to "stable" on https://pip.pypa.io, or does that happen automatically when we release? We probably shouldn't do that until the release, although part of me wants to just to improve visibility of the new docs.

[dstufft]

10.0.0b1 isn't a stable release, so it doesn't get promoted automatically, but basically the way RTD is setup is:

• latest always points to whatever is on the master branch.
• stable looks at the tags on pypa/pip, finds any that looks like a version, sorts them, and uses the docs for the latest non-prerelease one of them.

So once 10.0.0 is tagged, it should get promoted to stable.

[pfmoore]

Not sure where else to put this, but just as a reminder, I'll be creating 10.0 final this weekend (likely Saturday morning). We currently have no release blockers or outstanding issues/PRs on the 10.0 milestone, so we're in pretty good shape, it seems. Thanks to everyone who's helped get us to this point.

I'd prefer to have a final moment of calm before the release, rather than a last minute rush, so here's hoping for no surprises!!!

The new PyPI goes live on Monday, AIUI, so it should be a fun weekend!

[cjerdonek]

Is it wise or necessary to have these two changes (10.0 and PyPI) go out so close to each other e.g. since it could complicate troubleshooting or compound problems?

…

On Thu, Apr 12, 2018 at 11:24 AM Paul Moore ***@***.***> wrote: Not sure where else to put this, but just as a reminder, I'll be creating 10.0 final this weekend (likely Saturday morning). We currently have no release blockers or outstanding issues/PRs on the 10.0 milestone, so we're in pretty good shape, it seems. Thanks to everyone who's helped get us to this point. I'd prefer to have a final moment of calm before the release, rather than a last minute rush, so here's hoping for no surprises!!! The new PyPI goes live on Monday, AIUI, so it should be a fun weekend! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVt7msF27wpVHD80JPI4MdvB9VJj_Bqks5tn5vzgaJpZM4Rlmgm> .

[dstufft]

We've had a lot of testing of both of them, so it should be fine. Warehouse is time constrained due to the MOSS award coming to an end and pip is limited by time and the fact we've already made announcements. We also don't want to release pip 10 or PyPI too closely to PyCon, to give it time to shake any issues out before then.

[sbidoul]

I reported #5193 against 10.0.0beta.

Have you seen it? It looks like a regression.

[pfmoore]

@sbidoul Sorry about that, it looks like none of the other developers have picked up on it (or possibly they have, but don't consider it a blocker) - IIRC, I saw it, but it's outside my area of expertise, so I can't comment myself.

I'm going to assume it's relatively low-impact, as no-one else has reported anything similar, and so not make it a release blocker - but I'm open to releasing a quick 10.0.1 fix if one of the other @pypa/pip-committers feels it's worth doing so and provides a fix.

[pfmoore]

Looks like there will be a 10.0.1, as @vsajip is in the process of putting together a distlib fix for #5223. I'd like to get it out ASAP, as it's affecting a lot of Windows users. So my plan is that as soon as the distlib release is ready, I'll revendor and release 10.0.1 from master. Assuming things go to plan, that'll be before next weekend (I don't have much time this coming weekend, so a midweek release suits my availability better).

If there's anything else that's critical for a 10.0.1, it needs to be completed and put into master ASAP - but please do not rush non-critical changes into master. I'd like to avoid needing to create a branch, but I also don't want to destabilise 10.0.1.

Issues marked as blockers for 10.0.1:

• "Display a message on Windows if pip is being modified" is triggered by get-pip.py #5219 breaks normal usage of get-pip.py for Windows users. @pradyunsg I'm going to merge Make the Windows "self-modification" check more strict #5233 unless you want to do anything more on it (or merge it yourself).
• Build dependencies don't appear to support environment markers #5230 environment markers in build dependencies has a PR Fix pep518 support: support markers in build requirements #5245. I don't feel like I know enough to review this - @pradyunsg can you check it over and approve/merge? If you're not happy with it, I suggest we postpone it till 10.1.
• pip 10.0.0 fails silently if it cannot install due to missing write permissions #5237 is "only" a missing explanation of an error. There's a PR fixing this at Improve Error Messages on EnvironmentErrors when installing #5239. I'm OK with this going in, but I'm wary of it taking ages to debate the precise wording of the message. At the moment, it's waiting on the user confirming the PR looks OK, but I think we shouldn't wait too long on that. Frankly, I don't think this counts as a blocker but I don't mind it going in.
• There's also Switch pip to use Warehouse by default #5215 which is not a blocker but which I think should go in if @dstufft can get it merged.

The only other issue targeted for the 10.0.1 milestone is #5231. It's a bug in a vendored dependency - they have a fix, but it's not clear if they are going to do a release. If someone wants to push a release through in the next day or two, or prepare and merge a PR putting the fix into our vendored version, then that's OK with me, but otherwise it'll miss 10.0.1.

Apart from all that, please can people not merge stuff to master if there's a chance it could destabilise 10.0.1? Thanks.

[benoit-pierre]

What about #5235/#5227?

[cjerdonek]

Wasn’t there already a branch for 10.x before? You could have kept using that.

…

On Mon, Apr 16, 2018 at 8:16 AM Paul Moore ***@***.***> wrote: Looks like there will be a 10.0.1, as @vsajip <https://github.com/vsajip> is in the process of putting together a distlib fix for #5223 <#5223>. I'd like to get it out ASAP, as it's affecting a lot of Windows users. So my plan is that as soon as the distlib release is ready, I'll revendor and release 10.0.1 from master. Assuming things go to plan, that'll be before next weekend (I don't have much time this coming weekend, so a midweek release suits my availability better). If there's anything else that's critical for a 10.0.1, it needs to be completed and put into master ASAP - but please do *not* rush non-critical changes into master. I'd like to avoid needing to create a branch, but I also don't want to destabilise 10.0.1. Issues marked as blockers for 10.0.1: - #5219 <#5219> breaks normal usage of get-pip.py for Windows users. @pradyunsg <https://github.com/pradyunsg> I'm going to merge #5233 <#5233> unless you want to do anything more on it (or merge it yourself). - #5230 <#5230> environment markers in build dependencies has a PR #5245 <#5245>. I don't feel like I know enough to review this - @pradyunsg <https://github.com/pradyunsg> can you check it over and approve/merge? If you're not happy with it, I suggest we postpone it till 10.1. - #5237 <#5237> is "only" a missing explanation of an error. There's a PR fixing this at #5239 <#5239>. I'm OK with this going in, but I'm wary of it taking ages to debate the precise wording of the message. At the moment, it's waiting on the user confirming the PR looks OK, but I think we shouldn't wait too long on that. Frankly, I don't think this counts as a blocker but I don't mind it going in. - There's also #5215 <#5215> which is not a blocker but which I think should go in if @dstufft <https://github.com/dstufft> can get it merged. The only other issue targeted for the 10.0.1 milestone is #5231 <#5231>. It's a bug in a vendored dependency - they have a fix, but it's not clear if they are going to do a release. If someone wants to push a release through in the next day or two, or prepare and merge a PR putting the fix into our vendored version, then that's OK with me, but otherwise it'll miss 10.0.1. Apart from all that, please can people not merge stuff to master if there's a chance it could destabilise 10.0.1? Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVt7hh3rVzy5gGB6fF3NU6VfLUGI5s-ks5tpLXZgaJpZM4Rlmgm> .

[pfmoore]

What about #5235/#5227?

That's a testsuite fix, with no user impact, isn't it? No need for it to be in 10.0.1 in that case.

Wasn’t there already a branch for 10.x before?

Yes, but as per our processes, it was merged into master when we released. We don't maintain separate release and development branches.

[dstufft]

The typical process is we would branch off of the tag, and backport fixes, but since master is already in a reasonable state and nothing else has landed, it's easier to just release off master again, as long as nobody merges something that would prevent that.

[pfmoore]

@dstufft yeah, I'm cheating and using the "full release" process (from https://pip.pypa.io/en/latest/development/#release-process for interested watchers) rather than the "bugfix release" process, precisely because master's still clean. I'm not confident enough of my git skills to backport individual fixes if I don't need to ;-)

[cjerdonek]

Yes, but the suggestion being considered is to tell people in advance not to commit to master, not to do what you’re suggesting in case someone does. (By the way, note that merging a branch doesn’t imply / necessitate deleting it.)

…

On Mon, Apr 16, 2018 at 1:35 PM Donald Stufft ***@***.***> wrote: The typical process is we would branch off of the tag, and backport fixes, but since master is already in a reasonable state and nothing else has landed, it's easier to just release off master again, as long as nobody merges something that would prevent that. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVt7sVs3KrNJzWSnMXZC6J6a6qxZY8nks5tpQCRgaJpZM4Rlmgm> .

[benoit-pierre]

#5235 is necessary for the tests in #5227 to pass.

[pfmoore]

@cjerdonek For a couple of days, and there's only 2 other active committers, both of whom know the situation here. Let's not over-engineer things :-)

[pfmoore]

@benoit-pierre Oh, sorry - I missed that #5227 was the user-facing one. I've merged #5235, so let's see how the tests for #5227 fare after that. There were also some review comments from @pradyunsg - I'll let him follow up on that and merge when he's happy. I've marked #5224 as a blocker so I don't forget this again.

[benoit-pierre]

@pfmoore: thanks.

[pfmoore]

Bah. The build for #5235 failed - looks like our tests had dependencies on the old PyPI (the errors look to be related to the URLs where distributions are hosted). @dstufft can you check what's going on here?

[dstufft]

I just merged #5215, which should fix those errors.

[pfmoore]

@dstufft cool, I wasn't sure if that would be enough. Thanks.

[dstufft]

I think #5231 is important enough to get in to 10.0.1, I'll get it landed after I eat dinner.

[pfmoore]

OK, we now have no release blockers or issues on the 10.0.1 milestone,and the CI for master is clean. On that basis, I plan on releasing 10.0.1 tomorrow night (approx 24 hours from now). If anyone knows of any blockers that have been missed, please speak up now, and ideally point me to a PR that's ready to be merged :-)

[pfmoore]

OK, pip 10.0.1 has been released. Thanks to everyone who helped fix issues, and who put up with my nagging. At this point I'm going to call it a wrap for 10.0, and we can start to look forward to 10.1.

Master is now open for business again. I'm going to close this issue now - although I've found it useful, and I suggest that it might be worth having a "Release x.y.z" issue open for future release, to give a central point for discussions and for people who want to keep an eye on progress.

🎉 ✨ 🍺

[pradyunsg]

🎉

@pfmoore I've taken the liberty of closing the 10.0.1 milestone. :)

I suggest that it might be worth having a "Release x.y.z" issue open for future release

+1