Start using pyup/requires.io?

[njsmith]

So that's twice in three days that we've had CI broken by other projects releasing stuff: #646, #651

I suppose this is a gentle nudge to set up pyup.io/requires.io, so these things result in a nice PR with failing tests, rather than a fire drill where it breaks random other people's work. (The way they work is you pin your test dependencies, and then when one of the dependencies gets a new release they automagically send a PR to update the pin, which means you get to run CI on just that dependency upgrade as a standalone thing. So upgrades can't break the world.

The downside of these services is that they tend to create a lot of noise with trivial PRs that someone has to merge...

[njsmith]

Hmm, I hadn't heard of renovate before, but it claims to have python support and a lot of tools for reducing noise: https://renovatebot.com/docs/noise-reduction/

[Zac-HD]

Hypothesis has been using pyup.io for about a year now, and it's been fantastic - just set it to weekly PRs (per linked config). It turns out that this insulates us a bit from breakage too, because many problems are fixed by a quick patch release by upstream before the update PR is opened 😄

Then either we confine CI breakage to that single PR where it can be fixed, or the first maintainer to see it can click merge.

[njsmith]

Whoops, looks like we never cross-linked... there's some discussion of using renovate in #654

[njsmith]

Update: renovate looks great on paper, but it just... doesn't work. We've had stale pins for days, and it's not sending PRs for them. Also see #687. So I've just disabled it.

I tried setting up requires.io to try, but (a) it was really confusing, (b) now that I finally seem to have everything set up, it doesn't seem to be sending an initial PR? Maybe that's intentional, because it's edge-triggered by releases rather than being level-triggered by dependencies being out of date? Still, not very helpful. Basically zero configurability, too. I guess I'll give up on this.

I guess there's also dependabot? Apparently I turned that on a few weeks ago too and it just never did anything.

I just turned on pyup.io, and it immediately took the lead over the other options by sending a PR (!), that updated our dependencies (!!) (#688).

[njsmith]

I guess there's also dependabot? Apparently I turned that on a few weeks ago too and it just never did anything.

Oh, I'm wrong: I apparently turned it on for some random personal repo, and it never did anything, which makes sense because there was nothing to do. I've gone ahead and enabled it for python-trio/trio now, so we can at least see what it does...

[njsmith]

The nice thing about dependabot is that it has an automerge option. The not so nice thing is that it seems to always send individual PRs for each dependency. (It has options for "daily", "weekly", etc., but its initial "get caught up" update sends a bunch of individual PRs so I'm assuming the other forms of batching work the same.) This means that it simply can't handle upgrades like the ipython 6.5 → 7.0 release, which depends on a new version of prompt-toolkit.

I guess tentatively we're going to end up with pyup.io, set to update weekly to minimize noise.

[njsmith]

Huh! I am more impressed with dependabot than I expected.

It's true that it doesn't support batching unrelated updates together, though apparently it's on the roadmap: https://github.com/dependabot/feedback/issues/185

However, it can handle updates like ipython 6.5 → 7.0. Check out this PR it generated, which not only bumps the dependencies like prompt-toolkit, but also drops some now-unnecessary dependencies: https://github.com/python-trio/trio/pull/690/files

Apparently that's because it actually understands pip-compile: https://dependabot.com/blog/dependabot-now-supports-pip-compile/

Also interesting: when I manually bumped yapf (#694), dependabot noticed and automatically closed its PR (#689 (comment)). pyup didn't notice, so now I'm not sure how to fix its batched PR (#688).

The in-github commands are also kind of impressive, e.g. if a dependency does a broken release and we want to wait for the next release instead of working around it, you can @dependabot ignore this minor version to tell it to wait for the next release.

[njsmith]

OK, dependabot seems to be working pretty well, so I've disabled pyup and we'll see how it goes. 🤞

Going to close this because hopefully we're done now; we can always re-open if that's not true.