Projects that depend on lxml fail to build in newer Pythons

[jaraco]

I support a number of projects that depend on lxml (svg.charts, openpack, jaraco.site, ...). Every time a new Python is released, lxml lags in supplying binary artifacts for that Python, meaning that builds will fail unless those platforms are excluded from the build matrix, requiring a per-project disablement of new platforms and then a later re-enablement. Furthermore, the rollout of binary artifacts for lxml is irregular, often landing on some platforms before others. I've tried building my own lxml artifacts, but found that the process isn't readily replicable and is often broken. It wouldn't be so bad if these projects could be configured with a PyPI mirror that had advance builds of lxml available, but supporting that is infeasible.

Maybe there's some way that the builds can be configured to fail quietly when lxml isn't available. It would need to be a tox plugin or wrapper, as the failure occurs before the test suite is run. Ideally, this step would be cheap enough that it could be run in all projects and not just the ones that depend on lxml.

[jaraco]

I looked at extending tox, but it doesn't seem to have a hook that would allow for suppressing an error during an install operation.

I also [searched](https://www.google.com/search?q=pip install --ignore failed dependencies) for a way to ignore failed dependencies in the installer, but there seems to be no such option.

Now I'm considering other options:

fake package

Consider a proxy package (lenient-lxml) that will always install but will only depend on lxml when it's available. That would require that this fake package be maintained to always capture where lxml is viable. It would still require the test suite to additionally detect when lxml is missing and skip relevant tests.

tox wrapper

A separate program could be installed that would first check if lxml is viable and only invoke tox if it might be expected not to fail. Such an operation would be expensive, so would need to be aggressively cached (ideally beyond a single project).

CI plugin

A CI plugin could do something similar to the tox wrapper, especially if a CI plugin could terminate the execution with a non-failure status.

hacked tox

Someone could hack the tox code to hook into the install step and inspect a failed pip failed output, allowing for specific failures.

custom installer

Someone could contribute to pip or hack the code or provide a custom installer to allow for failures.

---

None of these options are great.

[jaraco]

I've started work on the tox wrapper in jaraco/safety-tox that runs tox normally, but suppresses the failure if it's related to a known flaky package. So far, it's been rough going because it requires teeing output from a process, which is a difficult problem.

[jaraco]

I've spent all morning on this and it's still broken. I'm going back to just manually excluding environments that fail.