Allow "twine check" to run multiple checks

[CyrilRoelandteNovance]

Checkers are registered as entry points (twine.registered_checkers). A
new "check_license" checker has been added.

[bhrutledge]

Thanks for making an effort to improve Twine. However, this seems like a non-trivial change in the application architecture, which I think should have been preceded by a proposal and discussion on the issue tracker. There are already a handful of issues related to making check more useful, and the thinking thus far has been that such logic should be implemented in the packaging library (see pypa/packaging#147), so that it could be used by multiple projects, including PyPI itself. It might be time to re-think that, but until then, I'm wary of "bolting on" additional functionality.

[bhrutledge]

What's the value in defining an entrypoint for registered_checkers? At first glance, it feels like an extra layer of complexity. Why not just call the check_* functions explicitly in twine.commands.check._check_file?

[CyrilRoelandteNovance]

Yes, calling the check_* functions directly would work as well. My goal here was to allow users to define project-specific checkers that may not be useful to all Python developers, or are far too strict to be included in twine itself.

[bhrutledge]

I appreciate the instinct to refactor, but it makes for a big diff that's hard to review. I probably would have just renamed the existing test functions to include the word description.

[CyrilRoelandteNovance]

Indeed, I only realized that when viewing the diff myself :-(

The tests have been slightly modified as well, since they used to be strongly tied to the "long description checking" logic. Now some tests focus on the long description checker, and some tests focus on the infrastructure, and are checker-agnostic.

[sigmavirus24]

These have effectively become a public API and - were we to merge this - must not live in twine.commands.

[sigmavirus24]

So I'm not actually opposed to this. We have a lot on the check roadmap that we can't do anything with. This would allow us to develop things in separate packages that we may want to ditch at some point without them being part of the core command. We can experiment outside of twine and we can allow for custom checkers like mentioned here. I could see my organization wanting to take advantage of custom checkers too.

I'm vaguely +0 on the concept.

To play to Brian's point, we already have entry-points that an organization could use to build its own checkers. You could register a command that looks like myorg.check and then run twine check && twine myorg.check. Is it as efficient? No. Is it less susceptible to breakage? Maybe. Is it available today? Absolutely.

[bhrutledge]

@CyrilRoelandteNovance Apologies for letting this languish. Based on your and @sigmavirus24's comments, I see the value in enabling custom checkers. However, before implementing that, I'd like to have an architecture discussion in a new issue, possibly soliciting opinions from folks who've asked for improvements to check. I'm happy to write up that issue, in which case I'd close this PR.

@sigmavirus24 How does that sound to you?

[sigmavirus24]

However, before implementing that, I'd like to have an architecture discussion in a new issue, possibly soliciting opinions from folks who've asked for improvements to check.

Please do create a new issue but I'm not exactly sure what you expect to come from the architecture discussion. We don't have requirements for what to implement in the first place, how can we discuss architecture without what users need? Do you mean you'd like to discuss an API proposed here? I don't know that it warrants that until we know what people would need to implement the checks they want to implement.

My point stands that today people can already "extend" twine. They can register their own commands and implement their own checks without us having to change what check does. Are you saying you would rather complicate check to allow people to register plugins for that?

[bhrutledge]

Caveat: I have very little experience with proposing and implementing substantial new features/APIs in open-source projects.

We don't have requirements for what to implement in the first place, how can we discuss architecture without what users need? Do you mean you'd like to discuss an API proposed here?

I was thinking that requirements gathering could be part of the discussion, using the API proposed here as an example.

My point stands that today people can already "extend" twine. They can register their own commands and implement their own checks without us having to change what check does. Are you saying you would rather complicate check to allow people to register plugins for that?

I like the idea of plugging in functionality to check, to meet the demand we've seen in other issues. In that way, maybe check would function as a general-purpose "package linter", in the spirit of flake8? That gives folks one CLI command, with consistent command-line options and API.

As it stands, folks can write their own sub-commands, but they're on the hook for argument parsing and reading packages from disk.

[sigmavirus24]

So flake8 has some problems that were I to create a flake8 like package today, I'd do it very differently. I do think we have a bunch of issues to draw upon, and to your point about sub-commands, I think the library APIs we've been slowly building in twine's namespace become useful and I believe those are what we should pass into APIs. Namely, people want to check parts of the metadata, if we can pass that in or make it easy to parse/extract via PackageFile (which I believe we've done already) then we make validating that information significantly easier. So this API, I agree isn't correct and if people want the filename that's available via the PackageFile object as well.

Let's start a new issue, but again, I want to start from practical requirements and talk about what we can do to make this a pleasant experience versus the things learned the hard way in Flake8

[bhrutledge]

Closing in favor of #848.