Regression testing pyright with open-source libraries

[erictraut]

For a long time, the maintainers of mypy have used this nifty tool called mypy_primer to run regression tests for every PR submitted to the mypy repo. This tool runs mypy on a large number of open-source typed Python libraries to determine whether a change to the mypy source code breaks something. The mypy_primer tool is also used by the maintainers of typeshed to detect regressions in typeshed stub files.

The author of mypy_primer (@hauntsaninja) has graciously extended the tool to support pyright. By using mypy_primer within pyright's CI workflows, we will now have better visibility into the impact of individual changes made to pyright.

For now, I've configured mypy_primer to run on just a handful of open-source libraries.

If you are a library maintainer and use pyright for your project, let us know if you'd like us to add your library to the pyright regression tests.