dev-python/poetry: Add new package, fix bug #831720

[martin-kokos]

Add new package poetry and its dependencies. poetry-core is already present in portage and this package is it's cli tool.
Closes: https://bugs.gentoo.org/831720

Dependency tree of missing packages:

poetry
    poetry-plugin-export
    shellingham
    deepdiff
        (clevercsv)
            (tabview)
            (wilderness)
        (orjson)
    cachy
    (pytest-randomly)
        (factory-boy)
            (mongoengine)
    flatdict

Checklist:

• deepdiff: Merge fix tests: Use tmp_path fixture seperman/deepdiff#362
• orjson - Get from Guru? (up the python_targets)
• wilderness: no tests picked up by unittest; Mege fix Add MANIFEST.in GjjvdBurg/wilderness#7 (optional)
• cachy: package abandoned, used only in tests
  • fix upstream test: remove cachy dependency and cache compatiblity tests python-poetry/poetry#7437
  • patch poetry tests to skip tests requiring cachy
• pytest-randomly: no tests picked up by pytest (also in Guru (but with different license?)) Add tests in Manifest pytest-dev/pytest-randomly#519
• factory-boy: tests fail, need to run against local mongo instance; need to start it first?
• flatdict:
  • drop flatdict dependency python-poetry/poetry#7369 (planned for release with poetry-1.4)
  • no test found by unittest; Fix MANIFEST.in: Add tests.py gmr/flatdict#55
  • patch poetry tests to skip test requiring flatdict
• Split into separate commits
• Remove python-3.8

Help needed:

• poetry: network sandbox violation (tests try pip install ), disabled tests
• How to solve poetry <-> poetry-plugin-export circular dependency (as defined in respective pyproject.toml)
  • Removed poetry as from poetry-plugin-export ebuild
  • Is PDEPEND the way to go?
• clevercsv: some kind of setuptools problem building; see (optional)
• tabview: tests failing; needs tty to test curses, see (optional)
• Verify dependencies are actually satisfied on a clean system. Does some QA pipeline check this automatically or needs to be checked manually?

[gentoo-bot]

Pull Request assignment

Submitter: @martin-kokos
Areas affected: ebuilds
Packages affected: dev-python/poetry, dev-python/poetry-plugin-export, dev-python/shellingham

dev-python/poetry: @gentoo/proxy-maint (new package)
dev-python/poetry-plugin-export: @gentoo/proxy-maint (new package)
dev-python/shellingham: @gentoo/proxy-maint (new package)

Linked bugs

No bugs to link found. If your pull request references any of the Gentoo bug reports, please add appropriate GLEP 66 tags to the commit message and request reassignment.

New packages

This Pull Request appears to be introducing new packages only. Due to limited manpower, adding new packages is considered low priority. This does not mean that your Pull Request will not receive any attention, however, it might take quite some time for it to be reviewed. In the meantime, your new ebuild might find a home in the GURU project repository: the ebuild repository maintained collaboratively by Gentoo users. GURU offers your ebuild a place to be reviewed and improved by other Gentoo users, while making it easy for Gentoo users to install it and enjoy the software it adds.

Missing GCO sign-off

Please read the terms of Gentoo Certificate of Origin and acknowledge them by adding a sign-off to all your commits.

---

In order to force reassignment and/or bug reference scan, please append [please reassign] to the pull request title.

Docs: Code of Conduct ● Copyright policy (expl.) ● Devmanual ● GitHub PRs ● Proxy-maint guide

[thesamesam]

• Please split into one commit per package (so any dependencies will be first).
• Please add metadata.xml, hopefully including yourself as a maintainer if you're willing (see https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers/User_Guide#Proxied_maintainer_in_metadata.xml and the rest of that page)

[ghost]

You have forgot metadata.xml and Manifest is wrong: didn't you used pkgdev, did you?

EDIT

You may want to install pkgdev and run pkgdev manifest, pkgdev commit and pkgdev push and be sure to add the following:

[DEFAULT]
# Add a 'Signed-off-by' tag to all commits, regardless of repository
commit.signoff = true

# Ask before creating a commit with detected serious QA issues
push.ask = true

inside ~/.config/pkgdev/pkgdev.conf

SIDENOTE: be sure to have an OpenGPG key too to sign your packages

[martin-kokos]

I have pushed a new commit (for the time being bundle into single, splitting added to checklist)

I have updated the description with an overview of missing dependencies and a checklist with issues I couldn't resolve by myself. A recurring issue is tests not being found, some new licenses.

I'd appreciate some pointers on how to go about resolving those issues.

[martin-kokos]

deepdif upstream fix was accepted and released. Version in PR was bumped and test pass.
Also deepdiff got new dependency orjson, so added that ebuild too based off of older version in GURU.

[ghost]

Generally good looking, I have only one question: I see you have added python project to it but not yourself, why? I hope you are going to add and maintain them in future, right?

[martin-kokos]

I haven't included myself in metadata because I'm barely orienting myself in gentoo documentatio, I just barely understand how portage works am not familiar with maintainership at all. I don't really have any experience packaging python or troubleshooting problems that might arise. I am not exactly against writing myself up, but in short I don't feel that competent. I'd rather someone evaluate that after I get some more exposure?

[martin-kokos]

I just "remembered" some of these packages like pytest-randomly do not include the tests in the package on PYPI (on purpose pytest-dev/pytest-randomly#519 ), therefore I don't think I can use pypi.eclass to get them - reverted to getting package from GH.

Yeah, that's inevitable then.

Do we need pytest-randomly though? I thought its only purpose is to randomize test order — and there is really no reason to bother ourselves with that.

You're right. It's not needed. Removed the whole dep branch

 pytest-randomly
        factory-boy
            mongoengine

[mgorny]

I've merged shellingham, flatdict and orjson.

[martin-kokos]

I've merged shellingham, flatdict and orjson.

I had a poetry PR merged to remove flatdict, which is now live as of new poetry-1.4 release, so it won't be needed. I have some failing test on poetry-1.4, but I will bump it as I figure it out.

[martin-kokos]

test that are broken for me on new poetry-1.4.0 are:

FAILED tests/installation/test_chef.py::test_prepare_sdist - poetry.installation.chef.ChefBuildError: Backend 'poetry.core.masonry.api' is not available.
FAILED tests/installation/test_chef.py::test_prepare_directory - poetry.installation.chef.ChefBuildError: Backend 'poetry.core.masonry.api' is not available.
FAILED tests/installation/test_chef.py::test_prepare_directory_with_extensions - poetry.installation.chef.ChefBuildError: Backend 'poetry.core.masonry.api' is not available.
FAILED tests/installation/test_chef.py::test_prepare_directory_editable - poetry.installation.chef.ChefBuildError: Backend 'poetry.core.masonry.api' is not available.

maybe something rings a bell

[l1nd3r0th]

Hi!

Just dropping by to say: Thanks for your work on this! Will be great to have poetry in Portage. 👌

[martin-kokos]

I have bumped poetry and poetry-plugin-export to current version. Same test are failing as in 1.4.0, but they pass with network so I've disabled them for now. I don't know the reason.

[martin-kokos]

I've come across a bit of a blocker. Poetry supports plugins. These plugins can be installed with a command such as poetry self add poetry-plugin-up. This process tries to installed requirements via pip globally which is not compatible with Gentoo and results in an error such as:

  EnvCommandError

  Command ['/usr/bin/python3.11', '-m', 'pip', 'uninstall', 'urllib3', '-y'] errored with the following return code 1

  Output:
  error: externally-managed-environment

  × This environment is externally managed
  ╰─>
      The system-wide Python installation in Gentoo should be maintained
      using the system package manager (e.g. emerge).

[martin-kokos]

Bumped deepdiff to a newly released version that doesn't depend on retired toml.

[mgorny]

Thank you for your perseverance! I hope I'll manage some time to look via it all in the coming days.

[mgorny]

Thanks, again. I've pushed deepdiff and commented on poetry*.

[mgorny]

 * python3_10: running distutils-r1_run_phase python_test
python3.10 -m pytest -vv -ra -l -Wdefault --color=yes -o console_output_style=count -p no:cov -p no:flake8 -p no:flakes -p no:pylint -p no:markdown -p no:sugar -p no:xvfb -p no:tavern
ImportError while loading conftest '/tmp/portage/dev-python/poetry-plugin-export-1.4.0/work/poetry_plugin_export-1.4.0/tests/conftest.py'.
tests/conftest.py:11: in <module>
    from poetry.config.config import Config as BaseConfig
E   ModuleNotFoundError: No module named 'poetry.config'

Missing poetry test-dep?

[mgorny]

Ping.

[gentoo-repo-qa-bot]

Pull request CI report

Report generated at: 2023-07-12 18:55 UTC
Newest commit scanned: 410bb7b
Status: ❌ broken

New issues caused by PR:
https://qa-reports.gentoo.org/output/gentoo-ci/1769e2ae1e/output.html#dev-python/poetry

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/1769e2ae1e/output.html

[mgorny]

I'm sorry about the delay. Once again, thank you!

I've had to remove py3.12 because of non-3.12 dependencies but other than that, all green and merged!

Also, please remember to run pkgcheck scan --commits --net to make sure everything is good.

[mgorny]

Ah, sorry, one more thing:

poetry 1.5.1 has requirement cachecontrol[filecache]<0.13.0,>=0.12.9, but you have cachecontrol 0.13.1.
poetry 1.5.1 has requirement keyring<24.0.0,>=23.9.0, but you have keyring 24.2.0.
poetry 1.5.1 has requirement urllib3<2.0.0,>=1.26.0, but you have urllib3 2.0.4.

I guess we need to sed the dependencies.