Please publish source distribution

[zsimic]

I would love to try opencv-python, but running into issues pip-installing it (via pygradle, https://github.com/linkedin/pygradle)

Could you by any chance also publish an sdist with your pypi package?
Wheels is great, but having an sdist (as usual) would be awesome too for things that don't quite use wheels yet (and also if one wants to rebuild the complete package on target machine)

[warsaw]

Not publishing an sdist on PyPI could also prevent it from being included in various Linux distros.

[skvark]

I understand that an sdist is needed to enable source builds. This project was originally created to provide a very easy way to install OpenCV as a pre-built package. All the build scripts are targeted directly for CI services (Travis, Appveyor). They are decoupled from setup.py which means that with the current project structure there's no point to push sdist to PyPI because setup.py install won't simply work as it's supposed to work.

It's certainly possible to publish an sdist in the future, but it requires major changes to the build logic and it's really hard to say how much work it would need to make sdist happen.

You can install the current pre-built packages to any glibc based Linux distro with pip. Including this distribution in Linux distros is not a priority.

[zsimic]

I took a look at the setup, and I see it's not easy indeed.
Would be an awesome addition though :)

[native-api]

I looked into this. It would require to override generate_source_manifest (subcommand of sdist).

• would override skbuild.command.generate_source_manifest
  • in repo checkout
    • writes custom envvars into files
  • adds opencv/*, opencv_contrib/* if needed, custom envvars files, OpenCV version file to the list of files to package
• OpenCV version detection logic also needs to be tweaked to unconditionally use the prerecorded file if not in repo checkout

The idea is to save all data derived from custom envvars and Git metadata (including a snapshot of the submodules needed) and include it into the sdist tarball -- so that sdist will be built with setup.py build in the same configuration as it was created in.

[skvark]

python setup.py sdist runs fine currently and produces an archive (including the correct snapshot of the OpenCV submodules), but trying to install it produces errors because the archive is not a Git repo and therefore the correct version (Git tag) cannot be determined (find_version.py fails). So yes, the version and other possible needed variables should be written to file and read from there in the case of source distribution install.

[zsimic]

Not sure if it would indeed help here or not (since this is a quite advanced setup.py), but with setupmeta I definitely dealt with precisely this case.

You can take a look in case it applies to you:

• https://github.com/zsimic/setupmeta
• versioning doc: https://github.com/zsimic/setupmeta/blob/master/docs/versioning.rst

In your case, you would use the "advanced" section of the versioning doc linked above, provide a function for main that returns the opencv version (no need to store it anywhere), and extra would be another function that computes your 4th component...

However, after looking into setup.py, and trying to simply run setup.py --name, I realized it's full of side effects: it installed numpy and scikit-build into my clean python installation (!)... had to re-install it to get it clean again.

So, wanted to chime in to say I solved the exact same problem: determine version from git tags, yet find a way to make it work without git repo as well. It's very doable, you just need to find version from packaging metadata when there is no .git folder basically...

[zsimic]

I think long term you'll be better off if you can manage to write setup.py without any on-the-fly installations, ie get rid of get_or_install(), import skbuild, don't run pip install (!) etc

[skvark]

setup.py should work now with sdist. I spent some time refactoring it and making it compatible with pip wheel command. The build dependencies can be now defined in pyproject.toml so there are no side effects related to those dependencies. However, for example scikit-build does not currently implement PEP 517 (scikit-build/scikit-build#124) so there are still many things which need to be fixed in the future when the Python packaging tools get better.

Source distributions will be in PyPI in next release. From now on, local builds shoud be done with pip wheel command. There's more guidance in the readme file.

[tedski]

Do you have an ETA on the next release that will push source distributions to PyPI?

[skvark]

@tedski #362 has to be fixed before any new releases are pushed to PyPI.

[skvark]

Source distributions will be up in PyPI in the next 24 hours.