RFC drop support for python 3.8 ? bump min dependencies ?

[jeremiedbb]

The 1.3 release being scheduled for the first half of june, it's time to think about bumping the min dependencies.

Up to now we've been trying to roughly follow NEP29 regarding supported Python versions. According to this timeline we should drop support for python 3.8 in april 2023, hence for the 1.3 release. @scikit-learn/core-devs do we agree on this ?

Dropping support for python3.8 means bumping numpy min version to 1.19.3 (currently 1.17.3) according to oldest-supported-numpy

Scipy min version has already been bumped in #24665.
Pytest has also already been bumped in #26373.
Joblib min version is just 1 release before the latest so we probably don't want to bump it yet.

Any objection or request ?

[jjerphan]

Up to now we've been trying to roughly follow NEP29 regarding supported Python versions. According to this timeline we should drop support for python 3.8 in april 2023, hence for the 1.3 release. https://github.com/orgs/scikit-learn/teams/core-devs Do we agree on this ?

I agree with this.

---

Are the latest version of NumPy and SciPy distributed on Ubuntu PPA the ones retaining scikit-learn from upgrading versions?

Also, I think SPEC 0 is the new and up-to-date version of NEP29. If we want scikit-learn to be SPEC 0-compliant, we might need to change the way we update the minimum version of SciPy and NumPy because scikit-learn's dependences minimum supported version are several versions behind the ones of the schedule of SPEC 0 (we should soon drop support for SciPy < 1.8 and NumPy < 1.22).

What do you think of being SPEC 0-compliant?

[betatim]

I think moving forwards to newer versions (that include improvements and bug fixes that others have worked on) is generally a good thing. At the same time forcing people to newer versions, by upping the minimum requirement, can be tricky or at least make their install story trickier (compare "just works with the Python I get from my OS and OS package manager" with "have to install conda to then install scikit-learn"). So I'd make dropping support for older versions dependent on there being a reason to drop it other than "we drop it because we can". For example being able to remove fixes that are vendored into scikit-learn, being able to use newer concepts to start a transition, etc.

So I think being SPEC0 compliant just for the sake of following SPEC0 wouldn't get me excited.

A different view on which Python versions to support is https://devguide.python.org/versions/. I think this is much slower than SPEC0. Python 3.7 is only just coming up to EOL.

[GaelVaroquaux]

What Tim said! :) What are the benefits of dropping 3.8 for us?

[lesteve]

I am fine with dropping support for Python 3.8:

• Scipy has already dropped support for Python 3.8 in main and has mentioned a rough timeline of June for its next release MAINT: update supported versions of Python and NumPy to follow NEP29 scipy/scipy#17959
• numpy has already dropped Python 3.8 in main MAINT: Replace Python3.8 by Python3.9. numpy/numpy#22902 and according to MAINT: update supported versions of Python and NumPy to follow NEP29 scipy/scipy#17959 (comment), they tend to aim for a release around December/January and one around June/July
• at one point Google colab was an argument to not drop older Python versions. Looks like it is a lot more up-to-date now, e.g. the default runtime has Python 3.10 according to https://medium.com/google-colab/colab-updated-to-python-3-10-27eb02daa162

[adrinjalali]

I'm also happy to drop 3.8, but we can do it for 1.4

[jeremiedbb]

For example being able to remove fixes that are vendored into scikit-learn, being able to use newer concepts to start a transition, etc.

There's one vendored fix that would be dropped, see

scikit-learn/sklearn/utils/fixes.py

Lines 131 to 179 in 4ccf41b

	###############################################################################
	# Backport of Python 3.9's importlib.resources
	# TODO: Remove when Python 3.9 is the minimum supported version
	def _open_text(data_module, data_file_name):
	if sys.version_info >= (3, 9):
	return resources.files(data_module).joinpath(data_file_name).open("r")
	else:
	return resources.open_text(data_module, data_file_name)
	def _open_binary(data_module, data_file_name):
	if sys.version_info >= (3, 9):
	return resources.files(data_module).joinpath(data_file_name).open("rb")
	else:
	return resources.open_binary(data_module, data_file_name)
	def _read_text(descr_module, descr_file_name):
	if sys.version_info >= (3, 9):
	return resources.files(descr_module).joinpath(descr_file_name).read_text()
	else:
	return resources.read_text(descr_module, descr_file_name)
	def _path(data_module, data_file_name):
	if sys.version_info >= (3, 9):
	return resources.as_file(resources.files(data_module).joinpath(data_file_name))
	else:
	return resources.path(data_module, data_file_name)
	def _is_resource(data_module, data_file_name):
	if sys.version_info >= (3, 9):
	return resources.files(data_module).joinpath(data_file_name).is_file()
	else:
	return resources.is_resource(data_module, data_file_name)
	def _contents(data_module):
	if sys.version_info >= (3, 9):
	return (
	resource.name
	for resource in resources.files(data_module).iterdir()
	if resource.is_file()
	)
	else:
	return resources.contents(data_module)

It's the only one I found, and I don't think that 3.9 has features that we deseperately want to use (there's the dict update with the syntax d1 | d2 but not very urgent 😄), so I'm fine keeping 3.8 for this release

[jeremiedbb]

Also, I think SPEC 0 is the new and up-to-date version of NEP29. If we want scikit-learn to be SPEC 0-compliant

SPEC 0 is still in draft mode, right ? I don't think it's been widely accepted yet. We've been a lot more conservative so far, so I think we'll need a slow transition if we decide to adopt it at some point

[jeremiedbb]

Are the latest version of NumPy and SciPy distributed on Ubuntu PPA is what retains scikit-learn from upgrading versions?

Usually yes, although #24665 is a counter example. For numpy we used to but we now try to follow oldest-supported-numpy instead.

[betatim]

Just re-read #24665, I think the loophole we ended up with is that "we want scikit-learn to work on a Ubuntu LTS version". In this case it meant that 20.04 (which is a LTS) might not work, but 22.04 (also a LTS) has to work. But this was somehow more tricky because it involved ATLAS (the Ubuntu specific BLAS implementation). So for example I don't know if what we actually ended up saying is "20.04 will work, but you will have to use a different (slower) BLAS implementation".

For reference: Ubuntu 22.04 has scipy 1.8.0, numpy 1.21.5 and Python 3.10.4. Ubuntu 20.04 is still on Python 3.8.4

Repeating Gael's question: can someone think of a benefit to the project from dropping Python 3.8?

[jjerphan]

To make my previous comment #26438 (comment) more precise: I am fine dropping Python 3.8 but I do not think there are strong reasons to.

[ogrisel]

+1 for dropping support of 3.8 once 3.12 is out, probably for scikit-learn 1.4.

[jeremiedbb]

Another argument raised in an irl discussion for not keeping a too large range of supported versions is because it requires more CI resources. That being said, keeping 3.8 means that we support 4 python versions which is acceptable. Let's say we try to not go beyond that.

Since supporting py38 for one more release doesn't provide major benefits and doesn't make our life more difficult, let's keep it for now and drop it for 1.4.

[EwoutH]

NEP 29 was superseded by the scientific python ecosystem coordination guideline SPEC 0 — Minimum Supported Versions. Where NEP had a 42 month support windows (3.5 years) for each Python version, SPEC 0 recommends a 3 year support window.

If we were to follow SPEC 0, than Python 3.9 support can already be dropped, since it was released October 5th, 2020.

[glemaitre]

I would consider to be pragmatic: if at the time our own release, the latest version of our release (NumPy, SciPy, pandas?) are still supporting Python 3.9, then it is safer to not drop support until this is not becoming a maintenance burden for some reason.