[MRG] Update to JupyterLab 4

[jtpio]

Extract the part of #1302 to update to JupyterLab 4, without the Notebook 7 update.

[yuvipanda]

Thanks @jtpio! This looks good to me. I think the following are breaking-ish changes:

1. jupyter-server version to 2.x (how much of a breaking change is this?)
2. Support for python 3.7 in the base environment is now gone. How does this affect repos that specify python 3.7?
3. jupyterlab to 4.x (so some extensions would break)

I don't fully understand (2), but otherwise with labeling this looks good to me.

[jtpio]

jupyter-server version to 2.x (how much of a breaking change is this?)

The default CMD still launches the old server via jupyter notebook it seems though?

repo2docker/repo2docker/buildpacks/base.py

Line 194 in f667caf

CMD ["jupyter", "notebook", "--ip", "0.0.0.0"]

Support for python 3.7 in the base environment is now gone. How does this affect repos that specify python 3.7?

Python 3.7 reached end of life in June 2023: jupyterlab/jupyterlab#13745. JupyterLab 4 requires Python 3.8 now.

jupyterlab to 4.x (so some extensions would break)

Yes, although many have been updated now: jupyterlab/jupyterlab#14590

Also JupyterLab 3 will reach end of maintenance soon: jupyterlab/jupyterlab#15921

[minrk]

How does this affect repos that specify python 3.7?

They are unaffected at the moment. As this PR stands, 3.7 stops getting updates, but continues to be supported for the server environment.

The next stage of reduced support is bumping kernel_env_cutoff_version to 3.8, at which point 3.7 becomes a 'kernel-only' version, like 3.6 and older. I think would be reasonable to make this change here as well, but not required.

Pro bumping kernel_env_cutoff:

• all installs get the same jupyterlab/server versions
• we will need to make this transition eventually, when e.g. a security requirement requires bumping a no-longer-supported package

Con:

• bigger, more disruptive change for 3.7-using repos

Since there is no visible effect on 3.7 repos, I think it's fine to merge this as-is.

[manics]

This may break users who are installing extensions which aren't compatible with JupyterLab 3. I think we should make an announcement on https://discourse.jupyter.org before merging.

[minrk]

announced: https://discourse.jupyter.org/t/ann-repo2docker-upgrading-to-jupyterlab-4/25845