elyra/kernel-py based off python 3.9

[amit-chandak-unskript]

Hi @kevin-bates any plans to support elyra kernel based off python 3.9. We use the native jupyterlab (without EG), which is based off 3.9, whereas the elyra kernel is still based off 3.7, causing us python pkg version manageability issues.
Is it easy to build a kernel based off 3.9? I was not able to find the Dockerfile for the same. This link talks its based off scipy-notebook which has moved to 3.10, hence the question if elyra also plans to move. I can help out if needed. Thanks

[kevin-bates]

Hi @amit-chandak-unskript - great question!

TL;DR - yes, let's update the tag of the kernel-py base image to 2022-01-24.

---

Yes, we should update our version of python within the applicable images. The Dockerfiles can be found in etc/docker and, at a minimum, I think we should update the version of Python in kernel-py, kernel-tf-py, kernel-tf-gpu-py and enterprise-gateway.

As you've noticed, Python version updates are inherited, so we really need to update the corresponding base image. Once done, a rebuild of kernel-spark-py (for example) will pickup the change made in kernel-py.

Of these images, all but kernel-tf-gpu-py derives from the docker-stacks images and I'd like to take this opportunity to convert to using the date-based tags for those.

I think we should try to move to Python 3.10, if that works for you. [*] (If not, I'm okay with 3.9, but a more current (yet stable) version is preferred.) It looks like the docker-stacks images were moved to 3.10 in late May - so you could probably look for date-based tags reflecting 2022-05-28 or more recent.

I suspect the tag on the base image for kernel-tf-gpu-py specifies the version of Python. If we decide to only move to 3.9, then this would not require an update.

[*] Hmm - I see that the 2022-01-24 tagged images (used by enterprise-gateway and kernel-tf-py) are also already using Python 3.9, so now I'm inclined to simply update kernel-py to use jupyter/scipy-notebook:2022-01-24 and be done. We can look at moving to 3.10 later and this minimizes the change prior to a release.

Calling docker build within each directory is not sufficient to build the images since these are composed from various artifacts. I would do the following:

make clean dist
make kernel-py kernel-spark-py

The inclusion of kernel-spark-py is just to ensure there aren't build issues introduced from the upgrade of kernel-py.

I can help out if needed.

Your help would be greatly appreciated! (Sorry for the ramble)