KERNEL_ prefixed environment variable to enterprise gateway

[Chiragasourabh]

Hi, I want to set a variable prefixed with KERNEL_ to the enterprise gateway in deployment yaml and use that variable to build the kernel-pod.yaml.

This was working in 2.6.0 version of enterprise-gateway . but in 3.2.2 this doesn't work and os.environ() from launch_kubernetes.py doesn't find the KERNEL_ prefixed variable set for the enterprise gateway.

However, the set environment variable is available within the enterprise gateway pod.

Is this a regression issue or has something changed here?

Possible workaround,

I can pass kernel-prefixed variables as env in the payload. However, as this is more sensitive info, I want it to be available only for the enterprise gateway.

What would be more cleaner approach?

[kevin-bates]

Hi @Chiragasourabh. There was a regression like what you describe but that should have been resolved via #1164 which should be present in 3.1.0. One thing I'm wondering is that this fix required changes to the launcher and pod yaml files and I wonder if you updated those locations (where your configured kernel specifications reside).

[Chiragasourabh]

this fix sets all the environment variables from the EG to kernels .

but I'm using a KERNEL_ prefixed variable to build a kernel_pod.yaml

for example, I'm setting the KERNEL_CUSTOM_LABEL environment variable to the enterprise gateway deployment.yaml and then using

  labels:
    kernel_custom_label: "{{ kernel_custom_label }}"

in kernel_pod.yaml

It looks like there is an issue.
If any env is passed to BaseProcessProxy.launch_process() then none of the variables from the enterprise gateway container are used.
https://github.com/jupyter-server/enterprise_gateway/blob/a2e053aee947c6ffc2f0991012b14ca7b9510ca4/enterprise_gateway/services/processproxies/processproxy.py#L502C1-L505C45

I believe it should be os.environ should be used initially and then it should be updated with the incoming env. (so that incoming env is given more preference over the set env)

[kevin-bates]

hi @Chiragasourabh. We intentionally do not transfer all environment variables from the EG process to each kernel process for security reasons. To transfer selected environment variables from EG to the the kernel process, you must configure the inherited_envs parameter. The kernel's start endpoint then transfers those selected envs from the EG process to the kwargs set here.

Since this is a list-valued parameter, its probably best using the env approach by configuring the EG_INHERITED_ENVS variable to something like:

EG_INHERITED_ENVS=KERNEL_CUSTOM_LABEL,MY_OTHER_ENV_1,MY_OTHER_ENV_2`

The handler will then transfer these envs (and their values) to the kwargs['env'] dictionary.