Add Voila configuration option for default kernel environmental variables

[dcnadler]

Currently, to use partially pre-rendered notebooks a user may have to set environmental variables in notebook kernels to access the correct websocket address.

The docs give the example of using voila.json to set these. However, for some deployment platforms these values (e.g. base_url or port) may not be known ahead of time or differ between deployments. Having a command line option provides easier passing of env variables (as opposed to modifying voila.json in-place).

Code changes

A default_kernel_env_variables attribute is added to VoilaConfiguration, which is then passed to the default kernel_pools_config of VoilaKernelManager.

[github-actions]

👈 Launch a Binder on branch dcnadler/voila/kernel_ws_env_vars

[github-actions]

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

Results table

Test file	voila-tree-classic.ipynb	voila-tree-light.ipynb	voila-tree-dark.ipynb	voila-tree-miami.ipynb	basics.ipynb	bqplot.ipynb	dashboard.ipynb	gridspecLayout.ipynb	interactive.ipynb	ipympl.ipynb	ipyvolume.ipynb	multiple_widgets.ipynb	query-strings.ipynb	reveal.ipynb
Render
chromium
actual	90 <- [101 - 120 - 158] -> 269	69 <- [75 - 85 - 101] -> 146	70 <- [75 - 86 - 89] -> 121	65 <- [69 - 78 - 95] -> 137	2172 <- [2299 - 2363 - 2428] -> 3275	2085 <- [2095 - 2138 - 2153] -> 2500	2377 <- [2422 - 2449 - 2558] -> 2782	2377 <- [2386 - 2415 - 2418] -> 2472	1589 <- [1611 - 1669 - 1703] -> 1845	2997 <- [3022 - 3078 - 3321] -> 4031	6717 <- [9766 - 9984 - 10152] -> 11650	5536 <- [5542 - 5560 - 5627] -> 5659	1145 <- [1208 - 1228 - 1252] -> 1320	2788 <- [2898 - 3120 - 3572] -> 4851
expected	3379 <- [3442 - 3517 - 3701] -> 3876	2976 <- [3227 - 3321 - 3421] -> 3604	3608 <- [3623 - 3709 - 3793] -> 3825	4453 <- [4453 - 4523 - 4661] -> 4748	2559 <- [2655 - 2656 - 2660] -> 2674	3982 <- [4079 - 4213 - 4356] -> 4743	12183 <- [18509 - 19553 - 20811] -> 21515	15319 <- [15660 - 15796 - 15912] -> 16056	1517 <- [1920 - 1997 - 2103] -> 2113

❗ Test metadata have changed

--- /dev/fd/63	2022-07-28 13:35:45.958110444 +0000
+++ /dev/fd/62	2022-07-28 13:35:45.958110444 +0000
@@ -4,37 +4,37 @@
     "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
-    "chromium": "97.0.4666.0"
+    "chromium": "94.0.4595.0"
   },
   "systemInformation": {
     "cpu": {
-      "brand": "Xeon® Platinum 8370C",
+      "brand": "Xeon® E5-2673 v3",
       "cache": {
-        "l1d": 98304,
+        "l1d": 65536,
         "l1i": 65536,
-        "l2": 2097152,
-        "l3": 50331648
+        "l2": 524288,
+        "l3": 31457280
       },
       "cores": 2,
       "family": "6",
-      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear",
+      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt md_clear",
       "governor": "",
       "manufacturer": "Intel®",
-      "model": "106",
+      "model": "63",
       "physicalCores": 2,
       "processors": 1,
       "revision": "",
       "socket": "",
-      "speed": 2.8,
+      "speed": 2.4,
       "speedMax": null,
       "speedMin": null,
-      "stepping": "6",
+      "stepping": "2",
       "vendor": "GenuineIntel",
       "virtualization": false,
       "voltage": ""
     },
     "mem": {
-      "total": 7281008640
+      "total": 7291699200
     },
     "osInfo": {
       "arch": "x64",
@@ -42,11 +42,11 @@
       "codename": "Focal Fossa",
       "codepage": "UTF-8",
       "distro": "Ubuntu",
-      "kernel": "5.15.0-1014-azure",
+      "kernel": "5.8.0-1040-azure",
       "logofile": "ubuntu",
       "platform": "linux",
-      "release": "20.04.4 LTS",
-      "serial": "6acefdd3c3134d8096096c3373e75e18",
+      "release": "20.04.3 LTS",
+      "serial": "cfc067bfcb844f35865e279a1b0e66c5",
       "servicepack": "",
       "uefi": false
     }

[trungleduc]

Thanks @dcnadler for your contribution, for now, setting the default environmental variables for the pre-heated kernel is possible by configuring the VoilaKernelManager class. For example, this command will start kernels with the variable foo

voila  --preheat_kernel=true --VoilaKernelManager.kernel_pools_config="{'default':{'kernel_env_variables':{'foo':'bar'}}}"

It's pretty verbose so I think we can make default_kernel_env_variables a setting of VoilaKernelManager instead of VoilaConfiguration

[dcnadler]

Thanks @trungleduc! Did not realize that was an option. I am new to trailets.

Seems like trailets may not let you pass a nested dict in the future?

/anaconda3/envs/voila/lib/python3.10/site-packages/traitlets/traitlets.py:3258: FutureWarning: --kernel_pools_config={"default":{"kernel_env_variables":{"FOO":"BAR"}}} for dict-traits is deprecated in traitlets 5.0. You can pass --kernel_pools_config <key=value> ... multiple times to add items to a dict.

I moved the default_env_variables to the VoilaKernelManager as you suggested. Let me know what you think.

[trungleduc]

Thanks!