Slow Jupyter lab response following Panel 0.13 installation #3531

ItayKishon · 2022-05-17T14:25:42Z

ALL software version info

Windows 10 Business, 10.0.19044 Build 19044
Python 3.9.12
Conda 4.12.0
jupyterlab 3.4.2
panel 0.13.0
(and various others; see requirements.txt below)

Description of expected behavior and the observed behavior

Following installation of Panel 0.13.0, Jupyter lab becomes very slow to respond. This can be seen in various operations, e.g. cell evaluation, autocompletion, or interactive widget response. Operation which are normally instantaneous (e.g. evaluating a=5) can take several seconds (2 up to 5) to complete.
Note that this is without importing anything, specifically not even Panel.

Trying to narrow down the origin of the problem, it seems that previous versions of Panel do not exhibit this behavior. Specifically, I've downgraded to Panel=0.12.6 or 0.12.7, and everything goes back to normal.

Complete, minimal, self-contained example code that reproduces the issue

Creating a new minimal env:

conda create --name Panel_test_0 python=3.9 jupyterlab panel

yields the following slow environment - requirements[Panel_0.13.0].txt

This is fixed by downgrading to Panel=0.12:

conda install panel=0.12

which yields a normal-response environment - requirements[Panel_0.12.6].txt

(checked with 0.12.7, which produces the same results as 0.12.6)

Stack traceback and/or browser JavaScript console output

N/A

Screenshots or screencasts of the bug in action

(I can added if needed)

The text was updated successfully, but these errors were encountered:

philippjfr · 2022-05-17T14:59:17Z

Thanks for the issue. I'm a little bit skeptical because I (and multiple devs I know) use JupyterLab daily with latest Panel installed and experience no such slowdown. Can you provide some more background? Is there anything non-standard about your environment? Are you running it locally?

xavArtley · 2022-05-17T15:32:30Z

I experienced too a slowdown of jupyterlab with panel>=0.13 and jupyterlab>=3.
I switched to vscode jupyter extension to develop with panel now.
My OS is windows 10 too

jbednar · 2022-05-17T15:41:38Z

Hmm; wonder if it's somehow Panel + JupyterLab + Windows 10? Any other Windows 10 users happily using JupyterLab + Panel and not noticing this issue?

philippjfr · 2022-05-17T15:43:39Z

Only thing I can imagine is that the Jupyter serverextension which gets activated automatically in 0.13 is slowing things down. As an experiment could you maybe try to install the older version (0.12.4) and then run:

jupyter serverextension enable panel.io.jupyter_server_extension

If that also slows things down we've found the culprit.

philippjfr · 2022-05-17T15:44:14Z

Oh also could you both report your tornado versions.

ItayKishon-Vayyar · 2022-05-17T16:30:37Z

Tornado version: 6.1
(in both scenarios; nothing is downgraded when going from 0.13 to 0.12, apart from the explicit Panel).

Re. panel.io.jupyter_server_extension: it makes no difference.
(I initially tried to enable that for 0.12.6, which - as I understand - was not yet supported? In 0.12.7 I was able to enable it, and it made no difference. I.e. - Jupyter responds swiftly, as it should.)

Also - local installation, and a completely fresh env (creation command specified above, along with requirements.txt).

xavArtley · 2022-05-18T11:40:34Z

Same tornado version

xavArtley · 2022-05-18T12:17:06Z

I made a small video comparing jupyter lab with panel 0.12.7 & 0.13
https://www.youtube.com/watch?v=r6nBdhKjMyk

panel.io.jupyter_server_extension does not change the 0.12.7 speed

philippjfr · 2022-05-18T16:09:49Z

I'm totally guessing at the cause of this but could you try if there's any change with #3534.

ItayKishon-Vayyar · 2022-05-19T06:01:19Z

I'm afraid #3534 doesn't fix it.

To make sure I was building the project correctly, I compared 3 versions: the potential fix, latest tag (v0.13.1a7) and the previous release (v0.12.7).

I've also recorded short videos demonstrating the problem. In each one, I evaluate the first cell (note the time it takes), and in the 2nd cell I press TAB immediately after the letter a. You can see that only v0.12 is responding swiftly.

panel_.v0.13.1a7.mp4

panel_.asyncio_loop_jupyter_fix.mp4

panel_.v0.12.7.mp4

xavArtley · 2022-05-19T08:59:42Z

Don't know if it's related or if I made a mistake but the preview does not work for me:

[E 2022-05-19 10:55:10.517 ServerApp] 500 GET /panel-preview/render/Untitled1.ipynb (::1) 190.84ms referer=http://localhost:8888/panel-preview/render/Untitled1.ipynb
[I 2022-05-19 10:55:13.389 ServerApp] Saving file at /Untitled1.ipynb
[E 2022-05-19 10:55:13.689 ServerApp] Uncaught exception GET /panel-preview/render/Untitled1.ipynb (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/panel-preview/render/Untitled1.ipynb', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\tornado\web.py", line 1704, in _execute   
        result = await result
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\jupyter_server_extension.py", line 116, in get
        page = server_html_page_for_session(
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\bokeh\embed\server.py", line 255, in server_html_page_for_session
        bundle = bundle_for_objs_and_resources(None, resources)
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\bokeh\embed\bundle.py", line 211, in bundle_for_objs_and_resources
        js_files.extend(js_resources.js_files)
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\resources.py", line 299, in js_files
        dist_dir = self.dist_dir
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\resources.py", line 259, in dist_dir
        if self.absolute:
    AttributeError: 'Resources' object has no attribute 'absolute'
[E 2022-05-19 10:55:13.694 ServerApp] {
      "Host": "localhost:8888",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
      "Referer": "http://localhost:8888/panel-preview/render/Untitled1.ipynb",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36"    }
[E 2022-05-19 10:55:13.696 ServerApp] 500 GET /panel-preview/render/Untitled1.ipynb (::1) 196.01ms referer=http://localhost:8888/panel-preview/render/Untitled1.ipynb

ItayKishon-Vayyar · 2022-05-19T09:09:22Z

Don't know if it's related or if I made a mistake but the preview does not work for me:

[E 2022-05-19 10:55:10.517 ServerApp] 500 GET /panel-preview/render/Untitled1.ipynb (::1) 190.84ms referer=http://localhost:8888/panel-preview/render/Untitled1.ipynb
[I 2022-05-19 10:55:13.389 ServerApp] Saving file at /Untitled1.ipynb
[E 2022-05-19 10:55:13.689 ServerApp] Uncaught exception GET /panel-preview/render/Untitled1.ipynb (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/panel-preview/render/Untitled1.ipynb', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\tornado\web.py", line 1704, in _execute   
        result = await result
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\jupyter_server_extension.py", line 116, in get
        page = server_html_page_for_session(
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\bokeh\embed\server.py", line 255, in server_html_page_for_session
        bundle = bundle_for_objs_and_resources(None, resources)
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\bokeh\embed\bundle.py", line 211, in bundle_for_objs_and_resources
        js_files.extend(js_resources.js_files)
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\resources.py", line 299, in js_files
        dist_dir = self.dist_dir
      File "D:\CIVA_RP\CIVA_SIE\civa-rep\ProjetCIVA\ceaComponents\dev\python\lib\site-packages\panel\io\resources.py", line 259, in dist_dir
        if self.absolute:
    AttributeError: 'Resources' object has no attribute 'absolute'
[E 2022-05-19 10:55:13.694 ServerApp] {
      "Host": "localhost:8888",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
      "Referer": "http://localhost:8888/panel-preview/render/Untitled1.ipynb",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36"    }
[E 2022-05-19 10:55:13.696 ServerApp] 500 GET /panel-preview/render/Untitled1.ipynb (::1) 196.01ms referer=http://localhost:8888/panel-preview/render/Untitled1.ipynb

@xavArtley - I'm not sure, but I've stumbled upon this exact problem, and I believe it has already been fixed (not yet released): #3442
Moreover - as that fix has been merged (AFAIK), and I've tested the latest master, I would guess it's not related to the slowdown issue.

xavArtley · 2022-05-19T09:29:42Z

Other experiment if I launch classic notebook from command line I have no slow down
However If I launch classic notebook from the help menu of jupyter lab I have the slow down

philippjfr · 2022-05-19T13:28:47Z

So the change in #3534 actually had no effect at all, will get another PR up which hopefully will have the effect that I originally intended.

philippjfr · 2022-05-20T08:47:03Z

I'm having a problem reproducing this. When I install 0.13.0 from pip or conda I see the slowdown but if I do a develop install locally I don't. How were you guys installing from the branch?

xavArtley · 2022-05-20T08:58:00Z

I don't use a developpement version. I only use pip version.

philippjfr · 2022-05-20T09:05:45Z

A proper pip install (without -e) does it, so I'm git bisecting now. Tedious but I'm close.

ItayKishon · 2022-05-20T09:07:32Z

My former results are from a plain python setup.py install.
I'm now setting up a dev environment (per https://panel.holoviz.org/developer_guide/index.html), to see if this disappears when on dev.

philippjfr · 2022-05-20T09:19:41Z

So git bisect does blame: Patch asyncio event loop policy on windows (#3339), which means my initial guess was correct, just the fix wasn't right. This does confuse me though because according to https://github.com/jupyter/notebook/pull/5907/files the WindowsProactorEventLoopPolicy should be fine for tornado>=6.1 and python>=3.8.

philippjfr · 2022-05-20T09:23:38Z

Nevermind, reading that PR, it was actually reverted because:

Looking back at tornado docs, I think we should still be using the selector loop all the time. Performance will be better, as ~everything we do is built add/remove_reader, which are slower when proactor is used do to the extra thread.

So I'll have to find a way to detect properly if we are running in Jupyter and not apply that patch.

ItayKishon · 2022-05-20T09:24:17Z

I'm afraid something weird is still going on -
I am reproducing that problem also on dev.

There's a caveat, though: I wasn't able to completely follow the dev setup guide, specifically doit env_create -c pyviz/label/dev -c conda-forge --name=panel_devdev --python=3.9 which gave some error (I'm afraid I haven't saved it. Can reproduce if helpful).

My exact setup commands are as follows:

git checkout v0.13.1a7
conda create --name panel_dev python=3.9 mamba jupyterlab
conda activate panel_dev
mamba install -c pyviz "pyctdev>0.5.0"
doit ecosystem_setup
mamba install pyct bokeh pyviz_comms
python setup.py develop

Above env reproduces the problem.

philippjfr · 2022-05-20T09:34:01Z

Fixed in #3537

ItayKishon · 2022-05-20T09:44:45Z

Hurray!
#3537 does indeed solve that.

On that note - although I've just recently started using Panel, I find it extremely valuable & powerful. Thank you, and all those involved, for this great project.

philippjfr · 2022-05-20T11:04:44Z

Thanks for the kind words and glad we could figure this out. Will release Panel 0.13.1 later today.

philippjfr mentioned this issue May 18, 2022

Do not switch asyncio loop in Jupyter #3534

Merged

philippjfr mentioned this issue May 20, 2022

Do not patch event loop on jupyter_server on windows #3537

Merged

philippjfr closed this as completed in #3537 May 20, 2022

philippjfr mentioned this issue Jun 15, 2022

Performance problems with panel > 0.13 and Lab3 #3627

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Slow Jupyter lab response following Panel 0.13 installation #3531

Slow Jupyter lab response following Panel 0.13 installation #3531

ItayKishon commented May 17, 2022 •

edited

philippjfr commented May 17, 2022

xavArtley commented May 17, 2022

jbednar commented May 17, 2022

philippjfr commented May 17, 2022 •

edited

philippjfr commented May 17, 2022

ItayKishon-Vayyar commented May 17, 2022 •

edited

xavArtley commented May 18, 2022

xavArtley commented May 18, 2022

philippjfr commented May 18, 2022

ItayKishon-Vayyar commented May 19, 2022

xavArtley commented May 19, 2022

ItayKishon-Vayyar commented May 19, 2022

xavArtley commented May 19, 2022

philippjfr commented May 19, 2022

philippjfr commented May 20, 2022

xavArtley commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

Slow Jupyter lab response following Panel 0.13 installation #3531

Slow Jupyter lab response following Panel 0.13 installation #3531

Comments

ItayKishon commented May 17, 2022 • edited

ALL software version info

Description of expected behavior and the observed behavior

Complete, minimal, self-contained example code that reproduces the issue

Stack traceback and/or browser JavaScript console output

Screenshots or screencasts of the bug in action

philippjfr commented May 17, 2022

xavArtley commented May 17, 2022

jbednar commented May 17, 2022

philippjfr commented May 17, 2022 • edited

philippjfr commented May 17, 2022

ItayKishon-Vayyar commented May 17, 2022 • edited

xavArtley commented May 18, 2022

xavArtley commented May 18, 2022

philippjfr commented May 18, 2022

ItayKishon-Vayyar commented May 19, 2022

xavArtley commented May 19, 2022

ItayKishon-Vayyar commented May 19, 2022

xavArtley commented May 19, 2022

philippjfr commented May 19, 2022

philippjfr commented May 20, 2022

xavArtley commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 20, 2022

philippjfr commented May 20, 2022

ItayKishon commented May 17, 2022 •

edited

philippjfr commented May 17, 2022 •

edited

ItayKishon-Vayyar commented May 17, 2022 •

edited