[Bug]: M1 Mac, maplotlib interactive fails

[shiba24]

Description

Hi, the matplotlib interactive figure in jupyter lab on Mac M1 is broken in my environment. I think this stackoverflow is the same issue.

https://stackoverflow.com/questions/70466222/m1-mac-python-assertion-failure-and-crashes-when-using-osx-as-backend-in-matplo

I posted the same issue in the matplotlib github repo: matplotlib/matplotlib#22545 and at least it is confirmed as reproducibly there.

Reproduce

%matplotlib osx

Then you see endless lines of

2022-02-23 19:00:11.233 Python[7544:10628886] *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:740

Expected behavior

Matplotlib interactive figures:

https://matplotlib.org/stable/users/explain/interactive.html

Context

• Operating System and version: Mac OS Monterey 12.2, M1 MacBook Pro2020
• Browser and version: Chrome Version 98.0.4758.109 (Official Build) (arm64)
• JupyterLab version: 3.2.1
• Matplotlib version: 3.4.3

Thank you!

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[JasonWeill]

Resolving in favor of the matplotlib issue linked above.

[tacaswell]

@jweill-aws Sorry to play hot-potato but Matplotlib relies on IPython / Jupyter to run the GUI event loop integrated with prompt toolkit / tornado (which we must because we can not see into the context we are being run from). I am 99% sure that the issue in on the jupyter side.

[JasonWeill]

@tacaswell OK. Thank you for the additional info. I've reopened this issue.

[tacaswell]

Thank you.

Copying over a comment I left on the mpl side, I would try to get some other OSX UI to run in a jlab context to see it has the same problem.

[ccordoba12]

I'd suggest that this issue should be moved/reopened in the IPykernel repo, which is the package in charge of handling event loops for Jupyter.

Furthermore, I guess this problem was introduced by the switch to asyncio in IPykernel 6. For instance, the Tk backend was broken on Windows until recently due to that: ipython/ipykernel#830

[JasonWeill]

From triage today: It is necessary to test whether this occurs in classic notebook on an M1 Mac. @ccordoba12 's suggestion about opening this in IPykernel is also worth considering especially if this bug occurs in both JupyterLab and classic notebook.

@shiba24 Have you tried using matplotlib on your Mac with classic notebook?

[dstansby]

I can reproduce this using jupyter notebook, with the following jupyter versions:

jupyter                   1.0.0            py39h2804cbe_7    conda-forge
jupyter-cache             0.4.3              pyhd8ed1ab_0    conda-forge
jupyter-server-mathjax    0.2.5              pyhc268e32_0    conda-forge
jupyter-sphinx            0.3.2              pyhd8ed1ab_1    conda-forge
jupyter_client            7.1.2              pyhd8ed1ab_0    conda-forge
jupyter_console           6.4.3              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.2            py39h2804cbe_0    conda-forge
jupyter_server            1.15.3             pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.2              pyhd8ed1ab_0    conda-forge

@ccordoba12 's suggestion about opening this in IPykernel is also worth considering

I'm not sure how to do/test this, could you post (a link to) instructions?

[dstansby]

Ah, perhaps just running %matplotlib osx in an ipython prompt? Doing that works fine for me with

ipython                   8.1.1            py39h2804cbe_0    conda-forge

[ccordoba12]

I can reproduce this using jupyter notebook

Ok, great!

I'm not sure how to do/test this, could you post (a link to) instructions?

Just open again the issue in this repo: https://github.com/ipython/ipykernel, and close this one.

Ah, perhaps just running %matplotlib osx in an ipython prompt? Doing that works fine for me

The mechanisms to handle event loops in IPython terminal and JupyterLab/notebook are totally different. That's why this bug needs to be solved in IPykernel.

[shiba24]

Thank you @dstansby for the info :)

@jweill-aws

The command fails both in jupyter notebook and jupyter lab, with same error. But it works no problem in ipython.

My environment are (not conda, pip)

jupyter notebook --version
6.4.5

jupyter lab --version
3.2.1

ipython --version
7.28.0

[JasonWeill]

@3coins can try to repro using the info from the original description.

[3coins]

I am able to reproduce this in M1 Pro in JupyterLab 3.x. Here is some more info that was logged in addition to what @shiba24 reported.

*** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:740
[I 2022-03-24 10:54:08.691 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports

 *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:740
2022-03-24 10:46:29.713 python3.9[35524:540761] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: _NSEventMask64FromType(type) & WeirdMask'
*** First throw call stack:
(
	0   CoreFoundation                      0x00000001c37d2b08 __exceptionPreprocess + 240
	1   libobjc.A.dylib                     0x00000001c351de14 objc_exception_throw + 60
	2   Foundation                          0x00000001c4701e18 -[NSCalendarDate initWithCoder:] + 0
	3   AppKit                              0x00000001c64ade88 +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:] + 292
	4   libffi.8.dylib                      0x000000010554804c ffi_call_SYSV + 76
	5   libffi.8.dylib                      0x000000010554574c ffi_call_int + 1208
	6   _ctypes.cpython-39-darwin.so        0x0000000105528744 _ctypes_callproc + 1236
	7   _ctypes.cpython-39-darwin.so        0x0000000105522b78 PyCFuncPtr[I 2022-03-24 10:46:32.033 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (2/5), keep random ports

Here is the notebook content, I used to reproduce this error.

%matplotlib osx
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ln, = ax.plot(range(5))
ln.set_color('orange')
plt.ioff()
plt.ion()

[lin-shuyu]

I'm having the same issue where no interactive plot is shown in my notebook cell and I see an Assertion Error in tornado in the jupyter log. Please see screenshots for the error msg.

I want to make an interactive plot as follows:

m = widgets.FloatSlider(min=-5,max=5,step=0.5, description="Slope")
c = widgets.FloatSlider(min=-5,max=5,step=0.5, description="Intercept")

# An HBox lays out its children horizontally
ui = widgets.HBox([m, c])

def plot(m, c):
    x = np.random.rand(10)
    y = m *x + c
    plt.plot(x,y)
    plt.show()

out = widgets.interactive_output(plot, {'m': m, 'c': c})

display(out, ui)

I've installed my python libraries using miniconda, following the commands below:

conda create --name jupyter-widget python=3.9
conda install -c anaconda jupyter
conda install -c conda-forge ipywidgets
pip install --user ipykernel

I'm using Macbook Pro 2021 with a M1 chip.

If I change the sliderbar to dropdown list, then the interactive plot can be shown.

[inchinn1]

I'm having the same issue with an M2 chip. Has anyone found a solution for this bug yet?

[tacaswell]

@lin-shuyu That looks like a different issue as I do not see anything about the osx backend in that traceback and I suspect you are using the inline backend?

I think the three relevant bits of code here are:

ipykernl inputhook: https://github.com/ipython/ipykernel/blob/main/ipykernel/_eventloop_macos.py

ipython inputphook: https://github.com/ipython/ipython/blob/main/IPython/terminal/pt_inputhooks/osx.py

mpl's osx binding: https://github.com/matplotlib/matplotlib/blob/main/src/_macosx.m

My first suspicion is that there is a bug in the ipykernel inputhook. It looks like in the IPython case it is waiting for the user to type something (which it can do because it is the terminal you are sitting in), however in the ipykernal case it is spinning the main loop for a fixed about of time. Rather than time-sharing with prompt-toolkit it is timesharing with tornado which may make a difference.

See https://matplotlib.org/devdocs/users/explain/figure/interactive_guide.html for an explanation of what the input hooks do and why they have to exist.

Given the wording in the errors I'm doubtful that Matplotlib is directly involved, but given that mpl builds an (objective) c extension, it is possible that there is something wrong with the wheels / conda package (the evidence one way or the other is if it reproduces with wheels but not when built from source). If you are crossing conda and wheels there can also be issues with extensions due to the limitations of the wheel metadata.

I also have a suspicion that the M1/M2 chip issue is a red-herring and it is the version of OSX that matters (and the M1/M2 are always on a new enough OSX to be broken).

[ivanlen]

I am having the same issue
M3 mac, same error.
Any ideas?

[krassowski]

Not many ideas, but just highlighting that this issue might be mis-filed because the code for matplotlib/IPython/ipykernel is not in this repository and maintainers of JupyterLab do not necessarily have the expertise to help with issues that may come from other projects.

In particular the issue above may arise from packaging problems where incompatible binaries are installed. My only suggestion is that if you installed using conda, try creating a new environment using native Python and install using pip (and vice versa) - if this helps please let us know as this will narrow it down to a packaging issue. Also, curious what python version you see the issue with.

[ivanlen]

Not many ideas, but just highlighting that this issue might be mis-filed because the code for matplotlib/IPython/ipykernel is not in this repository and maintainers of JupyterLab do not necessarily have the expertise to help with issues that may come from other projects.

In particular the issue above may arise from packaging problems where incompatible binaries are installed. My only suggestion is that if you installed using conda, try creating a new environment using native Python and install using pip (and vice versa) - if this helps please let us know as this will narrow it down to a packaging issue. Also, curious what python version you see the issue with.

Macbook M3
Sonoma 14.4.1
Python 3.11.9
iPython 8.24.0
matplotlib 3.8.4
ipykernel 6.29.3

I was using conda, I am going to give it a try to install everything with pip and I come back.

[ivanlen]

Yes, I just reinstalled everything using pip in a new venv and same error

2024-05-14 10:00:46.025 python[43338:1713281] *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:653

[tacaswell]

Can you try with the mpl 3.9.0rc ? (pip install --pre matplotlib)? We made some changes is how the application from osx is created on the mpl side.

[ianthomas23]

This is a long-standing problem in ipykernel affecting a number of downstream libraries. This issue is ipython/ipykernel#1124 and there is a fix in PR ipython/ipykernel#1237.