Select Mode of Points Layer broken in 4 dimensions #6680

jules-vanaret · 2024-02-21T13:35:02Z

🐛 Bug Report

When displaying points 4D with the 3D view, selecting the "Select" mode of the Points layer then dragging the mouse across the canvas (not moving point, just dragging the cursor as if to create the usual selection bounding box) does nothing and raises an Assertion error. If a point is clicked instead, it can be moved normally without any error.
Here's the error I get

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
File ~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/superqt/utils/_throttler.py:249, in ThrottledCallable._set_future_result(self=<superqt.utils._throttler.ThrottledCallable object>)
    248 def _set_future_result(self):
--> 249     result = self._func(*self._args[: self._max_args], **self._kwargs)
        self._func = <bound method VispyCanvas._on_mouse_move of <napari._vispy.canvas.VispyCanvas object at 0x7ff6c544fdf0>>
        self = <superqt.utils._throttler.ThrottledCallable object at 0x7ff6ac3160d0>
        self._args[: self._max_args] = (<MouseEvent blocked=False button=1 buttons=[1, 1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=() native=<PyQt5.QtGui.QMouseEvent object at 0x7ff6c545c310> pos=[527 398] press_event=MouseEvent source=<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070> sources=[<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070>] time=1708521467.3701262 type=mouse_move>,)
        self._kwargs = {}
        self._args = (<MouseEvent blocked=False button=1 buttons=[1, 1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=() native=<PyQt5.QtGui.QMouseEvent object at 0x7ff6c545c310> pos=[527 398] press_event=MouseEvent source=<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070> sources=[<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070>] time=1708521467.3701262 type=mouse_move>,)
        self._max_args = 1
    250     self._future.set_result(result)

File /napari/napari/_vispy/canvas.py:448, in VispyCanvas._on_mouse_move(self=<napari._vispy.canvas.VispyCanvas object>, event=<MouseEvent blocked=False button=1 buttons=[1, 1...459070>] time=1708521467.3701262 type=mouse_move>)
    436 def _on_mouse_move(self, event: MouseEvent) -> None:
    437     """Called whenever mouse moves over canvas.
    438 
    439     Parameters
   (...)
    446     None
    447     """
--> 448     self._process_mouse_event(mouse_move_callbacks, event)
        event = <MouseEvent blocked=False button=1 buttons=[1, 1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=() native=<PyQt5.QtGui.QMouseEvent object at 0x7ff6c545c310> pos=[527 398] press_event=MouseEvent source=<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070> sources=[<NapariSceneCanvas (PyQt5) at 0x7ff6c5459070>] time=1708521467.3701262 type=mouse_move>
        self = <napari._vispy.canvas.VispyCanvas object at 0x7ff6c544fdf0>

File /napari/napari/_vispy/canvas.py:409, in VispyCanvas._process_mouse_event(self=<napari._vispy.canvas.VispyCanvas object>, mouse_callbacks=<function mouse_move_callbacks>, event=<ReadOnlyWrapper at 0x7ff69675b680 for MouseEvent>)
    407 layer = self.viewer.layers.selection.active
    408 if layer is not None:
--> 409     mouse_callbacks(layer, event)
        event = <ReadOnlyWrapper at 0x7ff69675b680 for MouseEvent at 0x7ff6954096d0>
        layer = <Points layer 'points' at 0x7ff6eb790c40>
        mouse_callbacks = <function mouse_move_callbacks at 0x7ff6debac940>

File /napari/napari/utils/interactions.py:178, in mouse_move_callbacks(obj=<Points layer 'points'>, event=<ReadOnlyWrapper at 0x7ff69675b680 for MouseEvent>)
    175 obj._persisted_mouse_event[gen].__wrapped__ = event
    176 try:
    177     # try to advance the generator
--> 178     next(gen)
        gen = <generator object select at 0x7ff6978be890>
    179 except StopIteration:
    180     # If done deleted the generator and stored event
    181     del obj._mouse_drag_gen[func]

File /napari/napari/layers/points/_points_mouse_bindings.py:92, in select(layer=<Points layer 'points'>, event=<ReadOnlyWrapper at 0x7ff696756100 for ReadOnlyWrapper>)
     89     layer._drag_box = np.array([layer._drag_start, coord])
     91     # update the drag up and normal vectors on the layer
---> 92     _update_drag_vectors_from_event(layer=layer, event=event)
        layer = <Points layer 'points' at 0x7ff6eb790c40>
        event = <ReadOnlyWrapper at 0x7ff696756100 for ReadOnlyWrapper at 0x7ff69675b680>
     94     layer._set_highlight()
     95 yield

File /napari/napari/layers/points/_points_mouse_bindings.py:195, in _update_drag_vectors_from_event(layer=<Points layer 'points'>, event=<ReadOnlyWrapper at 0x7ff696756100 for ReadOnlyWrapper>)
    191 if n_display == 3:
    192     # if in 3D, set the drag normal and up directions
    193     # get the indices of the displayed dimensions
    194     ndim_world = len(event.position)
--> 195     layer_dims_displayed = layer._world_to_layer_dims(
        layer = <Points layer 'points' at 0x7ff6eb790c40>
        ndim_world = 4
        event = <ReadOnlyWrapper at 0x7ff696756100 for ReadOnlyWrapper at 0x7ff69675b680>
    196         world_dims=event.dims_displayed, ndim_world=ndim_world
    197     )
    199     # get the view direction in displayed data coordinates
    200     layer._drag_normal = layer._world_to_displayed_data_ray(
    201         event.view_direction, layer_dims_displayed
    202     )

File /napari/napari/layers/base/base.py:1640, in Layer._world_to_layer_dims(self=<Points layer 'points'>, world_dims=[1, 2, 3], ndim_world=4)
   1580 def _world_to_layer_dims(
   1581     self, *, world_dims: npt.NDArray, ndim_world: int
   1582 ) -> np.ndarray:
   1583     """Map world dimensions to layer dimensions while maintaining order.
   1584 
   1585     This is used to map dimensions from the full world space defined by ``Dims``
   (...)
   1638         The corresponding layer dimensions with the same ordering as the given world dimensions.
   1639     """
-> 1640     return self._world_to_layer_dims_impl(
        self = <Points layer 'points' at 0x7ff6eb790c40>
        world_dims = [1, 2, 3]
        ndim_world = 4
   1641         world_dims, ndim_world, self.ndim
   1642     )

File /napari/napari/layers/base/base.py:1652, in Layer._world_to_layer_dims_impl(world_dims=<class 'numpy.ndarray'> (3,) int64, ndim_world=4, ndim=4)
   1648 """
   1649 Static for ease of testing
   1650 """
   1651 world_dims = np.asarray(world_dims)
-> 1652 assert world_dims.min() == 0
        world_dims = <class 'numpy.ndarray'> (3,) int64
   1653 assert world_dims.max() == len(world_dims) - 1
   1654 assert world_dims.ndim == 1

AssertionError:

When the cursor is not dragged but clicked, an Axis error raises (axisa: axis -1 is out of bounds for array of dimension 0). I won't show the error stack as it is quite long.

💡 Steps to Reproduce

Display points in 4D. You can use the following snippet:

import napari
import numpy as np

np.random.seed(0)

data = np.random.random((2, 100,100,100))
points = np.random.random((3, 4)) * 100
points[:,0] = 0

viewer = napari.Viewer(ndisplay=3)
viewer.add_image(data)
viewer.add_points(points, size=10)
napari.run()

Go in 3D view mode
Make sure the "Select" mode of the Points Layer is selected. Click and drag your cursor across the canvas.

💡 Expected Behavior

Normally, the action described should draw a thin blue rectangular bounding box that allows selecting the points.

🌎 Environment

napari: 0.1.dev3244+g0747da5
Platform: Linux-5.15.0-91-generic-x86_64-with-glibc2.31
System: Ubuntu 20.04.6 LTS
Python: 3.9.0 (default, Nov 15 2020, 14:28:56) [GCC 7.3.0]
Qt: 5.15.2
PyQt5: 5.15.9
NumPy: 1.24.2
SciPy: 1.9.1
Dask: 2023.4.0
VisPy: 0.14.1
magicgui: 0.7.2
superqt: 0.6.1
in-n-out: 0.1.7
app-model: 0.2.4
npe2: 0.7.4

OpenGL:

GL version: 4.6 (Compatibility Profile) Mesa 21.2.6
MAX_TEXTURE_SIZE: 16384
GL_MAX_3D_TEXTURE_SIZE: 2048

Settings path:

/home/jvanaret/.config/napari/napari-dev_ebc9c46b35c7ad556420cc56947aae90c9e81949/settings.yaml
Plugins:
napari: 0.1.dev3244+g0747da5 (81 contributions)
napari-console: 0.0.9 (0 contributions)
napari-svg: 0.1.10 (2 contributions)
napari-threedee: 0.0.12 (28 contributions)

💡 Additional Context

No response

The text was updated successfully, but these errors were encountered:

dalthviz · 2024-02-21T16:45:14Z

Hi there, gave this a check and seems like changing the logic at _world_to_layer_dims_impl

napari/napari/layers/base/base.py

Lines 1644 to 1658 in 1ee84b3

    
               @staticmethod 
        
               def _world_to_layer_dims_impl( 
        
                   world_dims: npt.NDArray, ndim_world: int, ndim: int 
        
               ) -> npt.NDArray: 
        
                   """ 
        
                   Static for ease of testing 
        
                   """ 
        
                   world_dims = np.asarray(world_dims) 
        
                   assert world_dims.min() == 0 
        
                   assert world_dims.max() == len(world_dims) - 1 
        
                   assert world_dims.ndim == 1 
        
                   offset = ndim_world - ndim 
        
                   order = world_dims - offset 
        
                   order = order[order >= 0] 
        
                   return order - order.min()

for something like:

    @staticmethod
    def _world_to_layer_dims_impl(
        world_dims: npt.NDArray, ndim_world: int, ndim: int
    ) -> npt.NDArray:
        """
        Static for ease of testing
        """
        offset = ndim_world - ndim
        order = np.array(world_dims)
        if offset == 0:
            return order
        if offset < 0:
            return np.concatenate((np.arange(-offset), order - offset))

        return order[order >= offset] - offset

helps.

I think the relevant PR where the logic changed is #6210 where an attempt to simplify the logic for the _world_to_layer_dims logic was done (while checking a fix for #6205). Maybe that logic simplification didn't took into account the points layer selection case handling on that method? 🤔

psobolewskiPhD · 2024-04-06T23:00:45Z

Sorry this fell through the cracks.
Thanks for looking into this @dalthviz
@brisvag do you have any thoughts on this one?

brisvag · 2024-04-08T08:43:08Z

Not right now and it seems @dalthviz has a starting point. It would be good to see a PR with an example to see if this all makes sense!

jules-vanaret added the bug Something isn't working label Feb 21, 2024

dalthviz linked a pull request Apr 8, 2024 that will close this issue

Fix points data selection for 4D #6819

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Select Mode of Points Layer broken in 4 dimensions #6680

Select Mode of Points Layer broken in 4 dimensions #6680

jules-vanaret commented Feb 21, 2024

dalthviz commented Feb 21, 2024 •

edited

psobolewskiPhD commented Apr 6, 2024

brisvag commented Apr 8, 2024

Select Mode of Points Layer broken in 4 dimensions #6680

Select Mode of Points Layer broken in 4 dimensions #6680

Comments

jules-vanaret commented Feb 21, 2024

🐛 Bug Report

💡 Steps to Reproduce

💡 Expected Behavior

🌎 Environment

💡 Additional Context

dalthviz commented Feb 21, 2024 • edited

psobolewskiPhD commented Apr 6, 2024

brisvag commented Apr 8, 2024

dalthviz commented Feb 21, 2024 •

edited