Out-of-slice display of points with scale/size is inconsistent

[andy-sweet]

🐛 Bug Report

The video below shows an example with a 3D image layer and two 3D points layers that effectively contain the same data. One the points layers scales the visualization using the layer's transform scale (as with the image) and keeps the size of the points to 1 (named "scale-transform" with red points). The other points layer scales the visualization by scaling the data and size by the equivalent scale factor and keeps the transform scale to 1 (named "scale-data" with blue points).

napari-out-of-slice.mp4

There's a few confusing things here.

1. The peak point size is different for each layer.
   Obviously Points.size is literally different, but I would have expected the equivalent scaling to take care of that. Note that the layer with the smaller size (but bigger transform scale) makes bigger points.

2. The out-of-slice display is different for each layer.
   I think this is because out-of-display slicing does not take into account the layer's scale.

💡 Steps to Reproduce

Here's the code that reproduces the video above.

import napari
import numpy as np

viewer = napari.Viewer()

# Settings
num_pixels = 48
scale_factor = 12
should_scale_data = False

# 3D image with pixels spaced according to the scale factor.
viewer.add_image(np.random.rand(num_pixels, num_pixels, num_pixels), scale=(scale_factor,) * 3)

# 3D points with increasing coordinates ([0, 0, 0], [1, 1, 1], ...).
data = np.repeat(np.arange(num_pixels)[:, np.newaxis], 3, axis=1)

viewer.add_points(
        data * scale_factor,
        name="scale-data",
        face_color="blue",
        opacity=0.5,
        scale=(1,) * 3,
        size=scale_factor,
        out_of_slice_display=True,
)

viewer.add_points(
        data,
        name="scale-transform",
        face_color="red",
        opacity=0.5,
        scale=(scale_factor,) * 3,
        size=1,
        out_of_slice_display=True,
)

napari.run()

💡 Expected Behavior

I would have expected the two points layers to generate the same visualization.

That might just be a confusion on my part though!

🌎 Environment

napari: 0.5.0a2.dev694+g7b46a8446
Platform: macOS-10.16-x86_64-i386-64bit
System: MacOS 14.4.1
Python: 3.11.9 (main, Apr 19 2024, 11:44:45) [Clang 14.0.6 ]
Qt: 5.15.11
PyQt5: 5.15.10
NumPy: 1.26.4
SciPy: 1.13.0
Dask: 2024.5.0
VisPy: 0.14.2
magicgui: 0.8.2
superqt: 0.6.6
in-n-out: 0.2.1
app-model: 0.2.7
npe2: 0.7.5

OpenGL:

• GL version: 2.1 INTEL-22.5.10
• MAX_TEXTURE_SIZE: 16384
• GL_MAX_3D_TEXTURE_SIZE: 2048

Screens:

• screen 1: resolution 3440x1440, scale 1.0
• screen 2: resolution 1440x900, scale 2.0

Optional:

• numba: 0.59.1
• triangle: 20230923

Settings path:

• /Users/asweet/Library/Application Support/napari/napari-dev_94a2c6ecd2bb4e54de7c17f279d1ba0ad46bb93d/settings.yaml

💡 Additional Context

This may be somewhat related to #6729 , though that issue does not use the out-of-slice display.

This came up when debugging CryoET volume + points annotation visualization with https://github.com/chanzuckerberg/napari-cryoet-data-portal. The volumes now have non-unit scales (e.g. 13.48 Angstroms), which first caused a misalignment of some of the visualizations I was doing. Then I ran into the issue described here, as I was using out-of-slice display.

[andy-sweet]

@brisvag : you might be interested in this. And can probably explain why I'm being dumb! As I wrote this up, I felt less confident this is a bug, and more of a PBCAK, but thought it was worth sharing if only for the pulsating points video!

[andy-sweet]

Just noticed #6894 which fixes my first confusion described above of

The peak point size is different for each layer.
Obviously Points.size is literally different, but I would have expected the equivalent scaling to take care of that. Note that the layer with the smaller size (but bigger transform scale) makes bigger points.

There still remains the different out-of-slice behavior, but maybe that is just expected?

[brisvag]

Yeah the different size is indeed the issue fixed by #6894; I'm still surprised I took so long to notice it and pin it down 😢

Out of slicing should be identical in these cases, I agree; I guess we need to update _PointSliceRequest to also take the scale of the layer and use it to rescale sizes here:

napari/napari/layers/points/_slice.py

Line 121 in 6ce45ae

sizes = self.size[:, np.newaxis] / 2