MaxPool with NaNs in the input

[georgen117]

The specification for MaxPool has two outputs:
Outputs (1 - 2)
Y (differentiable) : T
Output data tensor from average or max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Floor value of the dimension is used

Indices (optional, non-differentiable) : I
Indices tensor from max pooling across the input tensor. The dimensions of indices are the same as output tensor. The values in indices of are the indices of the selected values during pooling. The indices are computed as flatten 1-D tensor, and the indices do not consider padding. So the values in indices are in [0, N x C x D1 x ... x Dn).

see: https://github.com/onnx/onnx/blob/master/docs/Operators.md#MaxPool

My question is regarding the MaxPool Indices output:

• What is the return when NaNs are an input? What about if there are two or more NaNs in the local maximum values? Would they all be ignored, or would a random one be returned?

"""
input_shape: [1, 1, 4, 4]
output_shape: [1, 1, 2, 2]
"""
node = onnx.helper.make_node(
    'MaxPool',
    inputs=['x'],
    outputs=['y', 'z'],
    kernel_shape=[2, 2],
    strides=[2, 2],
)
x = np.array([[[
    [1, 2, 3, 4],
    [5, 6, 7, float('nan')],
    [float('nan'), 10, float('nan'), float('nan')],
    [13, float('nan'), float('nan'), float('nan')],
]]]).astype(np.float32)
y = np.array([[[[6, ???],
                [???, float('nan')]]]]).astype(np.float32)
z = np.array([[[[5, ???],
                [???, ???]]]]).astype(np.int64)

what should the ???s be replaced with?

y = np.array([[[[6, 7],
                [13, float('nan')]]]]).astype(np.float32)
z = np.array([[[[5, 6],
                [12, ???]]]]).astype(np.int64)

or some other outputs. I have no clue what the last value in z should be.

[georgen117]

I decided to try this exact scenario out in OnnxRuntime to see what the result was.

the output from the MaxPool was

[6, 7, 
 13, 12]

I don't think the code knows how to handle NaN so the code seems to do something unusual with the output.

for the indices, the output is:

[5, 6],
[13, -5]

A negative index. Which is confusing. I think it is clear that the MaxPool implementation in OnnxRuntime is not capable of handling NaNs.

[georgen117]

Summarizing a conversation that occurred via email:

• This does identify an edge case in the MaxPool operation.
• The behavior will be undefined.
• However, this edge case is not causing any known issues in any real-world models.
• Until we have identified a real-world issue due to this undefined behavior, we will consider the current specification sufficient.
• If a real-world issue is identified then a pull request can be submitted to the ONNX specification to add clarification.