float16 normalization problem with numpy >= 1.21

[jmhessel]

Describe the bug

Hi!

I think sklearn.preprocessing.normalize doesn't work properly on float16 for the latest versions of numpy. I am not sure if this is a bug with numpy or sklearn, but figured folks might want to know. I think for dense matrices, np.einsum is used to compute norms, so the instability might be there. Regardless...

Steps/Code to Reproduce

import numpy as np
import sklearn
print(sklearn.__version__)

## if numpy is 1.20.3 or below, this works
## if numpy is 1.21.X or above, this breaks
print(np.__version__)
import sklearn.preprocessing

np.random.seed(1)
test = np.random.random((1,512)).astype(np.float16)
test_norm = sklearn.preprocessing.normalize(test)
print(np.linalg.norm(test_norm))

#output for sklearn 1.0.1 and numpy 1.21.2 (anything 1.21.+)
#1.0.1
#1.21.2
#13.25

#output for sklearn 1.0.1 and numpy 1.20.3
#1.0.1
#1.20.3
#1.0

Expected Results

regardless of numpy version, np.linalg.norm should be close to 1 after normalization

Actual Results

the norm is 13 for any numpy 1.21.+

Versions

see above --- I'm using the latest sklearn, but if i also use the latest numpy, float16 normalization no longer seems to work.

[ogrisel]

It's bad that we did not have a test to catch this regression earlier. I confirm I can reproduce the exact same results on macos/arm64, so this is not platform specific.

It seems that with numpy 1.21.2 sklearn.preprocessing.normalize(test) returns test without any modification...

[ogrisel]

On numpy 1.21.2:

>>> np.einsum("ij,ij->i", test, test)
array([0.], dtype=float16)

while on previous numpy:

>>> np.einsum("ij,ij->i", test, test)
array([175.6], dtype=float16)

and on both numpy versions:

>>> (test ** 2).sum()
175.6

so there is a regression in np.einsum.

[ogrisel]

I found a minimal reproducer, I will report it upstream.

[ogrisel]

This is seems to have already been reported as: numpy/numpy#20305.

[jmhessel]

Thanks @ogrisel ! Should I close this issue out because it's numpy? or keep it open ?

[glemaitre]

Thanks @ogrisel ! Should I close this issue out because it's numpy? or keep it open ?

We can keep it open just to track the regression.

[thomasjpfan]

Looks like the bug was fixed in numpy and will be backed ported in numpy 1.21.5

XREF: numpy/numpy#20462

[thomasjpfan]

Closing because this issue was fixed in NumPy.