BLD: Do not use clang default to ignore floating point exceptions

[tomgoddard]

Remarkably numpy.log() gives a runtime overflow warning on a float32 array containing values larger than 1e9. Numpy 1.19.4, macOS 10.15.7, MacBookPro15,3. Warning does not appear for float64 array even for much larger values (e.g. 1e35).

Reproducing code example:

import numpy as np
a = np.array([1e9], np.float32)
np.log(a)

Error message:

RuntimeWarning: overflow encountered in log

NumPy/Python version information:

1.19.4 3.8.6 (default, Nov 20 2020, 18:29:40)
[Clang 12.0.0 (clang-1200.0.32.27)]

[seberg]

@tomgoddard, I cannot reproduce this on linux. Since you are on 1.19.x, could you print:

np.core._multiarray_umath.__cpu_features__

Just so we have the information?

The most likely cause will be that you have AVX512 instructions available and our AVX512 code has an error here. If it is in the AV512 path, @r-devulap could you have a look?

Of course it could also be a clang bug in the glibc version...

[tomgoddard]

np.core._multiarray_umath.__cpu_features__ {'MMX': True, 'SSE': True, 'SSE2': True, 'SSE3': True, 'SSSE3': True, 'SSE41': True, 'POPCNT': True, 'SSE42': True, 'AVX': True, 'F16C': True, 'XOP': False, 'FMA4': False, 'FMA3': True, 'AVX2': True, 'AVX512F': False, 'AVX512CD': False, 'AVX512ER': False, 'AVX512PF': False, 'AVX5124FMAPS': False, 'AVX5124VNNIW': False, 'AVX512VPOPCNTDQ': False, 'AVX512VL': False, 'AVX512BW': False, 'AVX512DQ': False, 'AVX512VNNI': False, 'AVX512IFMA': False, 'AVX512VBMI': False, 'AVX512VBMI2': False, 'AVX512BITALG': False, 'AVX512_KNL': False, 'AVX512_KNM': False, 'AVX512_SKX': False, 'AVX512_CLX': False, 'AVX512_CNL': False, 'AVX512_ICL': False, 'VSX': False, 'VSX2': False, 'VSX3': False, 'NEON': False, 'NEON_FP16': False, 'NEON_VFPV4': False, 'ASIMD': False, 'FPHP': False, 'ASIMDHP': False, 'ASIMDDP': False, 'ASIMDFHM': False}

[tomgoddard]

Numpy 1.19.4 was installed using brew on macOS 10.15.7.

[seiko2plus]

could you run np.log(np.array([1e9]*8, np.float32)) and check if the overflow error still remains? newest versions of clang committing aggressive optimization to partial load operations lead to undefined the vector tail on AVX2.

[r-devulap]

I don't see this problem when using NumPy built with gcc 9.3 or clang 10.0. Makes me think it has to something to do with clang 12. Still figuring out how to install clang 12 on my Ubuntu to reproduce the error :/

[tomgoddard]

>> a = np.array([1e9]*8, np.float32) >> np.log(a)

<stdin>:1: RuntimeWarning: overflow encountered in log array([20.723267, 20.723267, 20.723267, 20.723267, 20.723267, 20.723267, 20.723267, 20.723267], dtype=float32)

…

could you run a = np.array([1e9]*8, np.float32) and check if the overflow error still remains? newest versions of clang committing aggressive optimization to partial load operations lead to undefined the vector tail on AVX2.

[tomgoddard]

No overflow warning for smaller values such as 1e8. Python 3.8.6 (default, Nov 20 2020, 18:29:40) [Clang 12.0.0 (clang-1200.0.32.27)] on darwin

>> import numpy as np >> a = np.array([1e8], np.float32) >> np.log(a)

array([18.420681], dtype=float32) No overflow warning with float64

>> import numpy as np >> a = np.array([1e9, 1e20, 1e30], np.float64) >> np.log(a)

array([20.72326584, 46.05170186, 69.07755279])

[r-devulap]

confirmed that this behavior is only with clang (clang 10 and clang 12), and does not happen with gcc. It decides to set an overflow flag for a blend instruction (which is weird and incorrect) x = _mm256_blendv_ps(x, temp, denormal_mask); here:

numpy/numpy/core/src/umath/simd.inc.src

Line 1554 in d7a75e8

x = _mm256_blendv_ps(x, temp, denormal_mask);

[seberg]

@r-devulap thanks for digging this down, I am assuming it is a clang bug. Can we file it there? I am not sure there is a good way for us to work around it in NumPy to begin with...

[r-devulap]

Trying to narrow down to a simple C program to replicate the bug, unsuccessful so far ..

[r-devulap]

Found the culprit! clang optimizes out a blend instruction which was precisely meant to avoid this reported issue of an overflow warning. While the compiler was being clever, it is still a bug.

PR #18030 has a simple work around for this issue. Let me know if that works.

EDIT: didn't mean to include the code snippet.

[matthew-brett]

Just checking - is this bug in clang reported somewhere?

[r-devulap]

@matthew-brett not sure, I couldn't find anything relevant but I didn't spend too much time searching. I am planning on submitting a bug report.

[matthew-brett]

@r-devulap - I'm sure that would be worthwhile, especially as the bug seems to have survived through at least two releases - right?

[r-devulap]

You can play with different version of compilers here https://godbolt.org/z/Yajjcs and looks like it got introduced somewhere between 7.0 (which produces the right code) and 8.0 and is still there in 12.0.

[h-vetinari]

@r-devulap
Thanks for your efforts on this! Could you please reference the clang ticket here once you open it (or let people know if you won't be pursuing it anymore)?

[r-devulap]

So, it turns out this is expected behavior in clang. Clang has a flag to force strict floating point exception compliance by setting -ffp-exception-behavior=strict. The default, however, is set to ignore which means:

The compiler assumes that the exception status flags will not be read and that floating point exceptions will be masked

No wonder I have had trouble with clang before too. The docs has a section on it which you can find by searching for ffp-exception-behavior here: https://clang.llvm.org/docs/UsersManual.html

[mattip]

Should we add that flag if we are on clang?

[seberg]

Sounds like we definitely have to. I wonder if we worked around other clang optimizations that could have been avoided with that flag. Probably just me, but I don't like compilers defaulting to fast-math or this kind of thing...

[r-devulap]

I would think so. Clang would end up disabling some floating point optimizations which will have some negative performance impact. Might be minimal though and probably necessary going forward to ensure FP exception compliance.

[r-devulap]

Probably just me, but I don't like compilers defaulting to fast-math or this kind of thing...

Fortunately clang doesn't default to fast-math. But yes, I don't like this either.

[seberg]

Reopened and changed title. We should probably have a quick look around for clang workaround, but I don't think we need to backport getting rid of those.

[r-devulap]

This was the other time clang give me an incorrect FP flag: #13586 (comment) and the associated PR #13623

[seberg]

Hmmm, I thought I would look into this. But I am honestly not sure where we should be adding this flag, or if we actually should (additionally) even prod Python itself to add this flag. Am I right to think that most clang specific flags currently are inherited from Python itself?

[charris]

What is the status of this?

[seberg]

I think this is important, but I am not very familiar with distutils, and am not sure where this flag should be inserted correctly. (It probably is also correct for SciPy, but do we want to set it by default for scipy as well?).
I am happy to check for the related code cleanups, but not too fond of digging through distutils to find the right place.

It shouldn't matter too much to push it off to 1.21 all known issues due to this are fixed after all. While the flag change is something we could do, the (probably) related code cleanup we do not want to backport anyway probably.

[seberg]

@rgommers I somewhat assume you just know where to put this. Can you point where we would add a clang specific compiler flag to distutils (maybe NumPy specific, although this flag is really also necessary for scipy, I bet).

[rgommers]

In CCompiler_customize: https://github.com/numpy/numpy/blob/main/numpy/distutils/ccompiler.py#L366

maybe NumPy specific, although this flag is really also necessary for scipy, I bet

It's hard to make it NumPy-specific, and I think other libraries should get this flag too, so I'd say just add it unconditionally.

[seiko2plus]

@seberb,

Can you point where we would add a clang specific compiler flag to distutils?

for a quickfix, we can link the flag with one of baseline features flags, etc SSE on x86.`

numpy/numpy/distutils/ccompiler_opt.py

Lines 309 to 314 in ddbff08

	on_x86 = self.cc_on_x86 or self.cc_on_x64
	is_unix = self.cc_is_gcc or self.cc_is_clang
	if on_x86 and is_unix: return dict(
	SSE = dict(flags="-msse"),
	SSE2 = dict(flags="-msse2"),