BUG: Questionable macOS wheel contents

[bwoodsend]

Describe your issue.

I'm not sure if this really is a bug. Merely I found it very unusual so I thought I'd check if it was deliberate. It has broken our pipelines and whilst that's more due to missasumptions on our end, I want to be sure before I start trying to fix it.

The latest release (1.8.0) of wheels for macOS includes both universal2 and separate architecture variants. Both the arm64 slice of the universal2 wheel and the arm64 wheel targets macOS >= 12 leaving macOS M1 Big Surs to compile from source?

Inside the universal2 wheel there is a mixture of fat and thin dylibs including one i386 slice:

> for file in scipy/.
    printf '%s\t%s\n' $file (llvm-lipo -archs $file)
end
scipy/.dylibs/libgcc_s.1.dylib		x86_64 i386 
scipy/.dylibs/libgcc_s.2.dylib		arm64 
scipy/.dylibs/libgfortran.3.dylib	x86_64 
scipy/.dylibs/libgfortran.5.dylib	arm64 
scipy/.dylibs/libopenblas.0.dylib	x86_64 arm64 
scipy/.dylibs/libquadmath.0.dylib	x86_64 

In particular, I find that seemingly redundant i386 slice and the lack of an arm64 variant of libquadmath unlikely.

Is the lack of an macOS 11 M1 compatible wheel and the above cocktail of binaries types intentional?

Reproducing Code Example

N/A

Error message

N/A

SciPy/NumPy/Python version information

1.8.0 1.22.2

[tylerjereddy]

Can maybe ping @isuruf for opinion maybe. I think we've been leaning toward ditching the universal2 and just shipping the separate wheels long-term?

[tylerjereddy]

Is the lack of an macOS 11 M1 compatible wheel and the above cocktail of binaries types intentional?

To be clear, you should be able to install a binary directly on MacOS M1, if you are using a recent enough OS.

[tylerjereddy]

Yes, you need MacOS 12.x for M1 because of issues on 11.x. I'm sure you could dig up the issues/reasons we had for this.. but they were serious like segfaults with 11.x on M1.

[tylerjereddy]

I'll close this, since it is working as intended--you need MacOS 12.x for M1 binary install b/c of serious segfault issues. I think Ralf worked a bit with Apple folks on that one too.

[tylerjereddy]

From a quick search of our issue tracker: #13409 (comment)

[tylerjereddy]

Maybe a separate issue would be appropriate for the "cocktail" inside the universal2 binary if it can be demonstrated that it actually causes a problem. You didn't report an actual i.e., test failure there?

[tylerjereddy]

I'll leave this open until i.e., Isuru comments on the wheel contents maybe

[bwoodsend]

Maybe a separate issue would be appropriate for the "cocktail" inside the universal2 binary if it can be demonstrated that it actually causes a problem. You didn't report an actual i.e., test failure there?

The cocktail has knocked out PyInstaller support but that's more PyInstaller's problem but other than that it doesn't appear to have broken anything. Like I said, I just wanted to confirm that it is intentional. Given how hard it is to get anything resembling macOS M1 CI/CD, I thought that that wheel might have had to be released untested.

[isuruf]

In particular, I find that seemingly redundant i386 slice

That's because the gfortran toolchain we are using for x86_64 has a libgfortran.dylib that has both x86_64 and i386 slices.
It'll be good to remove that i386 slice to reduce a little bit of space.

and the lack of an arm64 variant of libquadmath unlikely.

This is expected because libquadmath doesn't support arm64/aarch64.

[bwoodsend]

This is expected because libquadmath doesn't support arm64/aarch64.

Ahh so libquadmath is an optional dependency?

[tylerjereddy]

Ok, thanks Isuru--I think I can close this then. Ralf did carefully run our full test suite on his M1 Mac, but @bwoodsend is correct that the CI situation there is unfortunate of course.

Shrinking wheels is of interest to us too, we've had some "bloating" recently I think.

[isuruf]

Ahh so libquadmath is an optional dependency?

Yes. It's only needed for np.float128 support which is only supported in x86_64 and not arm64.

[bwoodsend]

Perfect. Thanks for clearing that up for me. The redundant i386 only wastes 150kB so it's probably not worth pulling teeth over.