Add msvcp140.dll to Windows 64 bit wheels

[cmarmo]

Reference Issues/PRs

Addresses #24612 together with #24627

What does this implement/fix? Explain your changes.

Vendor a missing library to the Windows 64bit wheel.

Any other comments?

Was previously in #24446.

[cmarmo]

Note that the current failure is no longer the missing library but a test failure as reported in #24446.

[thomasjpfan]

According to #24612 (comment), there is a way to remove our dependency on msvcp140.dll. In the long term, I think that is the best option.

In the short term, I am okay with PR vendoring msvcp140.dll, so we can move forward with the Python 3.11 wheels.

The failure on Windows Python 3.10 is concerning. I see that is also happening in the Python 3.11 PR. If no one picks this up, then I'll start investigating it tomorrow.

[cmarmo]

The failure on Windows Python 3.10 is concerning. I see that is also happening in the Python 3.11 PR. If no one picks this up, then I'll start investigating it tomorrow.

Thanks @thomasjpfan. Please note that it is the same failure than in linux python 3.11 and MacosX python3.11.

[thomasjpfan]

I was able to reproduce the Linux failure locally with debug symbols. Here is the backtrace:

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7de3859 in __GI_abort () at abort.c:79
#2  0x00007ffff7e4e26e in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7f78298 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#3  0x00007ffff7e562fc in malloc_printerr (str=str@entry=0x7ffff7f7a600 "free(): invalid next size (fast)") at malloc.c:5347
#4  0x00007ffff7e57bac in _int_free (av=0x7ffff7fadb80 <main_arena>, p=0x4893f50, have_lock=0) at malloc.c:4249
#5  0x00007ffff5afe2e1 in PyDataMem_UserFREE () from /home/thomasfan/Desktop/scikit-learn-5/.venv/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so
#6  0x00007ffff5b0178e in array_dealloc () from /home/thomasfan/Desktop/scikit-learn-5/.venv/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so

There is an error with malloc: "free(): invalid next size (fast)". pytest crashes with the same error, but at different places during a pytest run.

I'll continue investigating tomorrow.

[thomasjpfan]

I looked into this issue more and have not found the root cause yet.

If others are interested in debugging, I created https://github.com/thomasjpfan/scikit-learn-debugging-python3.11 that details the steps for reproducing the issue on Linux.

[cmarmo]

For the sake of completeness, when run with scipy 1.10-dev0 at the beginning of #24446 , linux and MacosX did not fail, Windows64 failed on another test, no issues about threads there.

[thomasjpfan]

Thank you for the hint that SciPy dev works. The issue is in SciPy and the fix is already in SciPy's main branch. I opened scipy/scipy#17224 to track the issue. We need to wait for the fix to be backported to SciPy 1.9.3.

[cmarmo]

The issue is in SciPy and the fix is already in SciPy's main branch. I opened scipy/scipy#17224 to track the issue. We need to wait for the fix to be backported to SciPy 1.9.3.

Wonderful! Thanks for your investigations!
Could this one be merged in the meanwhile, so I can consistently clean #24446? Thanks!

[thomasjpfan]

As noted in #24612 (comment), we can work toward not needing msvcp140.dll in a follow up PR.

In the short them, I am okay with vendoring it for now. LGTM

Edit: According to scipy/scipy#17191, we can not depend no SciPy to load the DLL anymore.

[jjerphan]

Temporary vendoring this DLL works for me. Thank you, @cmarmo!

I think it would be nice to identify what each DLL is needed exactly for to see if we could not depend on them on the long term.