BUG: SciPy requires OpenBLAS even when building against a different library #17736

AndreaCogliati · 2023-01-06T10:42:03Z

Describe your issue.

I tried to build SciPy against Arm Performance Libraries (ArmPL); however, the build fails because of missing OpenBLAS.

For context, I was able to build NumPy against ArmPL using the same site.cfg.

Reproducing Code Example

I used the following `site.cfg` file:

# ArmPL
# -----
[openblas]
libraries = armpl_lp64
library_dirs = /mnt/data/opt/lib
include_dirs = /mnt/data/opt/include
runtime_library_dirs = /mnt/data/opt/lib

Build with pip install .



### Error message

```shell
Processing /mnt/data/andreac/scipy-1.9.3
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      The Meson build system
      Version: 1.0.0
      Source dir: /mnt/data/andreac/scipy-1.9.3
      Build dir: /mnt/data/andreac/scipy-1.9.3/.mesonpy-_dtofmia/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.3
      C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
      C linker for the host machine: cc ld.bfd 2.30
      C++ compiler for the host machine: c++ (gcc 7.5.0 "c++ (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
      C++ linker for the host machine: c++ ld.bfd 2.30
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES (cached)
      Compiler for C supports arguments -Wno-unused-function: YES
      Compiler for C supports arguments -Wno-conversion: YES
      Compiler for C supports arguments -Wno-misleading-indentation: YES
      Compiler for C supports arguments -Wno-incompatible-pointer-types: YES
      Library m found: YES
      Fortran compiler for the host machine: gfortran (gcc 7.5.0 "GNU Fortran (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
      Fortran linker for the host machine: gfortran ld.bfd 2.30
      Compiler for Fortran supports arguments -Wno-conversion: YES
      Program cython found: YES (/tmp/pip-build-env-34ij7s89/overlay/bin/cython)
      Program pythran found: YES (/tmp/pip-build-env-34ij7s89/overlay/bin/pythran)
      Program cp found: YES (/bin/cp)
      Program python found: YES (/mnt/data/opt/bin/python3.9)
      Found pkg-config: /usr/bin/pkg-config (0.29.1)
      Run-time dependency threads found: YES
      Library npymath found: YES
      Library npyrandom found: YES
      Found CMake: /usr/bin/cmake (3.10.2)
      DEPRECATION: CMake support for versions <3.17 is deprecated since Meson 0.62.0.
      |
      |   However, Meson was only able to find CMake 3.10.2.
      |
      |   Support for all CMake versions below 3.17.0 will be removed once
      |   newer CMake versions are more widely adopted. If you encounter
      |   any errors please try upgrading CMake to a newer version first.
      
      WARNING: CMake Toolchain: Failed to determine CMake compilers state
      Run-time dependency openblas found: NO (tried pkgconfig and cmake)
      Run-time dependency openblas found: NO (tried pkgconfig and cmake)
      
      ../../scipy/meson.build:131:0: ERROR: Dependency "OpenBLAS" not found, tried pkgconfig and cmake
      
      A full log can be found at /mnt/data/andreac/scipy-1.9.3/.mesonpy-_dtofmia/build/meson-logs/meson-log.txt
      + meson setup --prefix=/mnt/data/opt /mnt/data/andreac/scipy-1.9.3 /mnt/data/andreac/scipy-1.9.3/.mesonpy-_dtofmia/build --native-file=/mnt/data/andreac/scipy-1.9.3/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

SciPy/NumPy/Python version information

------
os.name='posix'
------
sys.platform='linux'
------
sys.version:
3.9.15 (main, Jan  5 2023, 05:16:31) 
[GCC 7.5.0]
------
sys.prefix:
/mnt/data/opt
------
sys.path=':/mnt/data/opt/lib/python39.zip:/mnt/data/opt/lib/python3.9:/mnt/data/opt/lib/python3.9/lib-dynload:/mnt/data/opt/lib/python3.9/site-packages'
------
Found new numpy version '1.24.1' in /mnt/data/opt/lib/python3.9/site-packages/numpy/__init__.py
Found f2py2e version '1.24.1' in /mnt/data/opt/lib/python3.9/site-packages/numpy/f2py/f2py2e.py
error: module 'numpy.distutils' has no attribute '__version__'
------
Importing numpy.distutils.fcompiler ... ok
------
Checking availability of supported Fortran compilers:
Gnu95FCompiler instance properties:
  archiver        = ['/usr/bin/gfortran', '-cr']
  compile_switch  = '-c'
  compiler_f77    = ['/usr/bin/gfortran', '-Wall', '-g', '-ffixed-form', '-
                    fno-second-underscore', '-fPIC', '-O3', '-funroll-loops']
  compiler_f90    = ['/usr/bin/gfortran', '-Wall', '-g', '-fno-second-
                    underscore', '-fPIC', '-O3', '-funroll-loops']
  compiler_fix    = ['/usr/bin/gfortran', '-Wall', '-g', '-ffixed-form', '-
                    fno-second-underscore', '-Wall', '-g', '-fno-second-
                    underscore', '-fPIC', '-O3', '-funroll-loops']
  libraries       = ['gfortran']
  library_dirs    = ['/usr/lib/gcc/aarch64-linux-gnu/7',
                    '/usr/lib/gcc/aarch64-linux-gnu/7']
  linker_exe      = ['/usr/bin/gfortran', '-Wall', '-Wall']
  linker_so       = ['/usr/bin/gfortran', '-Wall', '-g', '-Wall', '-g', '-
                    shared']
  object_switch   = '-o '
  ranlib          = ['/usr/bin/gfortran']
  version         = LooseVersion ('7')
  version_cmd     = ['/usr/bin/gfortran', '-dumpversion']
Fortran compilers found:
  --fcompiler=gnu95  GNU Fortran 95 compiler (7)
Compilers available for this platform, but not found:
  --fcompiler=absoft   Absoft Corp Fortran Compiler
  --fcompiler=arm      Arm Compiler
  --fcompiler=compaq   Compaq Fortran Compiler
  --fcompiler=fujitsu  Fujitsu Fortran Compiler
  --fcompiler=g95      G95 Fortran Compiler
  --fcompiler=gnu      GNU Fortran 77 compiler
  --fcompiler=intel    Intel Fortran Compiler for 32-bit apps
  --fcompiler=intele   Intel Fortran Compiler for Itanium apps
  --fcompiler=intelem  Intel Fortran Compiler for 64-bit apps
  --fcompiler=lahey    Lahey/Fujitsu Fortran 95 Compiler
  --fcompiler=nag      NAGWare Fortran 95 Compiler
  --fcompiler=nagfor   NAG Fortran Compiler
  --fcompiler=nv       NVIDIA HPC SDK
  --fcompiler=pathf95  PathScale Fortran Compiler
  --fcompiler=pg       Portland Group Fortran Compiler
  --fcompiler=vast     Pacific-Sierra Research Fortran 90 Compiler
Compilers not available on this platform:
  --fcompiler=flang     Portland Group Fortran LLVM Compiler
  --fcompiler=hpux      HP Fortran 90 Compiler
  --fcompiler=ibm       IBM XL Fortran Compiler
  --fcompiler=intelev   Intel Visual Fortran Compiler for Itanium apps
  --fcompiler=intelv    Intel Visual Fortran Compiler for 32-bit apps
  --fcompiler=intelvem  Intel Visual Fortran Compiler for 64-bit apps
  --fcompiler=mips      MIPSpro Fortran Compiler
  --fcompiler=none      Fake Fortran compiler
  --fcompiler=sun       Sun or Forte Fortran 95 Compiler
For compiler details, run 'config_fc --verbose' setup command.
------
Importing numpy.distutils.cpuinfo ... ok
------
CPU information: CPUInfoBase__get_nbits getNCPUs is_64bit ------

The text was updated successfully, but these errors were encountered:

tupui · 2023-01-06T12:16:37Z

Hi @AndreaCogliati, thank you for reporting. I would suggest you have a read at our developer documentation: https://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#development-environment

There are a few build dependencies to install. You should not have to define a site.cfg.

rgommers · 2023-01-06T17:13:20Z

To expand on what @tupui said, we're in the middle of a build system transition from distutils to Meson. site.cfg is for a setup.py-based build (no longer the default, and soon to disappear completely). To select your BLAS library when building a wheel, do:

# ArmPL should be installed with pkg-config or CMake support for it to be found
python -m build -Csetup-args=-Dblas=armpl_lp64 -Csetup-args=-Dlapack=armpl_lp64
pip install dist/scipy*.whl

If there's no pkg-config or CMake detection support for ArmPL, then craft your own armpl_lp64.pc file (with paths, that's kinda analogous to site.cfg) - see #16308 (comment).
I hope that helps, using ArmPL should work with those CLI flags.

The instructions in https://scipy.github.io/devdocs/dev/contributor/meson_advanced.html#select-a-different-blas-or-lapack-library for this need a tweak, the meson-python CLI flags changed. I'll open a PR for that.

AndreaCogliati · 2023-01-07T10:16:18Z

@rgommers thanks! That saved me a lot of reading! I was confused by the site.cfg.example file inside the repo.

I tried your suggestion, but now the build fails with the error Dist currently only works with Git or Mercurial repos.

I downloaded version 1.9.3 source code files because it's the latest version supporting GCC 7.5.0, which I have to use on my system. Do I have to clone the git repo for the build to work?

The build ends in:

+ meson dist --allow-dirty --no-tests --formats gztar
Dist currently only works with Git or Mercurial repos

ERROR Backend subprocess exited when trying to invoke build_sdist

Full log:

* Creating venv isolated environment...
* Installing packages in isolated environment... (Cython>=0.29.32,<3.0, meson-python>=0.9.0, numpy; python_version>='3.12', numpy; python_version>='3.8' and platform_python_implementation=='PyPy', numpy==1.18.5; python_version=='3.8' and (platform_machine!='arm64' or platform_system!='Darwin') and platform_machine!='aarch64' and platform_machine!='loongarch64' and platform_python_implementation != 'PyPy', numpy==1.19.2; python_version=='3.8' and platform_machine=='aarch64' and platform_python_implementation != 'PyPy', numpy==1.19.3; python_version=='3.9' and (platform_machine!='arm64' or platform_system!='Darwin') and platform_machine!='loongarch64' and platform_python_implementation != 'PyPy', numpy==1.21.0; python_version=='3.8' and platform_machine=='arm64' and platform_system=='Darwin', numpy==1.21.0; python_version=='3.9' and platform_machine=='arm64' and platform_system=='Darwin', numpy==1.21.6; python_version=='3.10' and (platform_system!='Windows' and platform_machine!='loongarch64') and platform_python_implementation != 'PyPy', numpy==1.22.0; platform_machine=='loongarch64', numpy==1.22.3; python_version=='3.10' and platform_system=='Windows' and platform_python_implementation != 'PyPy', numpy==1.23.2; python_version=='3.11' and platform_python_implementation != 'PyPy', pybind11>=2.4.3,<2.11.0, pythran>=0.9.12,<0.13.0, wheel<0.38.0)
* Getting build dependencies for sdist...
* Building sdist...
+ meson setup --prefix=/mnt/data/opt /mnt/data/andreac/scipy-1.9.3 /mnt/data/andreac/scipy-1.9.3/.mesonpy-xvfpd5dm/build --native-file=/mnt/data/andreac/scipy-1.9.3/.mesonpy-xvfpd5dm/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2 -Dblas=armpl_lp64 -Dlapack=armpl_lp64
The Meson build system
Version: 1.0.0
Source dir: /mnt/data/andreac/scipy-1.9.3
Build dir: /mnt/data/andreac/scipy-1.9.3/.mesonpy-xvfpd5dm/build
Build type: native build
Project name: SciPy
Project version: 1.9.3
C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the host machine: cc ld.bfd 2.30
C++ compiler for the host machine: c++ (gcc 7.5.0 "c++ (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the host machine: c++ ld.bfd 2.30
Host machine cpu family: aarch64
Host machine cpu: aarch64
Compiler for C supports arguments -Wno-unused-but-set-variable: YES 
Compiler for C supports arguments -Wno-unused-but-set-variable: YES (cached)
Compiler for C supports arguments -Wno-unused-function: YES 
Compiler for C supports arguments -Wno-conversion: YES 
Compiler for C supports arguments -Wno-misleading-indentation: YES 
Compiler for C supports arguments -Wno-incompatible-pointer-types: YES 
Library m found: YES
Fortran compiler for the host machine: gfortran (gcc 7.5.0 "GNU Fortran (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
Fortran linker for the host machine: gfortran ld.bfd 2.30
Compiler for Fortran supports arguments -Wno-conversion: YES 
Program cython found: YES (/tmp/build-env-fna065k1/bin/cython)
Program pythran found: YES (/tmp/build-env-fna065k1/bin/pythran)
Program cp found: YES (/bin/cp)
Program python found: YES (/tmp/build-env-fna065k1/bin/python)
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency threads found: YES
Library npymath found: YES
Library npyrandom found: YES
Run-time dependency armpl_lp64 found: YES 3.9.0
Dependency armpl_lp64 found: YES 3.9.0 (cached)
Program _build_utils/cythoner.py found: YES (/mnt/data/andreac/scipy-1.9.3/scipy/_build_utils/cythoner.py)
Compiler for C supports arguments -Wno-maybe-uninitialized: YES 
Compiler for C supports arguments -Wno-discarded-qualifiers: YES 
Compiler for C supports arguments -Wno-empty-body: YES 
Compiler for C supports arguments -Wno-implicit-function-declaration: YES 
Compiler for C supports arguments -Wno-parentheses: YES 
Compiler for C supports arguments -Wno-switch: YES 
Compiler for C supports arguments -Wno-unused-label: YES 
Compiler for C supports arguments -Wno-unused-variable: YES 
Compiler for C++ supports arguments -Wno-cpp: YES 
Compiler for C++ supports arguments -Wno-deprecated-declarations: YES 
Compiler for C++ supports arguments -Wno-class-memaccess: NO 
Compiler for C++ supports arguments -Wno-format-truncation: YES 
Compiler for C++ supports arguments -Wno-non-virtual-dtor: YES 
Compiler for C++ supports arguments -Wno-sign-compare: YES 
Compiler for C++ supports arguments -Wno-switch: YES 
Compiler for C++ supports arguments -Wno-terminate: YES 
Compiler for C++ supports arguments -Wno-unused-but-set-variable: YES 
Compiler for C++ supports arguments -Wno-unused-function: YES 
Compiler for C++ supports arguments -Wno-unused-local-typedefs: YES 
Compiler for C++ supports arguments -Wno-unused-variable: YES 
Compiler for C++ supports arguments -Wno-int-in-bool-context: YES 
Compiler for Fortran supports arguments -Wno-argument-mismatch: YES 
Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
Compiler for Fortran supports arguments -Wno-intrinsic-shadow: YES 
Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES 
Compiler for Fortran supports arguments -Wno-uninitialized: YES 
Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES 
Compiler for Fortran supports arguments -Wno-unused-label: YES 
Compiler for Fortran supports arguments -Wno-unused-variable: YES 
Compiler for Fortran supports arguments -Wno-tabs: YES 
Compiler for Fortran supports arguments -Wno-argument-mismatch: YES (cached)
Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-label: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-variable: YES (cached)
Compiler for Fortran supports arguments -Wno-tabs: YES (cached)
Checking for function "open_memstream" : NO 
Configuring messagestream_config.h using configuration
Checking for size of "void*" : 8
Compiler for C supports arguments -Wno-unused-variable: YES (cached)
Compiler for C supports arguments -Wno-parentheses: YES (cached)
Compiler for C supports arguments -Wno-unused-label: YES (cached)
Compiler for C supports arguments -Wno-implicit-function-declaration: YES (cached)
Compiler for C supports arguments -Wno-switch: YES (cached)
Checking for size of "void*" : 8
../../scipy/optimize/_highs/meson.build:228: WARNING: Keyword argument "dependencies" defined multiple times.
WARNING: This will be an error in future Meson releases.
Build targets in project: 201

SciPy 1.9.3

  User defined options
    Native files: /mnt/data/andreac/scipy-1.9.3/.mesonpy-xvfpd5dm/.mesonpy-native-file.ini
    debug       : false
    optimization: 2
    prefix      : /mnt/data/opt
    blas        : armpl_lp64
    lapack      : armpl_lp64

Found ninja-1.11.1.git.kitware.jobserver-1 at /mnt/data/opt/bin/ninja
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_ufuncs_pxi_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_ufuncs_pxi_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_ufuncs_pxi_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_ufuncs_pxi_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_ufuncs_pxi_pxd' has more than one output! Using the first one.
WARNING: custom_target '_stats_pxd' has more than one output! Using the first one.
WARNING: custom_target 'cython_special' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_stats_gen_pyx' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_stats_gen_pyx' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_stats_gen_pyx' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_stats_gen_pyx' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_stats_gen_pyx' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_spatial_pxd' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_spatial_pxd' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target 'cython_linalg' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_lib_pxd' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_dummy_init_cyoptimize' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
WARNING: custom_target '_cython_tree' has more than one output! Using the first one.
+ meson dist --allow-dirty --no-tests --formats gztar
Dist currently only works with Git or Mercurial repos

ERROR Backend subprocess exited when trying to invoke build_sdist

rgommers · 2023-01-07T10:44:27Z

I downloaded version 1.9.3 source code files because it's the latest version supporting GCC 7.5.0, which I have to use on my system. Do I have to clone the git repo for the build to work?

It should work from an sdist too. To be sure: you did download an sdist (so scipy-1.9.3.tar.gz) from PyPI or from https://github.com/scipy/scipy/releases/tag/v1.9.3, and not the Source code thing from that same page?

AndreaCogliati · 2023-01-07T10:54:06Z

@rgommers I downloaded scipy-1.9.3.tar.gz from https://github.com/scipy/scipy/releases/tag/v1.9.3

rgommers · 2023-01-07T11:11:31Z

Okay, let me look into that now. We build wheels via sdists regularly, so I'm not sure what is going on there. In the meantime, cloning the git repo and checking out the v1.9.3 tag should get you past this hurdle.

Closes scipygh-17736 [skip cirrus] [skip azp] [skip actions]

AndreaCogliati · 2023-01-07T11:28:51Z

Not sure if it’s relevant, but I was able to build from the same folder (with OpenBLAS) with `pip install .`On Jan 7, 2023, at 12:11 PM, Ralf Gommers ***@***.***> wrote: Okay, let me look into that now. We build wheels via sdists regularly, so I'm not sure what is going on there. In the meantime, cloning the git repo and checking out the v1.9.3 tag should get you past this hurdle. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

rgommers · 2023-01-07T11:31:10Z

To test I downloaded that same sdist and then tried:

$ python -m build scipy-1.9.3.tar.gz -Csetup-args=-Dblas=openblas -Csetup-args=-Dlapack=openblas
ERROR Source scipy-1.9.3.tar.gz is not a directory

That reminded me that build doesn't support building from sdist directly (xref pypa/build#311). So I guess what you did is unpack the tarball - that won't work. So yes, you need to clone the repo in this case.

rgommers · 2023-01-07T11:33:01Z

I opened gh-17743 with doc updates for this issue.

AndreaCogliati · 2023-01-09T17:15:36Z

That reminded me that build doesn't support building from sdist directly (xref pypa/build#311). So I guess what you did is unpack the tarball - that won't work. So yes, you need to clone the repo in this case.

I did that, and it built succesfully. However, it doesn't seem to have linked the ArmPL libraries. Here's the output of scipy.show_config():

>>> import scipy
>>> scipy.show_config()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

rgommers · 2023-01-09T17:32:21Z

That show_config output isn't helpful right now. If the configure stage output of the build says that it picked up ArmPL, it will have linked it correctly. Another way to tell is to run ldd build/scipy/linalg/_fblas.cpython*.so.

AndreaCogliati · 2023-01-09T17:42:58Z

That show_config output isn't helpful right now. If the configure stage output of the build says that it picked up ArmPL, it will have linked it correctly. Another way to tell is to run ldd build/scipy/linalg/_fblas.cpython*.so.

I see. I think I saw the build output mentioning ArmPL. However, I don't have the log anymore. And the build folder is gone as well. Is it in a separate folder, by any chance, i.e., not under the cloned git folder?

rgommers · 2023-01-09T18:16:49Z

You can do python -c "import scipy; print(scipy.__file__)" to figure out where the package in installed, and inspect linalg/_fblas.so under that install.

Closes gh-17736 [skip cirrus] [skip azp] [skip actions]

…y#17743) Closes scipygh-17736 [skip cirrus] [skip azp] [skip actions]

AndreaCogliati added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Jan 6, 2023

j-bowhay added the Build issues Issues with building from source, including different choices of architecture, compilers and OS label Jan 6, 2023

rgommers removed the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Jan 6, 2023

rgommers added a commit to rgommers/scipy that referenced this issue Jan 7, 2023

DOC: improve the docs on using BLAS/LAPACK libraries with Meson

7b91757

Closes scipygh-17736 [skip cirrus] [skip azp] [skip actions]

rgommers mentioned this issue Jan 7, 2023

DOC: improve the docs on using BLAS/LAPACK libraries with Meson #17743

Merged

rgommers mentioned this issue Jan 7, 2023

--config-settings recognizes only the last option pypa/pip#11681

Closed

1 task

rgommers closed this as completed in #17743 Jan 11, 2023

rgommers added a commit that referenced this issue Jan 11, 2023

DOC: improve the docs on using BLAS/LAPACK libraries with Meson (#17743)

174579c

Closes gh-17736 [skip cirrus] [skip azp] [skip actions]

tupui added this to the 1.11.0 milestone Jan 11, 2023

ymzayek mentioned this issue Feb 1, 2023

SciPy installation failing in Documentation builder Action workflow nilearn/nilearn#3480

Closed

tylerjereddy modified the milestones: 1.11.0, 1.10.1 Feb 11, 2023

tylerjereddy pushed a commit to tylerjereddy/scipy that referenced this issue Feb 11, 2023

DOC: improve the docs on using BLAS/LAPACK libraries with Meson (scip…

2b82b27

…y#17743) Closes scipygh-17736 [skip cirrus] [skip azp] [skip actions]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: SciPy requires OpenBLAS even when building against a different library #17736

BUG: SciPy requires OpenBLAS even when building against a different library #17736

AndreaCogliati commented Jan 6, 2023 •

edited

tupui commented Jan 6, 2023 •

edited

rgommers commented Jan 6, 2023

AndreaCogliati commented Jan 7, 2023 •

edited by rgommers

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 7, 2023

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 7, 2023 via email

rgommers commented Jan 7, 2023

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 9, 2023

rgommers commented Jan 9, 2023

AndreaCogliati commented Jan 9, 2023

rgommers commented Jan 9, 2023

BUG: SciPy requires OpenBLAS even when building against a different library #17736

BUG: SciPy requires OpenBLAS even when building against a different library #17736

Comments

AndreaCogliati commented Jan 6, 2023 • edited

Describe your issue.

Reproducing Code Example

SciPy/NumPy/Python version information

tupui commented Jan 6, 2023 • edited

rgommers commented Jan 6, 2023

AndreaCogliati commented Jan 7, 2023 • edited by rgommers

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 7, 2023

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 7, 2023 via email

rgommers commented Jan 7, 2023

rgommers commented Jan 7, 2023

AndreaCogliati commented Jan 9, 2023

rgommers commented Jan 9, 2023

AndreaCogliati commented Jan 9, 2023

rgommers commented Jan 9, 2023

AndreaCogliati commented Jan 6, 2023 •

edited

tupui commented Jan 6, 2023 •

edited

AndreaCogliati commented Jan 7, 2023 •

edited by rgommers