Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

f2py fails to link program on windows using v1.26.0 - works on earlier versions of numpy #26352

Open
keithini opened this issue Apr 26, 2024 · 5 comments
Labels

Comments

@keithini
Copy link

Describe the issue:

I have an old fortran program that I access from python using f2py. I have used this successfully on both linux and windows. I have moved successfully to 1.26 on linux but get a link error (1120 unresolved externals) on windows 11. I replicated the problem using a basic fibonacci fortran program (below)

Reproduce the code example:

subroutine fib(a, n)
  use iso_c_binding
   integer(c_int), intent(in) :: n
   integer(c_int), intent(out) :: a(n)
   do i = 1, n
      if (i .eq. 1) then
         a(i) = 0.0d0
      elseif (i .eq. 2) then
         a(i) = 1.0d0
      else
         a(i) = a(i - 1) + a(i - 2)
      end if
   end do
end

Error message:

(aov) C:\fib_test>f2py -c fib1.f90 -m fib1 --backend meson
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.htmlReading fortran codes...
        Reading file 'fib1.f90' (format:free)
Post-processing...
        Block: fib1
                        Block: fib
In: :fib1:fib1.f90:fib
get_useparameters: no module iso_c_binding info used by fib
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
    Building module "fib1"...
    Generating possibly empty wrappers"
    Maybe empty "fib1-f2pywrappers.f"
        Constructing wrapper function "fib"...
          a = fib(n)
    Wrote C/API module "fib1" to file ".\fib1module.c"
The Meson build system
Version: 1.2.1
Source dir: C:\Users\keith\AppData\Local\Temp\tmp679okcip
Build dir: C:\Users\keith\AppData\Local\Temp\tmp679okcip\bbdir
Build type: native build
Project name: fib1
Project version: 0.1
Fortran compiler for the host machine: ifx (intel-llvm-cl 2022.1.0)
Fortran linker for the host machine: xilink.exe xilink 2022.1.0
C compiler for the host machine: cl (msvc 19.29.30146 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30146 for x64")
C linker for the host machine: link link 14.29.30146.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Users\keith\anaconda3\envs\aov\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1

Found ninja-1.10.2 at C:\Users\keith\anaconda3\envs\aov\Library\bin\ninja.EXE
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\Users\keith\anaconda3\envs\aov\Library\bin\ninja.EXE -C C:/Users/keith/AppData/Local/Temp/tmp679okcip/bbdir
ninja: Entering directory `C:/Users/keith/AppData/Local/Temp/tmp679okcip/bbdir'
[4/6] Compiling Fortran object fib1.cp312-win_amd64.pyd.p/fib1-f2pywrappers.f.obj
../fib1-f2pywrappers.f: remark #5133: The input stream is empty
[6/6] Linking target fib1.cp312-win_amd64.pyd
FAILED: fib1.cp312-win_amd64.pyd
"link"  /MACHINE:x64 /OUT:fib1.cp312-win_amd64.pyd fib1.cp312-win_amd64.pyd.p/fib1.f90.obj fib1.cp312-win_amd64.pyd.p/fib1module.c.obj fib1.cp312-win_amd64.pyd.p/fib1-f2pywrappers.f.obj fib1.cp312-win_amd64.pyd.p/42bc432c09fd0a50dc493f9b8a0dabb66e5dad74_.._.._f2py_src_fortranobject.c.obj "/nologo" "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:fib1.cp312-win_amd64.lib" "-shared" "C:\Users\keith\anaconda3\envs\aov\libs\python312.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : warning LNK4044: unrecognized option '/shared'; ignored
   Creating library fib1.cp312-win_amd64.lib and object fib1.cp312-win_amd64.exp
fib1module.c.obj : error LNK2001: unresolved external symbol fib_
fib1.cp312-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "C:\Users\keith\anaconda3\envs\aov\Scripts\f2py-script.py", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 732, in main
    run_compile()
  File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 705, in run_compile
    builder.compile()
  File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 131, in compile
    self.run_meson(self.build_dir)
  File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 124, in run_meson
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['meson', 'compile', '-C', 'bbdir']' returned non-zero exit status 1.

Python and NumPy Versions:

python 3.12.3
Numpy 1.26.0

Runtime Environment:

(aov) C:\Users\keith\temp\fib_test>python -c "import numpy; print(numpy.show_runtime())"
[{'numpy_version': '1.26.0',
'python': '3.12.3 | packaged by Anaconda, Inc. | (main, Apr 19 2024, '
'16:41:55) [MSC v.1916 64 bit (AMD64)]',
'uname': uname_result(system='Windows', node='LENNY2', release='11', version='10.0.22631', machine='AMD64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'filepath': 'C:\Users\keith\anaconda3\envs\aov\Library\bin\mkl_rt.2.dll',
'internal_api': 'mkl',
'num_threads': 8,
'prefix': 'mkl_rt',
'threading_layer': 'intel',
'user_api': 'blas',
'version': '2023.1-Product'}]
None

Context for the issue:

A user has upgraded to python 3.12 and cannot use our product

@ngoldbaum
Copy link
Member

Can you try again with numpy 1.26.4? Numpy 1.26.3 had a number of f2py-related fixes.

@keithini
Copy link
Author

keithini commented Apr 26, 2024 via email

@charris
Copy link
Member

charris commented Apr 27, 2024

@HaoZeke Thoughts?

@HaoZeke
Copy link
Member

HaoZeke commented Apr 28, 2024

@HaoZeke Thoughts?

Thanks for the ping. Since this is windows and related to not finding an executable, it might be fixed by #26248

@keithini could you try to reproduce this bug on main after #26248?

Beyond that I will try to reproduce this on a windows machine if I can

@keithini
Copy link
Author

keithini commented May 6, 2024

Hi
I was not exactly sure what I needed to do to implement "main after #26248" and so I started again with a clean environment and installed numpy 1.26.4. When I tried f2py I got the same errors:

C:\Users\keith\temp\fib_test>f2py -c fib1.f90 -m fib1 --backend meson
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.html
Reading fortran codes...
Reading file 'fib1.f90' (format:free)
Post-processing...
Block: fib1
Block: fib
In: :fib1:fib1.f90:fib
get_useparameters: no module iso_c_binding info used by fib
Applying post-processing hooks...
character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
Building module "fib1"...
Generating possibly empty wrappers"
Maybe empty "fib1-f2pywrappers.f"
Constructing wrapper function "fib"...
a = fib(n)
Wrote C/API module "fib1" to file ".\fib1module.c"
The Meson build system
Version: 1.3.1
Source dir: C:\Users\keith\AppData\Local\Temp\tmpjd_uuokj
Build dir: C:\Users\keith\AppData\Local\Temp\tmpjd_uuokj\bbdir
Build type: native build
Project name: fib1
Project version: 0.1
Fortran compiler for the host machine: ifort (intel-cl 2021.6.0)
Fortran linker for the host machine: xilink.exe xilink 2021.6.0
C compiler for the host machine: cl (msvc 19.29.30146 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30146 for x64")
C linker for the host machine: link link 14.29.30146.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\mesonmain.py", line 194, in run
return options.run_func(options)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 358, in run
app.generate()
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 181, in generate
return self._generate(env, capture, vslite_ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 226, in _generate
intr.run()
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreter\interpreter.py", line 3004, in run
super().run()
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 172, in run
self.evaluate_codeblock(self.ast, start=1)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 198, in evaluate_codeblock
raise e
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 190, in evaluate_codeblock
self.evaluate_statement(cur)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 208, in evaluate_statement
self.assignment(cur)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 644, in assignment
value = self.evaluate_statement(node.value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 210, in evaluate_statement
return self.method_call(cur)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 559, in method_call
res = obj.method_call(method_name, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreter\interpreterobjects.py", line 830, in method_call
ret = method(state, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 108, in wrapped
ret = f(*wrapped_args, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 260, in wrapper
return f(*nargs, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 579, in wrapper
return f(*wrapped_args, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 515, in find_installation
python = self._find_installation_impl(state, display_name, name_or_path, required)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 471, in _find_installation_impl
if python.sanity(state):
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 83, in sanity
ret = super().sanity()
^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\dependencies\python.py", line 120, in sanity
p, stdout, stderr = mesonlib.Popen_safe(cmd, env=env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\utils\universal.py", line 1526, in Popen_safe
p, o, e = Popen_safe_legacy(args, write=write, stdin=stdin, stdout=stdout, stderr=stderr, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\utils\universal.py", line 1548, in Popen_safe_legacy
p = subprocess.Popen(args, universal_newlines=False, close_fds=False,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\subprocess.py", line 1026, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\keith\anaconda3\envs\aov2\Lib\subprocess.py", line 1538, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified

ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov2\Scripts\f2py-script.py", line 10, in
sys.exit(main())
^^^^^^
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\numpy\f2py\f2py2e.py", line 766, in main
run_compile()
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\numpy\f2py\f2py2e.py", line 738, in run_compile
builder.compile()
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\numpy\f2py_backends_meson.py", line 178, in compile
self.run_meson(self.build_dir)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\numpy\f2py_backends_meson.py", line 171, in run_meson
self._run_subprocess_command(setup_command, build_dir)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\numpy\f2py_backends_meson.py", line 167, in _run_subprocess_command
subprocess.run(command, cwd=cwd, check=True)
File "C:\Users\keith\anaconda3\envs\aov2\Lib\subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['meson', 'setup', 'bbdir']' returned non-zero exit status 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants