Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1 #880

OleMchls · 2021-02-05T00:24:13Z

Hey there,

I'm trying to get rbnacl to function on my M1, but I am hitting the following error message:

ole@Oles-MBP-2 test % ruby -e "require 'rbnacl'" 
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:10: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:10: warning: previous definition of MINIMUM_LIBSODIUM_VERSION was here
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:11: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:11: warning: previous definition of MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2 was here
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:12: warning: already initialized constant RbNaCl::Sodium::Version::MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2ID
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:12: warning: previous definition of MINIMUM_LIBSODIUM_VERSION_FOR_ARGON2ID was here
Traceback (most recent call last):
	2: from -e:1:in `<main>'
	1: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- rbnacl (LoadError)
	15: from -e:1:in `<main>'
	14: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:156:in `require'
	13: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	12: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `require'
	11: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl.rb:12:in `<top (required)>'
	10: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	 9: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	 8: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:6:in `<top (required)>'
	 7: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:7:in `<module:RbNaCl>'
	 6: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:9:in `<module:Sodium>'
	 5: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:14:in `<module:Version>'
	 4: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium/version.rb:14:in `extend'
	 3: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/rbnacl-7.1.1/lib/rbnacl/sodium.rb:11:in `extended'
	 2: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:99:in `ffi_lib'
	 1: from /Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:99:in `map'
/Users/ole/.asdf/installs/ruby/2.7.2/lib/ruby/gems/2.7.0/gems/ffi-1.14.2/lib/ffi/library.rb:145:in `block in ffi_lib': Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError)
Could not open library 'libsodium.dylib': dlopen(libsodium.dylib, 5): image not found.
Could not open library 'libsodium.so.18': dlopen(libsodium.so.18, 5): image not found.
Could not open library 'libsodium.so.18.dylib': dlopen(libsodium.so.18.dylib, 5): image not found.
Could not open library 'libsodium.so.23': dlopen(libsodium.so.23, 5): image not found.
Could not open library 'libsodium.so.23.dylib': dlopen(libsodium.so.23.dylib, 5): image not found

While at first, this looks like I don't have libsodium installed, I want to point out that I do have it installed via brew.

ole@Oles-MBP-2 test % brew test libsodium          
==> Testing libsodium
==> /usr/bin/clang test.c -I/opt/homebrew/Cellar/libsodium/1.0.18_1/include -L/opt/homebrew/Cellar/libsodium/1.0.18_1/lib -lsodium -o test
==> ./test

But it seems it just does not pick it up. I've started my investigations over here before: RubyCrypto/rbnacl#216

I use asdf which uses ruby-build to build my ruby:

ole@Oles-MBP-2 test % ruby -v                      
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [arm64-darwin20]

Any pointers or workarounds would be appreciated 👏

The text was updated successfully, but these errors were encountered:

elja · 2021-02-06T03:32:30Z

Hello @OleMchls,

I think I'm having same issues on my fresh new macbook air m1. I bought it few days ago for my wife and wanted to try if one of my projects will work well natively on ARM without any Rosseta magic.

I installed homebrew then installed rbenv. I figured that on ARM homebrew installation directory is now /opt/homebrew and all libraries are linked into the /opt/homebrew/lib. Then I installed 2.7.2 ruby ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [arm64-darwin20]. After I installed other software like 'glib' and 'vips' and 'postgresql' and did bundle install without any issues.

However after I wanted to start bin/rails c I've got an error:
Could not open library 'libglib-2.0.0.dylib': dlopen(libglib-2.0.0.dylib, 5): image not found (LoadError)

If I do ls /opt/homebrew/lib I can see that required libraries are there. Then I tried to find a good way to make ffi to look into the /opt/homebrew/lib for the libraries. I tried to set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/DYLD_FALLBACK_LIBRARY_PATH to look into /opt/homebrew/lib but it looks like MacOS and dlopen just ignores it, or maybe I'm doing it wrong (doing it like export DYLD_LIBRARY_PATH=/opt/homebrew/lib/)

!!!!

If you open ffi code you will see that it's trying to resolve libraries that failed to load via dlopen with the following code

path = ['/usr/lib/','/usr/local/lib/','/opt/local/lib/'].find do |pth|
   File.exist?(pth + libname)
end

So I ended up with just linking /opt/homebrew/lib to /opt/local/lib. After doing that my rails console started without any issues. I know this workaround is not perfect but I already spent 4+ hours on this stuff and not really sure if its homebrew or ffi issue.

!!!

superscott · 2021-03-18T18:01:27Z

Can confirm the PR @OleMchls has open #882 fixes this issue.

jasonfb · 2021-07-12T19:28:13Z

also if you happen to have gotten here and upgrading ffi to the fixed version didn't fix your issue, try

brew install libsodium
(doh!)

southwolf · 2022-01-23T01:58:02Z

Should we at least put '/opt/local/lib/' before '/usr/local/lib/' to search M1 native libs first?
In apple silicon machines, when both versions are installed, the native arm64 libs ('/opt/local/lib/') should be the first option, and x64 ('/usr/local/lib/') as a failsafe fallback only when arm64 version is not available.

In my case: I've installed both x64 & arm64 version of glib, but ffi found x64 and then stopped, giving up to try /opt ones

While placing /opt before /usr/local

eregon · 2022-06-27T19:44:46Z

Some thoughts on this, and seeing many PRs trying to solve it: #965 by @dzprofessionerd, #936 by @southwolf and #941 by @davidperched.

The idea of a search path is we should search to find an existing file for which dlopen() succeeds, and keeps trying other paths otherwise.
The current logic seems to only look into the first match in this list of directories, because after the first absolute path try, unless libname.start_with?("/") will skip it. Fixing that would be best and most reliable, then the order would only matter if the library is of the right architecture and multiple times in the search path.

The second thing is why does Homebrew on darwin-aarch64 doesn't add its libraries on the default library search path, such that dlopen("libfoo.dylib") would just find it? Does anyone know that? That would be the expected behavior for any system package manager at least on Linux, it would just work because of using the system library search path.

eregon · 2022-06-27T20:12:34Z

And here is an issue for Homebrew as to why we need to special-case darwin-arm64 and why it does not just work using the system library search path: Homebrew/brew#13481

eregon · 2022-06-27T20:25:33Z

Let's reopen this as it's only solved if there is no Rosetta2 installation, if there is and libraries exist for both arch's it doesn't work.

* Try every prefix, not just the first one where the file exists. The first existing file might be for the wrong architecture or have other issues and cannot be loaded. * Show which directories were searched in error message. * Only consider /opt/homebrew/lib on darwin-aarch64. * Only rescue LoadError and RuntimeError, not Exception. * Refactor in smaller functions to improve readability. * Fixes ffi#880. * Example error message: Could not open library 'notexist': notexist: cannot open shared object file: No such file or directory. (LoadError) Could not open library 'libnotexist.so': libnotexist.so: cannot open shared object file: No such file or directory. Searched in <system library path>, /usr/lib, /usr/local/lib, /opt/local/lib

eregon · 2022-07-17T12:15:16Z

I made a PR with a general fix for #880 (comment): #968

SampsonCrowley · 2022-12-09T20:29:58Z

@here for thoses still waiting on any fix for arm, it seems to work to create a symbolic link from the opt dir of your homebrew prefix to the ffi gem install path.

this also solves the problem of if you have a non-standard homebrew path, (which was brought up apparently as a blocker to #968, which I personally think it shouldn't be, lets fix for the majority at least first)

ex:

ln -sf /opt/homebrew/lib/libsodium.dylib /Users/SampsonCrowley/.rubies/ruby-arm64-2.7.7/lib/ruby/gems/2.7.0/gems/ffi-1.15.5/lib/libsodium.dylib

OleMchls mentioned this issue Feb 12, 2021

dynlib support for homebrew 3 on Apple Silicon M1 #882

Merged

tduehr closed this as completed in #882 Jun 2, 2021

southwolf mentioned this issue Jan 23, 2022

Move /opt before /usr to search for M1 native libs first - solve the conflicts when both x64 and arm64 libs installed #936

Closed

eregon reopened this Jun 27, 2022

eregon mentioned this issue Jul 17, 2022

Refactor library lookup #968

Merged

larskanis closed this as completed in #968 Dec 11, 2022

ptrm mentioned this issue Jan 3, 2023

Makefile / library path modifications to make M1 building work (koreader/koreader#8797) koreader/koreader-base#1570

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1 #880

Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1 #880

OleMchls commented Feb 5, 2021 •

edited

elja commented Feb 6, 2021 •

edited

superscott commented Mar 18, 2021 •

edited by eregon

jasonfb commented Jul 12, 2021 •

edited

southwolf commented Jan 23, 2022 •

edited

eregon commented Jun 27, 2022 •

edited

eregon commented Jun 27, 2022

eregon commented Jun 27, 2022

eregon commented Jul 17, 2022

SampsonCrowley commented Dec 9, 2022 •

edited

Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1 #880

Could not open library 'sodium': dlopen(sodium, 5): image not found. (LoadError) on Apple Silicon M1 #880

Comments

OleMchls commented Feb 5, 2021 • edited

elja commented Feb 6, 2021 • edited

superscott commented Mar 18, 2021 • edited by eregon

jasonfb commented Jul 12, 2021 • edited

southwolf commented Jan 23, 2022 • edited

eregon commented Jun 27, 2022 • edited

eregon commented Jun 27, 2022

eregon commented Jun 27, 2022

eregon commented Jul 17, 2022

SampsonCrowley commented Dec 9, 2022 • edited

OleMchls commented Feb 5, 2021 •

edited

elja commented Feb 6, 2021 •

edited

superscott commented Mar 18, 2021 •

edited by eregon

jasonfb commented Jul 12, 2021 •

edited

southwolf commented Jan 23, 2022 •

edited

eregon commented Jun 27, 2022 •

edited

SampsonCrowley commented Dec 9, 2022 •

edited