Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to override OpenSSL version w/pkg_config #1412

Closed
x-yuri opened this issue Sep 14, 2017 · 15 comments · Fixed by #2648
Closed

Ability to override OpenSSL version w/pkg_config #1412

x-yuri opened this issue Sep 14, 2017 · 15 comments · Fixed by #2648

Comments

@x-yuri
Copy link

x-yuri commented Sep 14, 2017

Why don't you make use of pkg_config?

Steps to reproduce

$ PKG_CONFIG_PATH=/usr/lib/openssl-1.0/pkgconfig gem install puma -v 3.6.0

Expected behavior

Gem installs successfully

Actual behavior

[yuri@yuri-nb ~] gem install puma -v 3.6.0; n
Building native extensions.  This could take a while...
ERROR:  Error installing puma:
        ERROR: Failed to build gem native extension.

    current directory: /home/yuri/.gem/ruby/2.3.4/gems/puma-3.6.0/ext/puma_http11
/home/yuri/.rubies/ruby-2.3.4/bin/ruby -r ./siteconf20170914-19829-17mwin9.rb extconf.rb
checking for BIO_read() in -lcrypto... yes
checking for SSL_CTX_new() in -lssl... yes
checking for openssl/bio.h... yes
creating Makefile

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/yuri/.gem/ruby/2.3.4/extensions/x86_64-linux/2.3.0-static/puma-3.6.0/mkmf.log

current directory: /home/yuri/.gem/ruby/2.3.4/gems/puma-3.6.0/ext/puma_http11
make "DESTDIR=" clean

current directory: /home/yuri/.gem/ruby/2.3.4/gems/puma-3.6.0/ext/puma_http11
make "DESTDIR="
compiling io_buffer.c
compiling mini_ssl.c
mini_ssl.c: In function ‘get_dh1024’:
mini_ssl.c:90:5: error: dereferencing pointer to incomplete type ‘DH {aka struct dh_st}’
   dh->p = BN_bin2bn(dh1024_p, sizeof(dh1024_p), NULL);
     ^
mini_ssl.c: In function ‘engine_init_server’:
mini_ssl.c:139:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   ID sym_cert = rb_intern("cert");
   ^
mini_ssl.c:144:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   ID sym_ca = rb_intern("ca");
   ^
mini_ssl.c:166:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   DH *dh = get_dh1024();
   ^
mini_ssl.c:170:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   EC_KEY *ecdh = EC_KEY_new_by_curve_name(NID_secp521r1);
   ^
mini_ssl.c: In function ‘engine_init_client’:
mini_ssl.c:197:3: warning: ‘DTLSv1_method’ is deprecated [-Wdeprecated-declarations]
   conn->ctx = SSL_CTX_new(DTLSv1_method());
   ^
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from mini_ssl.c:15:
/usr/include/openssl/ssl.h:1630:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *DTLSv1_method(void)) /* DTLSv1.0 */
 ^
make: *** [Makefile:239: mini_ssl.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/yuri/.gem/ruby/2.3.4/gems/puma-3.6.0 for inspection.
Results logged to /home/yuri/.gem/ruby/2.3.4/extensions/x86_64-linux/2.3.0-static/puma-3.6.0/gem_make.out

System configuration

Ruby version: 2.3.4p301
Puma version: 3.6.0

@jc00ke
Copy link
Contributor

jc00ke commented Jul 7, 2018

current directory: /home/jesse/.asdf/installs/ruby/2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.3.0/ext/puma_http11
/home/jesse/.asdf/installs/ruby/2.3.1/bin/ruby -r ./siteconf20180706-736-l02lxz.rb extconf.rb
checking for BIO_read() in -lcrypto... yes
checking for SSL_CTX_new() in -lssl... yes
checking for openssl/bio.h... yes
creating Makefile

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/jesse/.asdf/installs/ruby/2.3.1/lib/ruby/gems/2.3.0/extensions/x86_64-linux/2.3.0-static/puma-3.3.0/mkmf.log

current directory: /home/jesse/.asdf/installs/ruby/2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.3.0/ext/puma_http11
make "DESTDIR=" clean

current directory: /home/jesse/.asdf/installs/ruby/2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.3.0/ext/puma_http11
make "DESTDIR="
compiling mini_ssl.c
mini_ssl.c: In function ‘get_dh1024’:
mini_ssl.c:90:5: error: dereferencing pointer to incomplete type ‘DH {aka struct dh_st}’
   dh->p = BN_bin2bn(dh1024_p, sizeof(dh1024_p), NULL);
     ^~
mini_ssl.c: In function ‘engine_init_server’:
mini_ssl.c:161:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   DH *dh = get_dh1024();
   ^~
mini_ssl.c:165:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   EC_KEY *ecdh = EC_KEY_new_by_curve_name(NID_secp521r1);
   ^~~~~~
mini_ssl.c: In function ‘engine_init_client’:
mini_ssl.c:192:3: warning: ‘DTLSv1_method’ is deprecated [-Wdeprecated-declarations]
   conn->ctx = SSL_CTX_new(DTLSv1_method());
   ^~~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from mini_ssl.c:15:
/usr/include/openssl/ssl.h:1645:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *DTLSv1_method(void)) /* DTLSv1.0 */
 ^
make: *** [Makefile:239: mini_ssl.o] Error 1

make failed, exit code 2

This would indeed be handy. I'm trying to install Puma in Ruby 2.3.1 with LibreSSL (or OpenSSL 1.0.1f but I can't get Ruby to install correctly, anyway)

@nateberkopec
Copy link
Member

nateberkopec commented Oct 4, 2019

I can't figure this one out. Not a c programmer, and mkmf is pretty confusing! I can get pkg_config to return libressl, but the generated makefile still contains openssl. So I'm not sure where I'm going wrong.

For anyone attempting this, a successful patch will make:

bundle exec ruby -Ilib -e "require 'puma'; require 'puma/minissl'; require 'puma/puma_http11'; Puma::Server.class; Puma::MiniSSL.check; puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION"

...return LibreSSL, not OpenSSL.

@x-yuri
Copy link
Author

x-yuri commented Oct 4, 2019

@nateberkopec Not a C programmer either.

I can get pkg_config to return libressl, but the generated makefile still contains openssl.

Show the code, please. I mean, the way it is now, there's no way to make it search for openssl library in a non-standard location. Or so I think.

For example, on Arch Linux you can install openssl-1.0 alongside openssl-1.1. The former installs its headers into /usr/include/openssl-1.0, the latter into /usr/include/openssl. The same goes for imagemagick, and probably other packages.

And this one:

be ruby -Ilib -e "require 'puma'; require 'puma/minissl'; require 'puma/puma_http11'; Puma::Server.class; Puma::MiniSSL.check; puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION

is supposed to show what exactly? Is be a command or a prompt? I don't see any signs of specifying libressl. Or is it header-location compatilble with openssl?

@nateberkopec
Copy link
Member

nateberkopec commented Oct 4, 2019

ack sorry be is my shell alias for bundle exec.

What I'm saying is that, if you clone down Puma and then:

PKG_CONFIG_PATH=your_libresll_path bundle exec rake compile

...then you should compile Puma with Libressl. If that worked, then...

bundle exec ruby -Ilib -e "require 'puma'; require 'puma/minissl'; require 'puma/puma_http11'; Puma::Server.class; Puma::MiniSSL.check; puts Puma::MiniSSL::OPENSSL_LIBRARY_VERSION"

should return a string saying "LibreSSL"

@MSP-Greg
Copy link
Member

MSP-Greg commented Oct 4, 2019

Not a C programmer either

Might as well make it three.

I briefly tried the code from ruby/openssl, but it didn't work. I know some of the build options available should allow one to link to another openssl version, but I haven't checked. Also, I believe one can set build flags when using Bundler, but I can't recall how...

should return a string saying "LibreSSL"

Does a Ruby built with it show "LibreSSL"? Sorry, dumb question by a Windows guy...

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2019

What I'm saying is that, if you clone down Puma and then:

PKG_CONFIG_PATH=your_libresll_path bundle exec rake compile

...then you should compile Puma with Libressl. If that worked, ...

@nateberkopec Why would it? You don't tell it to use pkg-config in your extconf.rb file.

https://ruby-doc.org/stdlib-2.6.5/libdoc/mkmf/rdoc/MakeMakefile.html#method-i-pkg_config

pkg_config(pkg, option=nil)

Returns compile/link information about an installed library in a tuple of [cflags, ldflags, libs], by using the command found first in the following commands:

  1. If --with-{pkg}-config={command} is given via command line option: {command} {option}

  2. {pkg}-config {option}

  3. pkg-config {option} {pkg}

Where {option} is, for instance, --cflags.

The values obtained are appended to +$CFLAGS+, +$LDFLAGS+ and +$libs+.

You can take a look at the openssl package. From what I can see, you basically need to add the pkg_config line. I'm not sure about the fallback though.

But I tried anyway:

$ PKG_CONFIG_PATH=/home/yuri/src/libressl-2.9.2 bundle exec rake compile
...
checking for BIO_read() in -lcrypto... yes                                                           
checking for SSL_CTX_new() in -lssl... yes                                                           
checking for openssl/bio.h... yes                                                                    
checking for DTLS_method() in openssl/ssl.h... yes                                                   
checking for TLS_server_method() in openssl/ssl.h... yes                           
checking for SSL_CTX_set_min_proto_version in openssl/ssl.h... yes
...
$ ldd ./lib/puma/puma_http11.so | grep libssl
	libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007fed007d7000)

@MSP-Greg
Copy link
Member

MSP-Greg commented Oct 6, 2019

@x-yuri

I messed with pkg_config, just didn't seem to work. But, you might try the following:

gem install puma:3.6.0 -- --with-openssl-dir=/usr/lib/openssl-1.0

EDIT: I'm on windows, but my build system and the version of Ruby I was running both have OpenSSL 1.1.1d installed. When I ran the above using a folder with OpenSSL 1.0.2t, the so file was compiled against the 1.0.2 files, as it's easy to see, as the OpenSSL file names changed between 1.0.2 & 1.1.0...

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2019

@MSP-Greg Not in my case:

$ gem install --install-dir gems puma -- --with-openssl-dir=/home/yuri/src/libressl-2.9.2

$ find -name '*.so'
./gems/gems/puma-4.2.0/lib/puma/puma_http11.so
./gems/gems/puma-4.2.0/ext/puma_http11/puma_http11.so
./gems/gems/nio4r-2.5.2/lib/nio4r_ext.so
./gems/gems/nio4r-2.5.2/ext/nio4r/nio4r_ext.so
./gems/extensions/x86_64-linux/2.6.0-static/puma-4.2.0/puma/puma_http11.so
./gems/extensions/x86_64-linux/2.6.0-static/nio4r-2.5.2/nio4r_ext.so

$ ldd ./gems/gems/puma-4.2.0/lib/puma/puma_http11.so | grep libssl
        libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007fb5fcbbd000)

$ ldd ./gems/gems/puma-4.2.0/ext/puma_http11/puma_http11.so | grep libssl
        libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007f4c19cea000)

$ ldd ./gems/extensions/x86_64-linux/2.6.0-static/puma-4.2.0/puma/puma_http11.so | grep libssl
        libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007f6a883ed000)

@MSP-Greg
Copy link
Member

MSP-Greg commented Oct 6, 2019

@x-yuri

Interesting. Can you install the openssl gem using pkg_config?

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2019

Okay, probably my bad. I had includes in /usr/include/openssl-1.0, and libraries in /use/lib/openssl-1.0. So, this one works:

$ gem install --install-dir gems -V openssl -- --with-openssl-lib=/usr/lib/openssl-1.0 --with-openssl-include=/usr/include/openssl-1.0
$ ldd ./gems/gems/openssl-2.1.2/lib/openssl.so | grep libssl
        libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007fe59d82e000)

But this one doesn't:

$ cat Gemfile
source 'https://rubygems.org'
gem "openssl", '2.0.0'
$ bundle config --local build.openssl --with-openssl-lib=/usr/lib/openssl-1.0 --with-openssl-include=/usr/include/openssl-1.0
$ bundle install --path bundle
$ ldd bundle/ruby/2.6.0/gems/openssl-2.0.0/lib/openssl.so | grep libssl         
        libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007f28662c9000)

But that's a known issue. Passing multiple parameters will supposedly be fixed in bundler-2.1.0.

Now then, this one also works:

$ gem install --install-dir gems puma -- --with-openssl-lib=/usr/lib/openssl-1.0 --with-openssl-include=/usr/include/openssl-1.0; n
$ ldd gems/gems/puma-4.2.0/lib/puma/puma_http11.so | grep libssl
	libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007fd80076a000)

But no way to install it with bundler. With bundler-2.1.0 that will hopefully be fixed.

@MSP-Greg
Copy link
Member

MSP-Greg commented Oct 6, 2019

But no way to install it with bundler. With bundler-2.1.0 that will hopefully be fixed.

Sorry. My default Ruby is trunk/master/head.

You mentioned something about your lib & include folder layout above, I'm glad you got it to (sort of) work.

One question. Did you have any luck using pkg_config with OpenSSL? I'm kind of wrestling with something else, but if you can get it to work, we can probably get changes in Puma's extconf.rb...

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2019

I'm not sure I understand your question. openssl gem has pkg_config(...) line in its extconf.rb. And I can override openssl location this way:

$ PKG_CONFIG_PATH=/usr/lib/openssl-1.0/pkgconfig gem install --install-dir gems openssl; n
$ ldd gems/gems/openssl-2.1.2/lib/openssl.so | grep libssl
	libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007faecdeb3000)

/usr/lib/openssl-1.0/pkgconfig is the directory where *.pc files are located.

Maybe to make it work with puma we've got to add just this:

pkg_config('openssl')

@nateberkopec nateberkopec changed the title Ability to override OpenSSL version Ability to override OpenSSL version w/pkg_config Mar 5, 2020
@naox
Copy link

naox commented Jun 21, 2021

If you can't get PKG_CONFIG_PATH to work then please modify this gem source so env variables like OPENSSL-LIB and OPENSSL-INCLUDE populates --with-openssl-lib --with-openssl-include so sysadmin can set those and not instruct every single user to set --with-openssl-lib= --with-openssl-include ....

@MSP-Greg
Copy link
Member

@naox

If possible, can you try with PR #2648? I assume PKG_CONFIG_PATH works with pkg-config?

@naox
Copy link

naox commented Jun 22, 2021

#2648 works fine. I hope it will get included into main branch soon so my users don't have to use --with-openssl-lib --with-openssl-include=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants