--ext=rust unexpected build and install behavior

[skull-squadron]

Related but different from #6204 which also appears to be a problem.

Describe the problem as clearly as you can

bundle gem example --ext=rust doesn't mirror build and install behavior of native C extensions.

Did you try upgrading rubygems & bundler?

• rubygems 3.4.3
• bundler 2.4.3

Post steps to reproduce the problem

MCE Dockerfile

FROM quay.io/centos/centos:stream9

RUN dnf install -y ruby-devel rubygem-irb '@Development Tools' clang
RUN echo 'gem: --no-document' >~/.gemrc
RUN gem update --system
RUN curl https://sh.rustup.rs -sSf | sh -s -- --profile complete -y --no-modify-path
ENV PATH "/root/.cargo/bin:/root/.local/bin:/root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
WORKDIR /root
RUN bundle gem example --ext=rust --test=rspec --linter=rubocop
WORKDIR /root/example
RUN cargo generate-lockfile
RUN git add -A
RUN sed -i 's/TODO[^"]*//;/_uri/d' example.gemspec
RUN bundle
# Note: cargo unexpectedly builds a native lib here when a regular ruby gem should be built
RUN rake build
# Note: cargo doesn't run here
RUN rake install
WORKDIR /tmp
CMD ["irb -rexample"]

Which command did you run?

docker build .

What were you expecting to happen?

1. Rust native ext compiles at user-level gem install time.

2. If a user doesn't have rust, it's the same as not having a C toolchain. Maintainers can push precompiled --platform flavors of a release to rubygems, or not. Don't try to do parts of both, and especially not for a --platform=ruby gem.

What actually happened?

/usr/share/gems/gems/irb-1.3.5/lib/irb/init.rb:376: warning: LoadError: cannot load such file -- /usr/share/gems/gems/example-0.1.0/lib/example/example

1. Compiled rust native ext at maintainer gem build time.
2. Installed itself into the original source directory without asking.
3. Looked for native ext in the wrong directory.

### Notes

1. Rakefile includes problematic ext.lib_dir = "lib/example". Deployed native exts live in ext/.

2. Eliminating rb-sys not interesting for functionality first.

[skull-squadron]

The Rust extension build process needs to eliminate an extra directory level copied from cargo.

Actual /usr/lib64/gems/ruby/{{gem}}-{{version}}/release/{{gem}}.so

Expected /usr/lib64/gems/ruby/{{gem}}-{{version}}/{{gem}}/{{gem}}.so

[skull-squadron]

On Ubuntu, it's also a problem:

/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/{{gem}}-{{version}}/release/{{gem}}.so

and it's dropping another copy in the source directory:

/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/gems/{{gem}}-{{version}}/lib/{{gem}}.so

Please stop these kind of files and directories pollution and churn when things should go in one place and never change.

It should only be here:

/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/{{gem}}-{{version}}/{{gem}/{{gem}}.so

or perhaps the "old way" was here:

/opt/rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/{{gem}}-{{version}}/{{gem}}.so

[simi]

Related but different from #6204 which also appears to be a problem.

How this differs?

[simi]

Please stop these kind of files and directories pollution and churn when things should go in one place and never change.

Rust extensions are installed in wrong directory, that's know problem (#6204). There are various reasons why two directories are used for extensions these days (for dev purpose and git sources). Feel free to join us at #6204 with your ideas and suggestions.

[skull-squadron]

I fixed this already.

[simi]

I fixed this already.

Can you provide more details? Is #6269 your take on fixing this you mean?

[skull-squadron]

@simi Your "take" didn't do any good because it's still broken.

[indirect]

@steakknife hi. this is your warning under the conduct policy for the RubyGems issue tracker. If you're going to be rude and belligerent, we're just going to block you, even if you are reporting genuine issues. Please engage cooperatively and productively, so we don't have to do that.

[skull-squadron]

?

[simi]

To make it clear, RubyGems 3.4.6 is needed to fix the install problem of Rust (Cargo.toml) extensions.

Steps to reproduce the original problem and also steps to fix it.

# prepare the gem
$ gem install bundler:2.4.6 # install current bundler
$ bundle _2.4.6_ gem --ext=rust hello # generate gem skeleton using specified bundler
$ cd hello # step into gem directory
$ cargo generate-lockfile # generate Cargo.lock
$ git add Cargo.lock # add Cargo.lock to git to make it into spec.files
$ gem build --force # build gem, ignore strict errors like TODO metadata

# install using RubyGems 3.4.5 (before install was fixed)
$ gem update --system 3.4.5 # set RubyGems version
$ gem install hello-0.1.0.gem # install the gem
$ ruby -rhello -e "puts Hello.hello('World')" # running gem method result into require error
/home/retro/.gem/ruby/3.2.0/gems/hello-0.1.0/lib/hello.rb:4:in `require_relative': cannot load such file -- /home/retro/.gem/ruby/3.2.0/gems/hello-0.1.0/lib/hello/hello (LoadError)
        from /home/retro/.gem/ruby/3.2.0/gems/hello-0.1.0/lib/hello.rb:4:in `<top (required)>'
        from <internal:/home/retro/.rubies/ruby-3.2.0/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `require'
        from <internal:/home/retro/.rubies/ruby-3.2.0/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:159:in `rescue in require'
        from <internal:/home/retro/.rubies/ruby-3.2.0/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
<internal:/home/retro/.rubies/ruby-3.2.0/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- hello (LoadError)
        from <internal:/home/retro/.rubies/ruby-3.2.0/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

# install using RubyGems 3.4.6 (with fix included)
$ gem update --system 3.4.5 # set RubyGems version
$ gem install hello-0.1.0.gem # install the gem
$ ruby -rhello -e "puts Hello.hello('World')" # running gem method works as intended
Hello from Rust, World!