Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load rubygems plugins from RUBYLIB during bundle install and bundle update #3534

Merged
merged 1 commit into from Oct 23, 2020

Conversation

djmarcin
Copy link
Contributor

@djmarcin djmarcin commented Apr 20, 2020

Description:

Restore load_env_plugins so that plugins stored in RUBYLIB are loaded by bundler.

What was the end-user or developer problem that led to this PR?

rubygems/bundler#4954 broke plugins that are loaded by setting RUBYLIB. The comments in the PR make clear that the intention was to augment the set of plugins being loaded, not remove ENV plugins and replace with only plugins in the standard location. This hook does work when loaded by gem install because of

Gem.load_env_plugins rescue nil
.

This explains why when rbenv attempts to hook into bundler like in the following link, it doesn't work -- this code is never actually executed during bundle install. https://github.com/rbenv/rbenv/blob/0843745be9267de0296de8725c5b7bfcbbb6bddf/rbenv.d/exec/gem-rehash/rubygems_plugin.rb#L14-L35

What is your fix for the problem, implemented in this PR?

This PR restores the load from environment variables that was erroneously removed in rubygems/bundler#4954


Tasks:

  • Describe the problem / feature
  • Write tests
  • Write code to solve the problem
  • Get code review from coworkers / friends

I will abide by the code of conduct.

@welcome
Copy link

welcome bot commented Apr 20, 2020

Thanks for opening a pull request and helping make RubyGems and Bundler better! Someone from the RubyGems team will take a look at your pull request shortly and leave any feedback. Please make sure that your pull request has tests for any changes or added functionality.

We use GitHub Actions to test and make sure your change works functionally and uses acceptable conventions, you can review the current progress of GitHub Actions in the PR status window below.

If you have any questions or concerns that you wish to ask, feel free to leave a comment in this PR or join our #rubygems or #bundler channel on Slack.

For more information about contributing to the RubyGems project feel free to review our CONTRIBUTING guide

@deivid-rodriguez
Copy link
Member

I'm so sorry for the delay getting to this. Wow, this has been broken for such a long time! Can you add a test to ensure we don't regress again in the future?

@djmarcin
Copy link
Contributor Author

djmarcin commented Jul 16, 2020 via email

@deivid-rodriguez
Copy link
Member

Thanks, no rush!

Another option would be to write a high level test in a workflow. Like install rbenv and ruby-build, install ruby, install a Gemfile, make sure rehashing or whatever needs to happen in the rbenv plugin actually happens.

@djmarcin djmarcin force-pushed the fix-env-plugins branch 5 times, most recently from 6e2185a to d5c54cf Compare September 4, 2020 03:15
@djmarcin
Copy link
Contributor Author

djmarcin commented Sep 4, 2020

@deivid-rodriguez Thanks for the pointer to workflows, but I think I managed to get a good lightweight test using the install command suite. It creates a simple plugin that emits a message to stdout and verifies that the bundle install command runs the plugin and the message appears in the output. I verified that the test does not pass before this PR and passes after.

The only test that did not pass was ruby-core / ruby_core (bundler) (pull_request) workflow and I don't really know why (https://github.com/rubygems/rubygems/runs/1069643478?check_suite_focus=true). I expect it has something to do with this workflow having a unique test environment as I see several tests skipped due to being in ruby-core, but I'm not sure that's the right solution here. Do you have any insight?

@djmarcin
Copy link
Contributor Author

djmarcin commented Sep 4, 2020

I figured it out, I needed to be more careful setting the RUBYLIB var for the test and handle the platform specific separator.

@djmarcin
Copy link
Contributor Author

Hi @deivid-rodriguez, hoping you've had some time to think about this and get this landed. It would be nice to get automatic reshimming working again for tools like rbenv and asdf. Hopefully any larger refactor of plugins would not need to block this relatively targeted fix.

@deivid-rodriguez
Copy link
Member

Hei @djmarcin! Thanks a lot for your patience here. I'm putting this PR on my review queue and plan to get to this in the next couple of weeks. The fix itself looks fine from what I remember, so it will land, don't worry :)

Other than reviewing the fix itself, I also wanted to try out rbenv and asdf to verify what end user behaviour there is being fixed. Could you explain what's actually being fixed in both version managers? I use rbenv on a daily basis and it's been literally years without running into any issues, so I wonder whether their rubygems plugin is actually necessary at all.

@djmarcin
Copy link
Contributor Author

Thanks @deivid-rodriguez, appreciate your time.

Other than reviewing the fix itself, I also wanted to try out rbenv and asdf to verify what end user behaviour there is being fixed. Could you explain what's actually being fixed in both version managers? I use rbenv on a daily basis and it's been literally years without running into any issues, so I wonder whether their rubygems plugin is actually necessary at all.

Both version managers use similar "shim" strategies for making installed binaries available on your PATH. They both attempt to use the plugin mechanism to "reshim" (asdf) or "rehash" (rbenv) to pick up new binaries installed by new gems. When you use gem install this works via the expected plugin mechanism, but when you use bundle install it does not (due to the issue fixed by this PR). One reason that you may not see issues is that once a shim exists it does not need to be recreated, even when switching to new versions of ruby or new projects. As long as you don't rm your shims directory, the issue may remain hidden after the first resolution for a given gem.

rbenv works around this by always running a rehash during rbenv init which it encourages you to put in your shell startup script, so each time you open a new shell it fixes any missing shims at the cost of making your new shells open more slowly. This workaround narrows the window of the issue to between bundle install and opening a new shell (which fixes the problem in all shells).

asdf does not have a corresponding workaround, meaning that the issue persists until users run a reshim operation manually.

With this PR, the need for a manual rehash/reshim goes away in both version managers as new gems always trigger the plugin mechanism, whether they come via gem install or bundle install.

Copy link
Member

@deivid-rodriguez deivid-rodriguez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djmarcin I'm really sorry for the delay. I had a look at this and looks great. Thanks for the great explanation of how this improves things for version managers too!

@deivid-rodriguez
Copy link
Member

Could you rebase on top of the latest default branch so that CI runs one more time and I'll merge after it's green 👍.

@deivid-rodriguez deivid-rodriguez changed the title load plugins from RUBYLIB in bundler Load rubygems plugins from RUBYLIB during bundle install and bundle update Oct 23, 2020
@deivid-rodriguez
Copy link
Member

No worries, I rebased it for you, and also added a super tiny tweak to the spec just to make the style a bit more consistent with other specs manipulating RUBYLIB.

@deivid-rodriguez
Copy link
Member

Thanks @djmarcin!! 🎉

@deivid-rodriguez deivid-rodriguez merged commit 5bb5583 into rubygems:master Oct 23, 2020
deivid-rodriguez added a commit that referenced this pull request Dec 7, 2020
Load rubygems plugins from RUBYLIB during `bundle install` and `bundle update`

(cherry picked from commit 5bb5583)
deivid-rodriguez added a commit that referenced this pull request Dec 7, 2020
Load rubygems plugins from RUBYLIB during `bundle install` and `bundle update`

(cherry picked from commit 5bb5583)
deivid-rodriguez added a commit that referenced this pull request Dec 7, 2020
Load rubygems plugins from RUBYLIB during `bundle install` and `bundle update`

(cherry picked from commit 5bb5583)
deivid-rodriguez added a commit that referenced this pull request Dec 7, 2020
Load rubygems plugins from RUBYLIB during `bundle install` and `bundle update`

(cherry picked from commit 5bb5583)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants