Rubygems should not install test files

[anatol]

I see that gemspec has test_files and many gemfiles include tests. I believe it is artifact from gem test times. I think tests in gemfiles are OK (e.g. in case if you decide to reintroduce gem test back #730). But gem install should not install these files, gem tests are useless for users (unless I missed something).

Bottomline: gem install should not install files from test_files.

[drbrain]

This is not something I can change for RubyGems 2.x due to backward compatibility. I have placed this in the Future milestone.

[anatol]

In addition to test files other types of files unneeded at runtime are installed. gems that have native extensions should not install object files, C source files, Makefile. I believe only shared library (*.so) is needed.

[luislavena]

That part depends on the gem author excluding build artifacts from the gems
they build.

Normally pre compiled binaries only ship .so files, and if not, is a bug to
report the gem author.

Sorry for top posting. Sent from mobile.
On Dec 12, 2013 10:05 PM, "Anatol" notifications@github.com wrote:

In addition to test files other types of files unneeded at runtime are
installed. gems that have native extensions should not install object
files, C source files, Makefile. I believe only shared library (*.so) is
needed.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/735#issuecomment-30478268
.

[anatol]

Normally pre compiled binaries only ship .so files

Not sure what do you mean here, but all the examples of gems with native extensions that I see build *.so at install time. Just a few examples: nokogiri, unicorn, rmagic, byebug,... They have ./ext/extconf.rb file that with help of mkmf generates a Makefile. make is called and it builds shared libraries. As far as I see these *.so are copied to ./lib folder where they later loaded into ruby library via 'require'.

Thus you cannot exclude extconf and C sources from gem file. These files are needed at installation time. But these C sources and build artifacts aren't needed after installation. I believe gemspec does not allow to specify such files ("needed in gem but only until installation").

It looks like ./ext folder can be just removed. I tested it with a dozen gems with native extensions on my machine and ruby libraries still work.

[drbrain]

The ext folder cannot be removed at this time for backward compatibility. Some gems may require files out of it through relative path manipulation.

[anatol]

Some gems may require files out of it through relative path manipulation.

Yep, one thing I can think of is a gem that has multiple shared libraries but copies only part of it to lib. Other shared libraries are loaded via absolute path in RPATH. nokogiri compiled without system libraries is one example here.

That is wrong thing IMO - I think all of the shared libraries should be copied from ext/ to lib/ and relative paths should be used. So ext/ can be completely removed. But as you said it is non-compatible change.

[drbrain]

RubyGems 2.2 will copy built extensions to a platform-specific extension directory to support sharing of gems across ruby versions and platforms. RubyGems 3.0 will not copy built extensions to lib.

It is possible RubyGems 3.0 can delete ext/, but that is some time off so I don't want to make a decision now.

[luislavena]

@anatol

Not sure what do you mean here, but all the examples of gems with native extensions that I see build *.so at install time.

You need to see beyond install-time compilation. There are platforms and workflows that will use precompiled gems and not compiled at installation.

As far as I see these *.so are copied to ./lib folder where they later loaded into ruby library via 'require'.

That is mkmf behavior combined with RubyGems.

That is wrong thing IMO - I think all of the shared libraries should be copied from ext/ to lib/ and relative paths should be used.

Talk to gem authors, even better, send patches that correct those issues. Nobody likes a fat gem directory, so I believe any contribution to put that on diet will be very welcome.

As said by @drbrain, these changes cannot be introduced without breaking the entire ecosystem and only can be introduced gradually.

In the meantime, you can send patches to these gem authors 😄

[maxlinc]

Could --no-test (similar to --no-document) be added without breaking backwards compatibility?

Ideally it'd be a flag on both install and build. I agree with @anatol and @luislavena that I don't want a fat gem directory, but ideally I'd also keep gem downloads and unpackaging fast. That would help with the Travis-CI recommendation to speed up your build by excluding non-essential dependencies.

Case in point: yajl-ruby currently produces a 500k gem which expands to 4M. If it excluded test files (spec, examples and benchmark directories) it would be a 50k gem that expands to 528k.

[anatol]

In fact we can go extreme here. At runtime only folders from require_paths are needed. Everything else can be avoided.

Would it be possible to have some kind of --thin installation flag that install only bare minimum needed at runtime?

[maxlinc]

@anatol I like the idea, but it might be too extreme. Two concerns come to mind:

• IANAL - so I'm cautious about skipping the installation of LICENSE files
• Many gems have non-code dependencies that aren't on require_paths, like templates in twbs/bootstrap-sass or resources in hoxworth/json-schema.

[drbrain]

I see nothing that would prevent this from appearing in a 2.x feature release offhand as it merely adds new behavior. Besides the stumbling points mentioned above I also see:

• how many gems fill in the test_files list? This should be populated automatically if possible to make it easier for authors.
• examples are not usually included in a special category so maybe metadata should be used to describe required directories for install instead?
• how easy will it be for authors to be sure a thin or test-only install works (since they typically test with all the files present)

[anatol]

If majority of people need only runtime files then maybe thin installation should be used by default?

And gem spec should contain additional information about runtime files that are not in require_paths (to handle bootstrap-sass example above). Something like

spec.runtime_files = ["templates/**"]

And maybe the same for license files: spec.license_files = 'LICENSE'. (While we are here - it makes sense to make this field mandatory).

Saying this I still think that having tests in *.gem file might be useful to check package at installation time, something like #730.

[drbrain]

A thin installation can't be default in rubygems 2.x

[davexunit]

Chiming in here on this old thread in an attempt to express my
thoughts to people with decision making power. I have been packaging
Ruby gems for the GNU Guix System Distribution for about a year now
and I've found out that including tests in gem archives has been
discouraged[0]. I think this is a major issue for a few reasons. Gem
archives represent source code releases, not binaries. They include
Ruby source code and sometimes C source code that native extensions
will be built from. Source releases for software written in other
languages typically come with their associated test suite. For C
applications that use the GNU autotools, one can run make check to
verify to some degree that the software they built works on their
computer. Python packages do something similar with setup.py. It
is simply good standard practice to ship the test suite with your
source releases. Without test suites, it becomes very difficult for
GNU/Linux distros to have confidence that their Ruby packages actually
work since the CI system cannot check automatically when building from
source.

Now, I agree that gem install shouldn't install test files, but that
doesn't mean that tests should be stricken from release archives.
Maybe test_files wasn't perfect, but I think there needs to be
something like it to distinguish files that are needed for runtime and
other files like the README, Rakefile, C extension source code, and
tests. There of course would be no obligation that everyone's test
suite must be run in the same way like in the gem test days, but it
would give users the chance to run the suite to test the integration
of the gem with their unique environment.

In summary, I would like the RubyGems maintainers to reconsider and
encourage that test suites (including their dependent files such as
the Rakefile to run rake spec, etc.) be included in gem archives.
Thank you.

[0] rails/arel#384

[mvz]

The problem with @davexunit's request is that many gem authors find their gems become too large if they include the full test suite.

I think it would make sense to instead have separate 'source' and 'binary' gem files available for download. (I use the term 'binary' very loosely). The 'source' version would include tests etc. needed for development, while the 'binary' would be more lightweight and suitable for installing with the --thin flag.

[e2]

@davexunit, @Mzv - check out #1364 for a potential solution/alternative.

[anatol]

Hi everyone.

I work on a project related to the topic of this discussion. It is an automatic rubygem to Linux Arch package converter. One of the project goals is to create lightweight Arch packages. As pointed in this thread many rubygems install tests, all sort of development scripts, intermediate compile files, git files that are unneeded at runtime.

To create a thin Arch package I only use bare minimum needed. For most of the packages it is lib/, bin/ gem folders and Gemspec. A small number of gems need nonstandard directories.

Here is the project itself - Quarry https://github.com/anatol/quarry . The most interesting part for you will be https://github.com/anatol/quarry/tree/master/config.pkg with config files that specify gem requirements.

And its corresponding Linux Arch repository - http://pkgbuild.com/~anatolik/quarry/x86_64/ Currently it handles ~450+ popular rubygems.

[Davidslv]

Any updates on this? Thank you

[kaikuchn]

What's the state on this? Should maintainers remove test_files from the specification? Is there a way to install gems without the test files?

[schmijos]

It's painful for me to see how more and more maintainers are removing tests from their gems and I think action is needed. Ruby is open source and gems are not binaries. Each time my IDE can find some bundled tests locally my heart fills with joy. It just happened today when I looked up Capybara's beautiful has_current_path specs. I love my IDE (and it's not GitHub)!

I can see that the distributed kilobytes add up for popular gems, but why did you even choose Ruby in the first place if this is your concern? Is it worth 33K to remove valuable docs from the gem for everyone?

We really need a pragmatic solution to reduce bundle size in production without sacrificing the tests per default. The default should be all the source code, not just transpiled, compiled, tree-shaken, minified and compressed machine-fodder!

[mroach]

It's not only about disk space. It's also about whether or not you want to be shipping unnecessary files to your production environment that can be read and executed like any other Ruby source file. For a while now I've had a release build step that deletes clearly unnecessary directories like benchmarks, doc, examples, spec, test. This is hardly a precision tool though and I have to make exceptions, and it doesn't get everything that it should.

I think it's a fair point that such files have a purpose for development so developers don't have to go back to the source repo and find the version they use to look at tests and embedded documentation files.

Given what's gone on lately with the xz exploit, which was done with test files, I think this deserves some thought.

When I look at some gems I use, there's quite a range in quality of files shipped. The Rails gems for example are neat and tidy, only shipping lib, a license file, and examples. Great!

Not to pick on any gem in particular but I was going in order of necessity, pg ships with its .github dir, .pryrc, Rakefile, its own .gemspec, and other files clearly unnecessary for runtime.

If there's a desire to keep shipping files useful for development but not runtime, perhaps the solution is to do what we do with gem dependencies: specify runtime vs development dependencies. This would allow for automated removal of development-only files during production builds.

A revised gemspec could look something like:

# ship it all, if you want
spec.files = Dir.chdir(File.expand_path("..", __FILE)) { %x(git ls-files -z).split("\x0") }

# new addition?
spec.runtime_files = spec.files.select { _1.start_with?("lib/") }

Having this metadata would enable a gem installation option that only includes files needed for runtime. This could be a great default behaviour for bundle install --deployment