gem version requirements notation for "rational version" compatibility

[ColinDKelley]

I would like to suggest a feature.

When a gemspec wants to express a version requirement, we typically use the '~> ' notation like this:

  spec.add_dependency 'nokogiri', '~> 1.8'

This indicates compatibility following the "rational versioning" as described here: https://github.com/ruby/ruby/blob/master/lib/rubygems/version.rb#L72
(basically the same as Semantic Versioning: https://semver.org/).

Anything >= 1.8 and < 2.0 is compatible.

But suppose a CVE comes out like this one: sparklemotion/nokogiri#1915
Many developers reacted to that CVE by changing the requirement to:

  spec.add_dependency 'nokogiri', '~> 1.10.4'

But that isn't correct, as it precludes an upgrade to 1.11. We need a notation that means >= 1.10.4 and < 2.0.

The only way to do that currently is to use a combination of two requirements:

  spec.add_dependency 'nokogiri', '>= 1.10.4', '< 2.0'

I propose we add a "rational compatible" option that would do the above. We could choose any prefix to mean that. For example, '=>'. Then the CVE requirement could be expressed succinctly:

  spec.add_dependency 'nokogiri', '=> 1.10.4'

And developers could use this "rational compatible" operator as their default for all gem requirements.

The implementation would involve adding one entry to the OPS hash in requirement.rb:

  "=>" =>  lambda { |v, r| v >= r && v._segments.first < (r._segments.first.to_i + 1) }

Please LMK if there's interest. I would be happy to submit a Pull Request including tests and documentation.

I will abide by the code of conduct.

[ColinDKelley]

I found a similar request here: #1919
That started with the idea of changing ~>, which would be a breaking change.
At the end of that ticket there's a suggestion to use ^<version> because npm uses that. But they use ~<version> rather than ~> <version> so it wouldn't be exactly parallel.

But in any case, I'm open to any notation, and happy to make a PR with that, including tests and documentation. I hope it would become the most common one to use. Certainly with Sem Ver becoming the norm, I think it should.

[indirect]

If you're already using ~>, you can keep it, and add an additional version floor, like this: '~> 1.10', '>= 1.10.4'.

Personally, I'm in favor of adding both ^ ("this version or higher, keep major static") and ~ (as an alias to ~>). As you note, the implementation is trivial.

The real problem is that any gems pushed containing those version numbers would be impossible to install for any existing Ruby users. Even if you implement it today, and we were to somehow (ha) ship it next week, it would be years before gems could use it and expect most developers to be able to install those gems.

[ColinDKelley]

If you're already using >, you can keep it, and add an additional version floor, like this: '> 1.10', '>= 1.10.4'

Agreed. That is another alternative for combining requirements.

Personally, I'm in favor of adding both ^ ("this version or higher, keep major static") and ~ (as an alias to ~>).

I like that idea. It brings these more into alignment with an existing convention (npm) vs. inventing another.

The real problem is that any gems pushed containing those version numbers would be impossible to install for any existing Ruby users.

Better late than never. :) If it gains traction, there'd always be an option later to backport it.

[halostatue]

I’m 👎 on either ^ or ~; these are the least semantically interesting things from npm (which is a model to avoid, IMO). The pessimistic versioning ~> 1.0 is IMO clearer. What would be better, IMO will also be a breaking change for older versions of Rubygems, but I think that the Elixir model is better: ~> 1.0 and >= 1.0.4, ~> 1.0 and >= 1.0.4 or ~> 2.0.

[ColinDKelley]

Hi @halostatue, this proposal stands on its own without involving npm.

1. Semantic Versioning (SemVer) has emerged as the preferred way to choose version numbers for gems. That's already documented in rubygems, where it's called "rational versioning".
2. Rubygems currently is lacking a requirements notation for "SemVer compatible". "Pessimistic versioning" notation ~> 1.10 comes close, but unfortunately, it written with a patch version included like ~> 1.10.4, it no longer functions as "SemVer compatible".
3. The shortcoming in (2) isn't just hypothetical. When CVEs are issued against gems, they very often point to a patch version with 2 decimals. For example: here, here, and here. I've seen developers of all levels of seniority make patches to Gemfiles and gemspecs in response to a CVE and accidentally use the patch version "pessimistic" notation like ~> 1.10.4 which is not following SemVer. That over-constrained version requirement is usually introducing a bug: it is prohibiting future minor version bumps.
4. Yes, there are work-arounds to (3) by using compound notation like '~> 1.10', '>= 1.10.4'. But this notation is clumsy to use and not DRY. I wouldn't want to recommend it as best practice.

I propose we address the above by supporting a succinct notation that means "SemVer (rational version) compatible". We would document that as best practice just after the concept of "rational version" is introduced.

Make sense?

I'm totally flexible on the notation we pick for "SemVer" compatible. Just as long as it's succinct.

[deivid-rodriguez]

Regarding compatibility, I was thinking if we could get rubygems.org to do the ~ x.y.z => ~> x.y, >= x.y.z conversion on gem push, so that gemspecs are still served with the old requirements, but clients can start using the new ones.

I tend to recall bundler replacing the compressed gemspec from the index with the one inside the pushed gem or something like that, which would break such an approach, so it would need to be evaluated carefully.

[ColinDKelley]

@deivid-rodriguez That's an interesting approach to get compatibility sooner. In addition to gemspecs through rubygems.org, we'd need to cover bundler itself since it parses these version requirements directly from the Gemfile, right?

[deivid-rodriguez]

Yes, we would need to add some compatibility layer to bundler, or at least explicitly reject using the operator in gems.rb files until we only support rubygems versions implementing it.

Regarding my idea for rubygems.org, I'm thinking it might not be necessary because that's what the required_rubygems_version field is for. We would need to reject any gemspecs using this feature and not declaring a compatible required_rubygems_version field. And also fix #2575 before we start depending more heavily on this field.

[dan42]

For some reason I always have trouble keeping track of what the ~> arrow means, let alone if a new operator is added. What about 1.10+ and 1.10+.4+, that would be so much easier to read (at least for me; ymmv).

[halostatue]

I don’t think a new sigil operator is needed at all. The Elixir version specification to me is extremely clear and builds on the conceptual grounds that Bundler has with multiple version specification: ~> 1.10 and >= 1.10.4. It could be grown to be more powerful (like the Elixir version specification) so that you can say something like (~> 1.10 and >= 1.10.4) or ~> 2.0. That particular type of version specification isn’t possible in RubyGems today, but it’s infinitely more readable than ^1.10.0 or ~1.10.0…whatever those actually end up meaning, because I have to look it up every single time I look at them.

The simple form (~> x and y) is also trivially convertible as per @deivid-rodriguez’s suggestion.