Skip to content

Commit

Permalink
Merge pull request #2125 from sparklemotion/2078-fix-rake-compiler-na…
Browse files Browse the repository at this point in the history
…tive-things

native gems are built with `ExtensionTask.no_native=true`

---

**What problem is this PR intended to solve?**

Specifically, following the advice of @kou at rake-compiler/rake-compiler#171, we set `Rake::ExtensionTask.no_native=true` within the rake-compiler-dock container ("guest") so that:

- the ExtensionTask `cross_compiling` block is called
  - so that we don't package the dependencies' tarballs in /ports
  - so that we don't have mini_portile2 as a dependency (#2078)
  - so that we don't have an extra nokogiri.so/nokogiri.bundle built
    and packaged (#2076)
- we no longer have to hotfix rake-compiler at build time

This also will enable us to more easily do things like removing the C extension source code from the native gem package (#2077).

This patch set also breaks out `lib/nokogiri/version.rb` into two new files:

- `lib/nokogiri/version/constant.rb`
- `lib/nokogiri/version/info.rb`

and `require_relative`s them both from `version.rb`. This is so that Hoe doesn't pull in `REQUIRED_LIBXML_VERSION` from `extconf.rb` after 652c6fd extracted that value into a constant.

This patch set also updates how Darwin native gems are built, to mirror the same rake task structure that's used for Linux and Windows; and it renames to "builder" the rake tasks that used to be "guest".


**Have you included adequate test coverage?**

I'm satisfied with the level of testing we have now on different platforms, though it could always be better. I will add some testing to the packaged/installed gem when I merge #1788 which introduces that test coverage pretty nicely.


**Does this change affect the behavior of either the C or the Java implementations?**

Should only impact how the native (precompiled) gems are built and packaged.
  • Loading branch information
flavorjones committed Dec 3, 2020
2 parents 3da6259 + 440dc4e commit b54c3fb
Show file tree
Hide file tree
Showing 9 changed files with 242 additions and 221 deletions.
81 changes: 41 additions & 40 deletions .hoerc
Expand Up @@ -2,45 +2,46 @@
---
# this regex is what `rake check_manifest` should consider excluded from the gem
exclude: !ruby/regexp '/
(^\.\/
((\.git
|.vagrant
|.yardoc
|concourse
|gems
|ports
|suppressions
|rakelib
|test
|tmp
)\/)
|\.(autotest
|cross_rubies
|editorconfig
|gemtest
|github
|gitignore
|hoerc
|travis\.yml
)
|Gemfile.*
|Manifest.txt
|Rakefile
|appveyor\.yml
|CHANGELOG.md
|CODE_OF_CONDUCT.md
|CONTRIBUTING.md
|ROADMAP.md
|SECURITY.md
|STANDARD_RESPONSES.md
|Vagrantfile
|Y_U_NO_GEMSPEC.md
|C_CODING_STYLE.*
|h1-.*
|[0-9][0-9][0-9][0-9]-.*
|scripts
|sorbet
|patches
(^\.\/(\.git|
\.github|
\.vagrant|
\.yardoc|
concourse|
gems|
patches|
ports|
pkg|
suppressions|
rakelib|
scripts|
sorbet|
test|
tmp
)\/
| (\.autotest|
\.cross_rubies|
\.editorconfig|
\.gemtest|
\.gitignore|
\.hoerc|
\.travis\.yml|
appveyor\.yml|
CHANGELOG\.md|
CODE_OF_CONDUCT\.md|
CONTRIBUTING\.md|
C_CODING_STYLE\.rdoc|
Gemfile.*|
Manifest\.txt|
ROADMAP\.md|
Rakefile|
SECURITY\.md|
STANDARD_RESPONSES\.md|
Vagrantfile|
Y_U_NO_GEMSPEC\.md|
[0-9]+-.*|
[a-z\.]+\.(log|out)
)$
)
|\.gitkeep
|
(.*\/nokogiri\.(so|bundle))$
/x'
2 changes: 2 additions & 0 deletions Manifest.txt
Expand Up @@ -188,6 +188,8 @@ lib/nokogiri/html/sax/push_parser.rb
lib/nokogiri/jruby/dependencies.rb
lib/nokogiri/syntax_error.rb
lib/nokogiri/version.rb
lib/nokogiri/version/constant.rb
lib/nokogiri/version/info.rb
lib/nokogiri/xml.rb
lib/nokogiri/xml/attr.rb
lib/nokogiri/xml/attribute_decl.rb
Expand Down
3 changes: 3 additions & 0 deletions Rakefile
Expand Up @@ -7,8 +7,11 @@ Hoe.plugin :git
Hoe.plugin :markdown

require_relative "rakelib/util"
require_relative "lib/nokogiri/version/constant"

HOE = Hoe.spec "nokogiri" do |hoe|
hoe.version = Nokogiri::VERSION

hoe.author = [
"Mike Dalessio",
"Aaron Patterson",
Expand Down
8 changes: 1 addition & 7 deletions concourse/tasks/gem-test/gem-build.sh
Expand Up @@ -25,13 +25,7 @@ bundle install --local || bundle install
bundle exec rake set-version-to-timestamp

if [ -n "${BUILD_NATIVE_GEM:-}" ] ; then
# TODO remove after https://github.com/rake-compiler/rake-compiler/pull/171 is shipped
find /usr/local/rvm/gems -name extensiontask.rb | while read f ; do
echo "rewriting $f"
sudo sed -i 's/callback.call(spec) if callback/@cross_compiling.call(spec) if @cross_compiling/' $f
done

bundle exec rake gem:x86_64-linux:guest
bundle exec rake gem:x86_64-linux:builder FORCE_CROSS_COMPILING=true
else
# TODO we're only compiling so that we retrieve libxml2/libxslt
# tarballs, we can do better a couple of different ways
Expand Down
156 changes: 2 additions & 154 deletions lib/nokogiri/version.rb
@@ -1,155 +1,3 @@
# frozen_string_literal: true
module Nokogiri
# The version of Nokogiri you are using
VERSION = "1.11.0.rc3"

class VersionInfo # :nodoc:
def jruby?
::JRUBY_VERSION if RUBY_PLATFORM == "java"
end

def engine
defined?(RUBY_ENGINE) ? RUBY_ENGINE : "mri"
end

def loaded_libxml_version
Gem::Version.new(LIBXML_LOADED_VERSION.
scan(/^(\d+)(\d\d)(\d\d)(?!\d)/).first.
collect(&:to_i).
join("."))
end

def compiled_libxml_version
Gem::Version.new LIBXML_COMPILED_VERSION
end

def loaded_libxslt_version
Gem::Version.new(LIBXSLT_LOADED_VERSION.
scan(/^(\d+)(\d\d)(\d\d)(?!\d)/).first.
collect(&:to_i).
join("."))
end

def compiled_libxslt_version
Gem::Version.new LIBXSLT_COMPILED_VERSION
end

def libxml2?
defined?(LIBXML_COMPILED_VERSION)
end

def libxml2_has_iconv?
defined?(LIBXML_ICONV_ENABLED) && LIBXML_ICONV_ENABLED
end

def libxml2_using_system?
!libxml2_using_packaged?
end

def libxml2_using_packaged?
NOKOGIRI_USE_PACKAGED_LIBRARIES
end

def warnings
warnings = []

if libxml2?
if compiled_libxml_version != loaded_libxml_version
warnings << "Nokogiri was built against libxml version #{compiled_libxml_version}, but has dynamically loaded #{loaded_libxml_version}"
end

if compiled_libxslt_version != loaded_libxslt_version
warnings << "Nokogiri was built against libxslt version #{compiled_libxslt_version}, but has dynamically loaded #{loaded_libxslt_version}"
end
end

warnings
end

def to_hash
{}.tap do |vi|
vi["warnings"] = []
vi["nokogiri"] = Nokogiri::VERSION
vi["ruby"] = {}.tap do |ruby|
ruby["version"] = ::RUBY_VERSION
ruby["platform"] = ::RUBY_PLATFORM
ruby["gem_platform"] = ::Gem::Platform.local.to_s
ruby["description"] = ::RUBY_DESCRIPTION
ruby["engine"] = engine
ruby["jruby"] = jruby? if jruby?
end

if libxml2?
vi["libxml"] = {}.tap do |libxml|
if libxml2_using_packaged?
libxml["source"] = "packaged"
libxml["patches"] = NOKOGIRI_LIBXML2_PATCHES
else
libxml["source"] = "system"
end
libxml["compiled"] = compiled_libxml_version.to_s
libxml["loaded"] = loaded_libxml_version.to_s
libxml["iconv_enabled"] = libxml2_has_iconv?
end

vi["libxslt"] = {}.tap do |libxslt|
if libxml2_using_packaged?
libxslt["source"] = "packaged"
libxslt["patches"] = NOKOGIRI_LIBXSLT_PATCHES
else
libxslt["source"] = "system"
end
libxslt["compiled"] = compiled_libxslt_version.to_s
libxslt["loaded"] = loaded_libxslt_version.to_s
end

vi["warnings"] = warnings
elsif jruby?
vi["xerces"] = Nokogiri::XERCES_VERSION
vi["nekohtml"] = Nokogiri::NEKO_VERSION
end
end
end

def to_markdown
begin
require "psych"
rescue LoadError
end
require "yaml"
"# Nokogiri (#{Nokogiri::VERSION})\n" +
YAML.dump(to_hash).each_line.map { |line| " #{line}" }.join
end

# FIXME: maybe switch to singleton?
@@instance = new
@@instance.warnings.each do |warning|
warn "WARNING: #{warning}"
end
def self.instance; @@instance; end
end

def self.uses_libxml?(requirement = nil) # :nodoc:
return false unless VersionInfo.instance.libxml2?
return true unless requirement
return Gem::Requirement.new(requirement).satisfied_by?(VersionInfo.instance.loaded_libxml_version)
end

def self.jruby? # :nodoc:
VersionInfo.instance.jruby?
end

# Ensure constants used in this file are loaded - see #1896
if Nokogiri.jruby?
require "nokogiri/jruby/dependencies"
end
begin
RUBY_VERSION =~ /(\d+\.\d+)/
require "nokogiri/#{$1}/nokogiri"
rescue LoadError
require "nokogiri/nokogiri"
end

# More complete version information about libxml
VERSION_INFO = VersionInfo.instance.to_hash
end
require_relative "version/constant"
require_relative "version/info"
5 changes: 5 additions & 0 deletions lib/nokogiri/version/constant.rb
@@ -0,0 +1,5 @@
# frozen_string_literal: true
module Nokogiri
# The version of Nokogiri you are using
VERSION = "1.11.0.rc3"
end

0 comments on commit b54c3fb

Please sign in to comment.