Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get Nokogumbo's test suite green for basic cases #2222

Merged
merged 22 commits into from Apr 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
bd39bd6
test: fix typo
flavorjones Apr 29, 2021
09c3743
test: Nokogumbo's contract expectations
flavorjones Apr 22, 2021
9fb7270
nokogumbo-import: HTML5 and Gumbo ruby modules
flavorjones Apr 22, 2021
fcb35d6
nokogumbo-import: gumbo-parser and C extension
flavorjones Apr 23, 2021
b9d281a
nokogumbo-import: separate ExtensionTask to get to green
flavorjones Apr 23, 2021
ad82b1d
nokogumbo-import: basic contract tests pass
flavorjones Apr 23, 2021
56bb7d9
nokogumbo-import: lift-and-shift the tests
flavorjones Apr 23, 2021
b6163dc
nokogumbo-import: get all tests green on my dev machine
flavorjones Apr 23, 2021
7f74f8c
nokogumbo-import: remove code to use the Ruby API
flavorjones Apr 26, 2021
12bd8af
style: formatting nokogumbo.c
flavorjones Apr 26, 2021
2374879
nokogumbo-import: line numbers are always supported
flavorjones Apr 26, 2021
8fa89ee
nokogumbo-import: restructure as a mini_portile recipe
flavorjones Apr 26, 2021
ce18f83
ci: introduce windows coverage via Github Actions
flavorjones Apr 28, 2021
19c1b02
dep: upgrade mini_portile2 to `~> 2.5.1`
flavorjones Apr 28, 2021
5c6f7c1
nokogumbo-import: get cross-compiling working on windows
flavorjones Apr 26, 2021
1e211b2
nokogumbo-import: get cross-compiling working on darwin
flavorjones Apr 26, 2021
fdd5f77
test: presence/absence of gumbo-parser files in gem packages
flavorjones Apr 28, 2021
cc366c6
chore: check_manifest ignores nokogumbo-import
flavorjones Apr 22, 2021
51dfe01
chore: fix check_manifest task to use fnmatch extended globs
flavorjones Apr 26, 2021
70546a4
style: consistently use File.fnmatch? to operate on gem spec files
flavorjones Apr 29, 2021
cd57221
nokogumbo-import: add gumbo parser source to gemspec
flavorjones Apr 29, 2021
ebd76e7
nokogumbo-import: remove gumbo source from native and jruby gems
flavorjones Apr 29, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/windows.yml
@@ -0,0 +1,50 @@
# this is a work in progress!
name: windows
on:
push:
branches:
- main
pull_request:
types: [opened, synchronize]
branches:
- '*'

jobs:
windows:
name: "windows, sys: ${{ matrix.sys }}, ${{ matrix.ruby }}"

env:
MAKEFLAGS: -j2

runs-on: windows-latest

strategy:
fail-fast: false
matrix:
sys: [ enable, disable ]
ruby: [ "2.5", "2.6", "2.7", "3.0", "mingw" ]

steps:
- name: configure git crlf on windows
run: |
git config --system core.autocrlf false
git config --system core.eol lf
- name: checkout
uses: actions/checkout@v2
- name: load Ruby and bundle install
uses: MSP-Greg/setup-ruby-pkgs@v1
with:
ruby-version: ${{ matrix.ruby }}
mingw: libxml2 libxslt
bundler-cache: true
- uses: actions/cache@v2
if: matrix.sys == 'disable'
with:
path: ports/archives
key: ${{ matrix.os }}-${{ matrix.ruby }}-tarballs-${{ hashFiles('**/dependencies.yml') }}
restore-keys: ${{ matrix.os }}-${{ matrix.ruby }}-tarballs-
- name: bundle exec rake compile
run: |
bundle exec rake compile -- --${{ matrix.sys }}-system-libraries
- name: bundle exec rake test
run: bundle exec rake test
2 changes: 2 additions & 0 deletions .gitignore
Expand Up @@ -9,6 +9,8 @@
/gems/
/lib/nokogiri/**/nokogiri.bundle
/lib/nokogiri/**/nokogiri.so
/lib/nokogumbo/**/nokogumbo.bundle
/lib/nokogumbo/**/nokogumbo.so
/lib/nokogiri/nokogiri.jar
/pkg/
/ports/
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Expand Up @@ -6,6 +6,11 @@ Nokogiri follows [Semantic Versioning](https://semver.org/), please see the [REA

## next / unreleased

### Dependencies

* [MRI] Upgrade mini_portile2 dependency from `~> 2.5.0` to `~> 2.5.1`.


### Changed

* Introduce `Nokogiri::XML::ParseOptions::DEFAULT_XSLT` which adds the libxslt-preferred options of `NOENT | DTDLOAD | DTDATTR | NOCDATA` to `ParseOptions::DEFAULT_XML`.
Expand Down
2 changes: 1 addition & 1 deletion ext/java/nokogiri/XmlDocument.java
Expand Up @@ -443,7 +443,7 @@ private static class DocumentBuilderFactoryHolder
return new_root;
}
if (!(new_root instanceof XmlNode)) {
throw context.runtime.newArgumentError("expected Nokogiri::XML::Node but received " + new_root.getType());
throw context.runtime.newArgumentError("expected Nokogiri::XML::Node but received " + new_root.getType());
}
XmlNode newRoot = asXmlNode(context, new_root);

Expand Down
58 changes: 54 additions & 4 deletions ext/nokogiri/extconf.rb
Expand Up @@ -14,7 +14,7 @@

# The gem version constraint in the Rakefile is not respected at install time.
# Keep this version in sync with the one in the Rakefile !
REQUIRED_MINI_PORTILE_VERSION = "~> 2.5.0"
REQUIRED_MINI_PORTILE_VERSION = "~> 2.5.1"
REQUIRED_PKG_CONFIG_VERSION = "~> 1.1"

# Keep track of what versions of what libraries we build against
Expand Down Expand Up @@ -402,7 +402,7 @@ def process_recipe(name, version, static_p, cross_p)
require 'mini_portile2'
message("Using mini_portile version #{MiniPortile::VERSION}\n")

if name != "libxml2" && name != "libxslt"
unless ["libxml2", "libxslt"].include?(name)
OTHER_LIBRARY_VERSIONS[name] = version
end

Expand Down Expand Up @@ -486,7 +486,7 @@ def process_recipe(name, version, static_p, cross_p)
end
end

message(<<~EOM)
message(<<~EOM) if name != "libgumbo"

The Nokogiri maintainers intend to provide timely security updates, but if
this is a concern for you and want to use your OS/distro system library
Expand All @@ -498,7 +498,7 @@ def process_recipe(name, version, static_p, cross_p)
EOM

message(<<~EOM) if name == 'libxml2'
Note, however, that nokogiri cannot guarantee compatiblity with every
Note, however, that nokogiri cannot guarantee compatibility with every
version of libxml2 that may be provided by OS/package vendors.

EOM
Expand Down Expand Up @@ -868,6 +868,56 @@ def compile
ensure_func("exsltFuncRegister", "libexslt/exslt.h")
end

libgumbo_recipe = process_recipe("libgumbo", "1.0.0-nokogiri", static_p, cross_build_p) do |recipe|
recipe.configure_options = []

class << recipe
def downloaded?
true
end

def extract
target = File.join(tmp_path, "gumbo-parser")
output "Copying gumbo-parser files into #{target}..."
FileUtils.mkdir_p target
FileUtils.cp Dir.glob(File.join(PACKAGE_ROOT_DIR, "gumbo-parser/src/*")), target
end

def configured?
true
end

def install
lib_dir = File.join(port_path, "lib")
inc_dir = File.join(port_path, "include")
FileUtils.mkdir_p([lib_dir, inc_dir])
FileUtils.cp File.join(work_path, "libgumbo.a"), lib_dir
FileUtils.cp Dir.glob(File.join(work_path, "*.h")), inc_dir
end

def compile
cflags = concat_flags(ENV["CFLAGS"], "-fPIC", "-g")

env = {"CC" => gcc_cmd, "CFLAGS" => cflags}
if config_cross_build?
if host =~ /darwin/
env["AR"] = "#{host}-libtool"
env["ARFLAGS"] = "-o"
else
env["AR"] = "#{host}-ar"
end
env["RANLIB"] = "#{host}-ranlib"
end

execute("compile", make_cmd, {env: env})
end
end
end
append_cppflags("-I#{File.join(libgumbo_recipe.path, "include")}")
$libs = $libs + " " + File.join(libgumbo_recipe.path, "lib", "libgumbo.a")
$LIBPATH = $LIBPATH | [File.join(libgumbo_recipe.path, "lib")]
ensure_func("gumbo_parse_with_options", "gumbo.h")

have_func('xmlHasFeature') || abort("xmlHasFeature() is missing.") # introduced in libxml 2.6.21
have_func('xmlFirstElementChild') # introduced in libxml 2.7.3
have_func('xmlRelaxNGSetParserStructuredErrors') # introduced in libxml 2.6.24
Expand Down