Information for Nokogumbo users about the Nokogiri merger #2205

flavorjones · 2021-03-13T21:51:05Z

Hello! If you've arrived at this issue because of a warning message emitted by Nokogumbo, you're in the right place. This issue exists to try to:

explain what action you should take in response
explain a bit more about why you're seeing an error message

👇

How To Resolve The Warnings

If you've installed Nokogiri >= v1.12.0, then you can simply drop Nokogumbo as a dependency. Nokogiri has imported Nokogumbo's complete public API and so this cutover should be painless.

Alternatively, if Nokogumbo is a transitive dependency, then you should be able to bundle update nokogumbo to v2.0.5 to eliminate the constant/method redefined warnings. In this case, until Nokogumbo is removed by the intermediate dependencies, you will continue to see a "Using Nokogiri::HTML5 provided by Nokogiri" warning at require-time.

☝️

A Deeper Explanation for the Curious

Why am I seeing this warning message?

The warning message you're seeing is probably:

NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See #2205 for more information.

You're seeing this message because Nokogumbo has been merged into Nokogiri >= 1.12.0, and you've loaded Nokogumbo >= 2.0.5.

No, that's not the warning I'm seeing.

Or you may be seeing error messages like:

nokogumbo-2.0.4/lib/nokogumbo/html5/document.rb:4: warning: method redefined; discarding old parse
nokogiri-1.12.0/lib/nokogiri/html5/document.rb:25: warning: previous definition of parse was here
nokogumbo-2.0.4/lib/nokogumbo/html5/document.rb:21: warning: method redefined; discarding old read_io
nokogiri-1.12.0/lib/nokogiri/html5/document.rb:42: warning: previous definition of read_io was here
nokogumbo-2.0.4/lib/nokogumbo/html5/document.rb:26: warning: method redefined; discarding old read_memory
nokogiri-1.12.0/lib/nokogiri/html5/document.rb:47: warning: previous definition of read_memory was here
nokogumbo-2.0.4/lib/nokogumbo/html5/document.rb:31: warning: method redefined; discarding old fragment
...
nokogumbo-2.0.4/lib/nokogumbo/html5.rb:13: warning: already initialized constant Nokogiri::HTML5::HTML_NAMESPACE
nokogiri-1.12.0/lib/nokogiri/html5.rb:227: warning: previous definition of HTML_NAMESPACE was here
nokogumbo-2.0.4/lib/nokogumbo/html5.rb:14: warning: already initialized constant Nokogiri::HTML5::MATHML_NAMESPACE
nokogiri-1.12.0/lib/nokogiri/html5.rb:228: warning: previous definition of MATHML_NAMESPACE was here
nokogumbo-2.0.4/lib/nokogumbo/html5.rb:15: warning: already initialized constant Nokogiri::HTML5::SVG_NAMESPACE
nokogiri-1.12.0/lib/nokogiri/html5.rb:229: warning: previous definition of SVG_NAMESPACE was here
...

You're seeing this message for the same reason -- Nokogumbo has been merged into Nokogiri >= 1.12.0 -- but you're running an older Nokogumbo, <= 2.0.4. If you upgrade to Nokogumbo >= 2.0.5 then you'll see the warning from the previous section.

Why did you do this?

Nokogiri <= 1.12.0 did not provide HTML5 support, and it was becoming increasingly clear that the community wanted and expected HTML5 support.

Nokogumbo was the best candidate for providing HTML5 support, but the installation experience for many users was challenging for many of the same reasons that Nokogiri's installation experience was challenging before v1.11.0 delivered precompiled native gems for Linux (#1983) and MacOS (#2063), and we collectively hesitated at placing that gem into the "Rails dependency" hot path.

After a brief conversation in #2064 the two teams reached the decision to merge, in order to better meet the needs of the Ruby community. Nokogiri gets HTML5 support out of the box, Nokogumbo gets shipped as a precompiled native gem to anyone (on CRuby) who needs it, and by shipping together the maintainers can ship more often and more reliably.

You can read #2204 for more information!

How do I make it stop?

If you've installed Nokogiri >= v1.12.0, then you can simply drop Nokogumbo as a dependency. Nokogiri has imported Nokogumbo's complete public API and so this cutover should be painless.

Alternatively, if Nokogumbo is a transitive dependency, then you should be able to bundle update nokogumbo to v2.0.5 to eliminate the constant/method redefined warnings. In this case, until Nokogumbo is removed by the intermediate dependencies, you will continue to see a "Using Nokogiri::HTML5 provided by Nokogiri" warning at require-time.

The text was updated successfully, but these errors were encountered:

Closes #170 A future version of Nokogiri will provide Nokogumbo's API (see sparklemotion/nokogiri#2204). This change will allow Nokogumbo to detect whether Nokogiri provides the HTML5 API and become a "shim" -- gracefully defer to Nokogiri by refusing to load itself. Some contractual assumptions I'm making about Nokogiri: - Nokogiri will faithfully reproduce the `::Nokogiri::HTML5` singleton method, module, and namespace (including classes `Nokogiri::HTML5::Node`, `Nokogiri::HTML5::Document`, and `Nokogiri::HTML5::DocumentFragment`) - Nokogiri will not provide a `::Nokogumbo` module/namespace, but will provide a similar `::Nokogiri::Gumbo` module which will provide the same constants and singleton methods as `::Nokogumbo`: - `Nokogumbo.parse()` will be provided as `Nokogiri::Gumbo.parse()` - `Nokogumbo.fragment()` → `Nokogiri::Gumbo.fragment()` - `Nokogumbo::DEFAULT_MAX_ATTRIBUTES` → `Nokogiri::Gumbo::DEFAULT_MAX_ATTRIBUTES` - `Nokogumbo::DEFAULT_MAX_ERRORS` → `Nokogiri::Gumbo::DEFAULT_MAX_ERRORS` - `Nokogumbo::DEFAULT_MAX_TREE_DEPTH` → `Nokogiri::Gumbo::DEFAULT_MAX_TREE_DEPTH` This change checks for the existence of `Nokogiri::HTML5`, `Nokogiri::Gumbo`, and an expected singleton method on each. We could do a more- or less-thorough check here. This change also provides an "escape hatch" using an environment variable `NOKOGUMBO_IGNORE_NOKOGIRI_HTML5` which can be set to avoid the "shim" behavior. This escape hatch might be unnecessary, but this change is invasive enough to make me want to be cautious. In "shim" mode, `Nokogumbo.parse()` and `.fragment()` will be forwarded to the Nokogiri implementation. The `Nokogumbo::DEFAULT*` constants will always be defined, but when in "shim" mode will be set to the `Nokogiri`-provided values. Nokogumbo will emit a single warning message at `require`-time when it is in "shim" mode. This message points users to sparklemotion/nokogiri#2205 which will explain what's going on and help people migrate their applications (but is an empty placeholder right now). I did not include deprecation warning messages in `Nokogumbo.parse` and `.fragment`. If you feel strongly that we should, let me know.

Closes #170 A future version of Nokogiri will provide Nokogumbo's API (see sparklemotion/nokogiri#2204). This change will allow Nokogumbo to detect whether Nokogiri provides the HTML5 API and whether to use Nokogiri's implementation or Nokogumbo's implementation. Some contractual assumptions I'm making about Nokogiri: - Nokogiri will faithfully reproduce the `::Nokogiri::HTML5` singleton method, module, and namespace (including classes `Nokogiri::HTML5::Node`, `Nokogiri::HTML5::Document`, and `Nokogiri::HTML5::DocumentFragment`) - Nokogiri will not provide a `::Nokogumbo` module/namespace, but will provide a similar `::Nokogiri::Gumbo` module which will provide the same public API as `::Nokogumbo`. This change checks for the existence of `Nokogiri::HTML5`, `Nokogiri::Gumbo`, and an expected singleton method on each. We could do a more- or less-thorough check here. This change also provides an "escape hatch" using an environment variable `NOKOGUMBO_IGNORE_NOKOGIRI_HTML5` which can be set to force Nokogumbo to use its own implementation. This escape hatch might be unnecessary, but this change is invasive enough to make me want to be cautious. Nokogumbo will emit a single warning message at `require`-time when it is uses Nokogiri's implementation. This message points users to sparklemotion/nokogiri#2205 which will explain what's going on and help people migrate their applications (but is an empty placeholder right now).

Closes #170 A future version of Nokogiri will provide Nokogumbo's API (see #2204). This change will allow Nokogumbo to detect whether Nokogiri provides the HTML5 API and whether to use Nokogiri's implementation or Nokogumbo's implementation. Some contractual assumptions I'm making about Nokogiri: - Nokogiri will faithfully reproduce the `::Nokogiri::HTML5` singleton method, module, and namespace (including classes `Nokogiri::HTML5::Node`, `Nokogiri::HTML5::Document`, and `Nokogiri::HTML5::DocumentFragment`) - Nokogiri will not provide a `::Nokogumbo` module/namespace, but will provide a similar `::Nokogiri::Gumbo` module which will provide the same public API as `::Nokogumbo`. This change checks for the existence of `Nokogiri::HTML5`, `Nokogiri::Gumbo`, and an expected singleton method on each. We could do a more- or less-thorough check here. This change also provides an "escape hatch" using an environment variable `NOKOGUMBO_IGNORE_NOKOGIRI_HTML5` which can be set to force Nokogumbo to use its own implementation. This escape hatch might be unnecessary, but this change is invasive enough to make me want to be cautious. Nokogumbo will emit a single warning message at `require`-time when it is uses Nokogiri's implementation. This message points users to #2205 which will explain what's going on and help people migrate their applications (but is an empty placeholder right now).

swanson · 2021-07-30T14:18:44Z

Thanks, this issue was helpful in debugging a bunch of warnings I started receiving:

/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:13: warning: already initialized constant Nokogiri::HTML5::HTML_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:227: warning: previous definition of HTML_NAMESPACE was here
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:14: warning: already initialized constant Nokogiri::HTML5::MATHML_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:228: warning: previous definition of MATHML_NAMESPACE was here
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:15: warning: already initialized constant Nokogiri::HTML5::SVG_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:229: warning: previous definition of SVG_NAMESPACE was here
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:16: warning: already initialized constant Nokogiri::HTML5::XLINK_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:230: warning: previous definition of XLINK_NAMESPACE was here
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:17: warning: already initialized constant Nokogiri::HTML5::XML_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:231: warning: previous definition of XML_NAMESPACE was here
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogumbo-2.0.4/lib/nokogumbo/html5.rb:18: warning: already initialized constant Nokogiri::HTML5::XMLNS_NAMESPACE
/Users/matt/.asdf/installs/ruby/2.7.3/lib/ruby/gems/2.7.0/gems/nokogiri-1.12.0.rc1-x86_64-darwin/lib/nokogiri/html5.rb:232: warning: previous definition of XMLNS_NAMESPACE was here

If you're not explicitly adding nokogiri or nokogumbo to your project, check your Gemfile.lock and search for the gem names to see what other gems are depending on them. You can then try upgrading/removing those gems to see if it removes the nokogumbo dependency. If you're still having problems, you may need to wait for new versions of those gems to be released with updated dependencies.

flavorjones · 2021-07-31T18:40:51Z

@swanson It's strange to me that you'd be in a situation where you're running a prerelease of Nokogiri, but an older version of Nokogumbo. Did bundle update nokogumbo address your issue (by eliminating the "method redefined"/"already initialized constant" errors)?

swanson · 2021-08-02T13:15:41Z

@swanson It's strange to me that you'd be in a situation where you're running a prerelease of Nokogiri, but an older version of Nokogumbo. Did bundle update nokogumbo address your issue (by eliminating the "method redefined"/"already initialized constant" errors)?

It was a quirk of my environment, mostly wanted to add the note for anyone else googling :)

(We were on the pre-release for a security fix but never switched off once a new official release was published)

flavorjones · 2021-08-02T15:20:24Z

I've updated the description text to address the "transitive dependency" case. Thanks for bringing that up, @swanson!

Nokogiri v1.12 merged Nokogumbo code https://github.com/sparklemotion/nokogiri/releases/tag/v1.12.0 Therefore, Nokogumbo is not required as a dependency anymore. If you run tests with latest Nokogiri you will see deprecation note: ```NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See sparklemotion/nokogiri#2205 for more information.```

Nokogiri v1.12 merged Nokogumbo code https://github.com/sparklemotion/nokogiri/releases/tag/v1.12.0 Therefore, Nokogumbo is not required as a dependency anymore. If tests are run with latest Nokogiri deprecation note is printed: ``` NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See sparklemotion/nokogiri#2205 for more information. ```

It's now part of Nokogiri. sparklemotion/nokogiri#2205

By removing Nokogumbo we can resolve a warning that downstream apps are emitting of "NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See sparklemotion/nokogiri#2205 for more information."

Addresses the following notification triggered on application launch ``` NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See sparklemotion/nokogiri#2205 for more information. ```

Partially addresses the following notification triggered on application launch. ``` NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri. See sparklemotion/nokogiri#2205 for more information. ``` This change updates `sanitize` to a verzion that removes the dependency on nokogumbo. The remaining dependency comes from the `linkeddata` data gem which has a more complicated dependency chain to address.

The Nokogumbo library has been deprecated by Nokogiri 1.12.0 and its whole API has been integrated into the latter. Update our dependencies to directly depend on Nokogiri only. See sparklemotion/nokogiri#2205 for details.

ghost · 2021-10-17T10:42:47Z

❤️

The gem has been merged into nokogiri 1.12.0. Remove its deprecation warnings (sparklemotion/nokogiri#2205).

This fixes a number of CVEs and drops the need for nokogumbo: sparklemotion/nokogiri#2205

This fixes a number of CVEs and drops the need for nokogumbo: sparklemotion/nokogiri#2205 Also drops Ruby 2.5 since it reached EOL on 2021-04-05 (https://www.ruby-lang.org/en/downloads/branches/). Closes premailer#208

This fixes a number of CVEs and drops the need for nokogumbo: sparklemotion/nokogiri#2205 Also drops Ruby 2.5 and 2.6 since they reached EOL (https://www.ruby-lang.org/en/downloads/branches/). Closes premailer#208

Nokogumbo has been merged with Nokogiri since the latter's version 1.13. This gets rid of the ‘NOTE: nokogumbo: Using Nokogiri::HTML5 provided by Nokogiri’ message during build. See sparklemotion/nokogiri#2205 for details.

flavorjones mentioned this issue Mar 13, 2021

Epic: merge Nokogumbo into Nokogiri #2204

Closed

33 tasks

flavorjones mentioned this issue Mar 14, 2021

feat: Nokogumbo detects Nokogiri's HTML5 API rubys/nokogumbo#171

Merged

flavorjones changed the title ~~Placeholder: user-facing information for Nokogumbo users who need help upgrading/migrating~~ Information for Nokogumbo users about the Nokogiri merger and their migration Jun 21, 2021

flavorjones changed the title ~~Information for Nokogumbo users about the Nokogiri merger and their migration~~ Information for Nokogumbo users about the Nokogiri merger Jun 23, 2021

thibaudgg added a commit to acp-admin/acp-admin that referenced this issue Aug 7, 2021

Remove Nokogumbo gem

09976ea

It's now part of Nokogiri. sparklemotion/nokogiri#2205

flavorjones pinned this issue Aug 26, 2021

flavorjones closed this as completed Aug 26, 2021

kevindew mentioned this issue Sep 30, 2021

Remove Nokogumbo dependency alphagov/govspeak#220

Merged

networm mentioned this issue Oct 6, 2021

To enable automatic compilation of books, please fix failing URLs progit/progit2-zh#441

Closed

mgrabovsky mentioned this issue Oct 12, 2021

Drop dependency on Nokogumbo faktaoklimatu/web-core#119

Merged

redsummernight added a commit to redsummernight/otwarchive that referenced this issue Feb 26, 2022

AO3-6085 Remove explicit dependency on nokogumbo

2a1332b

The gem has been merged into nokogiri 1.12.0. Remove its deprecation warnings (sparklemotion/nokogiri#2205).

redsummernight mentioned this issue Feb 26, 2022

AO3-6085 Remove explicit dependency on nokogumbo otwcode/otwarchive#4174

Merged

sarken pushed a commit to otwcode/otwarchive that referenced this issue Feb 26, 2022

AO3-6085 Remove explicit dependency on nokogumbo (#4174)

ec271b9

The gem has been merged into nokogiri 1.12.0. Remove its deprecation warnings (sparklemotion/nokogiri#2205).

stanhu added a commit to stanhu/premailer that referenced this issue May 3, 2022

Update nokogiri to v1.13.4

9f18f1b

This fixes a number of CVEs and drops the need for nokogumbo: sparklemotion/nokogiri#2205

stanhu mentioned this issue May 3, 2022

Update nokogiri to v1.13.4 premailer/premailer#416

Merged

mgrabovsky mentioned this issue Jun 24, 2022

Add check to get_url filter faktaoklimatu/web-core#174

Merged

flavorjones unpinned this issue Aug 18, 2022

cesswairimu mentioned this issue Oct 21, 2022

Fix nokogumbo gem warning publiclab/plots2#11528

Closed

woutermont mentioned this issue Jul 8, 2023

build: minimal toolchain update solid/solidproject.org#794

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Information for Nokogumbo users about the Nokogiri merger #2205

Information for Nokogumbo users about the Nokogiri merger #2205

flavorjones commented Mar 13, 2021 •

edited

swanson commented Jul 30, 2021

flavorjones commented Jul 31, 2021 •

edited

swanson commented Aug 2, 2021

flavorjones commented Aug 2, 2021

ghost commented Oct 17, 2021

Information for Nokogumbo users about the Nokogiri merger #2205

Information for Nokogumbo users about the Nokogiri merger #2205

Comments

flavorjones commented Mar 13, 2021 • edited

How To Resolve The Warnings

A Deeper Explanation for the Curious

Why am I seeing this warning message?

No, that's not the warning I'm seeing.

Why did you do this?

How do I make it stop?

swanson commented Jul 30, 2021

flavorjones commented Jul 31, 2021 • edited

swanson commented Aug 2, 2021

flavorjones commented Aug 2, 2021

ghost commented Oct 17, 2021

flavorjones commented Mar 13, 2021 •

edited

flavorjones commented Jul 31, 2021 •

edited