Pure Java Nokogiri for JRuby

What is “pure Java” Nokogiri?

Pure Java version of Nokogiri is a Java port for JRuby. Currently, FFI version of Nokogiri works on JRuby via FFI library, which needs libxml2 installed. On the other hand, pure Java version doesn’t use libxml2 and FFI library. Nokogiri’s libxml2 dependent methods have been reimplemented by Apache Xerces, nekoHTML, and a couple of pure Java APIs. This means we don’t have limitation to use Nokogiri even on a pure Java environment. Yes, Nokogiri is available on Google App Engine. At this moment, pure Java version is on the way to the first release, and has 1 failure, 1 error and 2 skips (ffi version also skips two) reported from rake test on “master” branch. Currently, 1.5.0.beta.3 is out. If you want to use it, install with —pre.

Try pure Java version out and give us your feedback. If you find a bug, file it with “pure-java” tag.

Installation

gem install nokogiri --pre

You need

• JDK 1.6.0 and later
• JRuby 1.5.1 and later

Google App Engine

Nokogiri 1.5.0.beta.3 works fine on Google App Engine. But, if you are using 1.5.0.beta.2, you need a small hack to run with google-appengine gem.

1) Comment out five require xxx.jar lines in .gems/bundler_gems/jruby/1.8/gems/nokogiri-1.5.0.beta.2-java/lib/nokogiri.rb

 
  1 # -*- coding: utf-8 -*-
  2 # Modify the PATH on windows so that the external DLLs will get loaded.
  3
  4 require 'rbconfig'
  5 ENV['PATH'] = [File.expand_path(
  6 File.join(File.dirname(__FILE__), "..", "ext", "nokogiri")
  7 ), ENV['PATH']].compact.join(';') if RbConfig::CONFIG['host_os'] =~ /(mswin|mingw)/i
  8
  9 if defined?(RUBY_ENGINE) && RUBY_ENGINE == "jruby"
 10 # require 'isorelax.jar'
 11 # require 'jing.jar'
 12 # require 'nekohtml.jar'
 13 # require 'nekodtd.jar'
 14 # require 'xercesImpl.jar'
 15 require 'nokogiri/nokogiri'
 16 else
 17 require 'nokogiri/nokogiri'
 18 end

2) Remove WEB-INF/lib/gems.jar (if you have this file)
3) Restart the server

Then, Nokogiri will start working. This bug was fixed in master, so 1.5.0 final release won’t have this problem.

Please note. Pure Java Nokogiri is not yet fully tested on Google App Engine. There might be GAE specific problems.

Pure Java Specific Rules

Porting to Java is not easy. Contributors have struggled over the different behaviors between libxml2 and Xerces. Almost all Nokogiri API are implemented as they are, but some were very hard to make. Thus, pure Java version has a few specific rules. Please be aware followings when you use pure Java version.

DTD validation

Add “dtdvalid” option when a document is read.

xml = Nokogiri::XML(File.open(XML_FILE)) {|cfg| cfg.dtdvalid}
list = xml.internal_subset.validate xml

The number of errors is not the same as libxml2 version. Java version doesn’t report errors of attributes whose elements have already reported errors.

Public ID in DOCTYPE declaration

Don’t forget to write the second parameter.

<!DOCTYPE foo PUBLIC “bar” "">

W3C XML Schema validation

Xerces strictly checks whether a document is valid based on a given schema, while libxml does not. This behavior is unable to adjust to the one of libxml. Please be aware if XML Schema validation fails only on pure Java version.

XSLT.register method

Pure Java version doesn’t support XSLT.register method. This method is used for mapping namespace and Ruby object so that XSLT extension function works as intended. XST extension function processing by Java API is very different from libxml2. Java API needs specific rules; besides, we need to be careful to choose tools for XML processing. In some combination of processors, the feature works but doesn’t for other combination. If you are interested in details, go to my blog, http://yokolet.blogspot.com/2010/10/pure-java-nokogiri-xslt-extension.html. You’ll learn why this feature is dropped from pure Java version further.

Get Involved: how to build

If you want to help pure Java Nokogiri, you need to build it after cloning the source. Charlie wrote a nice, easy-to follow blog entry,
Nokogiri Java Port: Help Us Finish It!, which will help you, definitely.

Don’t forget. The codebase of pure Java Nokogiri has been merged into master. You don’t need to checkout any branch.