Skip to content

RequireAndLoadBehavior

Jump to bottom
Isaiah Peng edited this page Feb 5, 2016 · 1 revision

» JRuby Project Wiki Home Page       » Design: Internals

Require and Load Behavior

This page provides documentation on LoadService behavior pre-patch and post-patch, in an attempt to simplify and speed loading. The two issues addressed are JRUBY-335 and JRUBY-288.

The altered version is simpler, more correct, and faster in some cases (rails, specifically). It's a pretty tangled web regardless.

Table of Contents

0.9.2 behavior

  • Kernel.load calls findLibrary with an exact filename only.
  • Kernel.require calls findLibrary with the specified filename appended with four possible extensions in turn:
    1. rb
    2. ast.ser
    3. rb.ast.ser
    4. jar
  • require also tries the filename as-is, which is incorrect behavior.

Default load path includes

  1. Current dir (implicit, during searches).
  2. The <JRuby lib> dir (as specified by jruby.lib system property).
  3. /lib/ruby/site_ruby/1.8(note this and subsequent paths are using jruby.home property, not jruby.lib property).
  4. /lib/ruby/site_ruby/1.8/java
  5. /lib/ruby/site_ruby
  6. /lib/ruby/1.8
  7. /lib/ruby/site_ruby/1.8/java
  8. /lib/ruby
  9. '.'

findLibrary

  1. Checks builtin libraries first for supplied filename.
  2. Calls findFile with given filename, returning null if result is null.
  3. If file (resource) is found:
    1. If file ends with .jar, returns a JarredScript with specified resource.
    2. If file ends with .rb.ast.ser, returns a PreparsedScript with specified resource.
    3. Otherwise returns ExternalScript with resource and supplied filename.

findFile

  1. If file starts with jar:, returns LoadServiceResource using supplied filename directly as a URL.
  2. If file is found in current dir and resulting current-dir + filename is absolute, returns LoadServiceResource with current-dir + filename as URL
  3. Retrieves a classloader from the JavaSupport object for this runtime (which ends up using the classloader in which that JavaSupport was loaded, usually System classloader).
  4. Replaces \\ with / in filename.
  5. If filename starts with /, attempts to load from classloader resources. (This is incorrect: classloader resources cannot start with a /.)
  6. Iterates over all entries in the load path:
    1. If load path entry starts with jar: attempts to load from an associated JarFile, caching JarFile objects as they are encountered and instantiated.
    2. Attempts to load file from the absolute path of current dir + load path entry. (Seems wrong to me, and creates two JRubyFile objects every time.)
    3. Attempts to load file from load path entry with \\ substituted with /. (Also seems quite wrong, since most files' load paths will be absolute paths for ruby/1.8 and similar directories, which would never work in a classloader.)
  7. Attempts to load the filename directly from the classloader (again) regardless of whether it starts with / or not.
  8. Failing all of the above, returns null, usually resulting in findLibrary returning nil. For require, this will then advance to the next extension.

Problems with 0.9.2 behavior

  • require should never attempt to load the supplied name without appending extensions. With a file foo in the current dir, require 'foo' should still fail, only accepting foo with some extension.
  • For each extension, all possible locations are checked first, many involving classloader resources. This means that resources on the filesystem can take longer to look up since they can come after many classloader hits.
  • We have no files with extension .ast.ser that aren't .rb.ast.ser, and I know of no reason why we ever would.
  • Classloader resources are checked for all supplied filenames that start with /, even though this is not valid for classloader resources.
  • Classloader resources are (possibly again) checked for each load path, even though most load paths will be absolute paths to ruby/1.8 and related directories.
  • Classloader resources are checked a third time if nothing has been found by using the same bare filename. If the file starts with / and nothing was found for the first classloader resource check above, this results in a repeat of the same search.
  • We do not have a site_ruby/1.8/java directory. I'm not sure we ever will need it.
  • We should not search "." since changing the current dir (Dir.chdir) should cause files in our initial PWD to no longer be implicitly locatable (and "." will always resolve to the JVM's startup PWD).

New behavior (provided by supplied patch in JRUBY-335)

  • Kernel.load behavior is unchanged, other than incidental changes in classloader resource lookup as a result of calling findLibrary and findLibraryWithClassloaders (documented below).
  • Kernel.require behavior excludes as-is filenames from search.
  • The ast.ser extension is eliminated.
  • The site_ruby/1.8/java load path entry is removed.
  • The '.' load path entry is removed.

The smartLoad method used for searching all expected extensions is changed as follows

  • It performs one search of all extensions against only filesystem-based locations.
  • It performs a second search of all extensions using classloader resource lookups or using direct URL-based resources (i.e. jar: urls).

Both searches:

  • Eliminate substituting \\ for / in filenames. (JRubyFile and Java's own logic should handle this ok, but this should be tested on Windows.)

The filesystem-based search:

  1. Checks for the file in the current directory.
  2. Checks for the file using each load path entry in turn.

The classloader and URL-based search:

  1. Checks for jar: URLs and loads if found.
  2. Retrieves Thread's context classloader (rather than always using the classloader JRuby was loaded in).
  3. For each entry in the load path:
    1. If entry is a jar: URL, uses that to load and cache JarFile objects that are then used to load the file if possible.
    2. If entry does not start with /, attempts to load from classloader resources.
  4. If filename does not start with /, attempts to load from classloader resources.
  5. If all else fails, returns null, usually resulting in the smartLoad search proceeding to the next extension.

Justification for Changes

The changes:

  • Simplify the loading process.
  • Eliminate extraneous paths that cannot or should not be searched.
  • Eliminate repetitive classloader resource searching.
  • Eliminate searching of classloaders or jar loading until all potential filesystem locations have been exhausted.
  • Appear to speed a simple jirb load by a very small amount.
  • Improve the load time of a rails script from 8.7 seconds to 7.7 seconds (time jruby script/console --help).
  • Improve the load time of the same script with a very large classpath from 9 seconds to 7.8 seconds.
Clone this wiki locally