Applications using Sprockets, Rails UJS, but not CoffeeScript cannot consume Rails via git prior to Rails UJS abandoning CoffeeScript

[sambostock]

Context

Our application continues to run into the bug fixed by #46110, as the fix is not present on any release. While discussing temporary monkey patches in Slack, @eileencodes recommended pointing our app to the 7-0-stable branch of Rails, which contains an unreleased backport. When trying to switch over, we ran into the following error:

LoadError: cannot load such file -- coffee_script

This same error occurs even if switching from the 7.0.4 published release of the gem to directly to the corresponding v7.0.4 git tag, indicating the issue is due to consuming Rails via git, as opposed to a commit introduced on 7-0-stable since the 7.0.4 release.

Investigation

@bryanparadis and I conducted an investigation and came to the following conclusion:

Prior to #45546, Rails UJS was written as .coffeescript files.

When a host app still using Sprockets for Javascript and including Rails UJS in its manifest compiles assets it tries to include those files. In the published versions of Rails, these files have been already compiled into regular Javascript, so no additional compilation is necessary.

If Rails is being consumed via git, the files only exist as .coffeescript files, so Sprockets tries to load coffee_script to compile them.

If the host application does not depend on coffee_script, then this raises a LoadError.

Steps to reproduce

Reproduction Script

# /usr/bin/env bash

set -euo pipefail

echo 'Preparing directory...'
rm -rf dummy
mkdir dummy
cd dummy

echo 'Installing Rails...'
echo 'source "https://rubygems.org"' > Gemfile
echo 'gem "rails", git: "https://github.com/rails/rails.git", branch: "7-0-stable"' >> Gemfile
bundle install --quiet

echo 'Initialize Rails app...'
rails new \
	--force \
	--no-rc \
	--quiet \
	--skip-action-cable \
	--skip-action-mailbox \
	--skip-action-mailer \
	--skip-action-text \
	--skip-active-job \
	--skip-active-record \
	--skip-active-storage \
	--skip-bootsnap \
	--skip-git \
	--skip-hotwire \
	--skip-javascript \
	--skip-jbuilder \
	--skip-keeps \
	--skip-system-test \
	--skip-test \
	.

echo 'Using Sprockets for JS...'
mkdir app/assets/javascripts
touch app/assets/javascripts/application.js
echo "//= link_directory ../javascripts .js" >> app/assets/config/manifest.js

echo "Using Rails UJS..."
echo "//= require rails-ujs" > app/assets/javascripts/application.js

echo "Precompiling assets using $(bundle info rails | grep '\*' | sed 's/ *\* //')..."
bundle exec rails assets:precompile

echo 'Switching to 7-0-stable version of Rails...'
sed -i '' 's/gem "rails", "[^"]*"/gem "rails", github: "rails\/rails", branch: "7-0-stable"/' Gemfile
bundle install --quiet

echo "Precompiling assets using $(bundle info rails | grep '\*' | sed 's/ *\* //')..."
bundle exec rails assets:precompile

Expected behavior

Consuming a released version of Rails or its equivalent git tag should behave the same.

Actual behavior

Applications that work with a released version of Rails (e.g. 7.0.4) do not work with the equivalent git tag (e.g. v7.0.4) under certain circumstances:

• Not depending on coffee-script
• Using Sprockets for JavaScript, not Import Maps
• Depending on Rails UJS

System configuration

Rails version: v7.0.4

Ruby version: 3.1.2

[sambostock]

@rafaelfranca @skipkayhil, any ideas on how we could address this?

• Would it make sense to backport Convert rails-ujs to ES2015 modules #45546 to branches like 7-0-stable?
• Could we commit transpiled versions of the CoffeeScript files, so the coffee-script gem isn't required?
• Is this something we want to leave as is, and just require consumers to include a dependency on coffee-script?

[ghiculescu]

@sambostock sorry this didn't get any attention until now, and thanks for the detailed writeup. A few thoughts, please let me know if they make sense or are on the wrong track.

• Would it make sense to backport Convert rails-ujs to ES2015 modules #45546 to branches like 7-0-stable?

It sounds to me like this is going to be an issue on any existing stable branch or tag, not just 7-0-stable. Is that right? To be fair no other stable is still maintained so maybe that doesn't matter, but I just want to ensure I understood.

• Could we commit transpiled versions of the CoffeeScript files, so the coffee-script gem isn't required?
• Is this something we want to leave as is, and just require consumers to include a dependency on coffee-script?

Is this also going to be an issue in 7-1-stable? Is it an issue on the main branch? There's still a build script but presumably it doesn't depend on coffeescript anymore.

[skipkayhil]

Thanks for following up @ghiculescu!

It sounds to me like this is going to be an issue on any existing stable branch or tag, not just 7-0-stable. Is that right? To be fair no other stable is still maintained so maybe that doesn't matter, but I just want to ensure I understood.

Yep, this will apply to all of the previous stable branches as well.

Is this also going to be an issue in 7-1-stable? Is it an issue on the main branch? There's still a build script but presumably it doesn't depend on coffeescript anymore.

This will not be an issue going forward as the compiled files are actually present in the repo now here.

1. Would it make sense to backport Convert rails-ujs to ES2015 modules #45546 to branches like 7-0-stable?
2. Could we commit transpiled versions of the CoffeeScript files, so the coffee-script gem isn't required?
3. Is this something we want to leave as is, and just require consumers to include a dependency on coffee-script?

Out of the three options:

1. I don't think backporting is feasible because of the scope of changes made to rails-ujs
2. Definitely possible, would need to investigate how much effort this would require (might be as simple as removing the folder from .gitignore?)
3. Definitely the easiest, although I agree its a bit unfortunate (and the issue seems to be rare since this is the first report?)

[ghiculescu]

(and the issue seems to be rare since this is the first report?)

I was thinking the same, + the workaround is just to add the coffee_script gem until Rails 7.1, right?

[zzak]

@ghiculescu @skipkayhil How do ya'll feel about closing this one?

I think that since 7.1 is released, and upgrading will fix it, the alternative being add coffee_script to the bundle resolves this error:

LoadError: cannot load such file -- coffee_script

Which seems totally reasonable to me.

[ghiculescu]

Yes if it’s fixed in 7.1 we should close.

[zzak]

I cannot reproduce the bug with current 7-0-stable branch, closing.

zzak@mbp16 46292 % bash bug.sh
Preparing directory...
Installing Rails...
Fetching https://github.com/rails/rails.git
Resolving dependencies...
Fetching gem metadata from https://rubygems.org/.......
Bundle complete! 1 Gemfile dependency, 45 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Initialize Rails app...
Using Sprockets for JS...
Using Rails UJS...
Precompiling assets using rails (7.0.8)...
I, [2023-10-15T10:03:48.714125 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/manifest-dad05bf766af0fe3d79dd746db3c1361c0583026cdf35d6a2921bccaea835331.js
I, [2023-10-15T10:03:48.714216 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/manifest-dad05bf766af0fe3d79dd746db3c1361c0583026cdf35d6a2921bccaea835331.js.gz
I, [2023-10-15T10:03:48.714330 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/application-e0cf9d8fcb18bf7f909d8d91a5e78499f82ac29523d475bf3a9ab265d5e2b451.css
I, [2023-10-15T10:03:48.714428 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/application-e0cf9d8fcb18bf7f909d8d91a5e78499f82ac29523d475bf3a9ab265d5e2b451.css.gz
I, [2023-10-15T10:03:48.714592 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/application-1fc700a7beaac60f463364fcba0edbe3921a0d3e0c4b828e7584a7ad7d574e59.js
I, [2023-10-15T10:03:48.714730 #96076]  INFO -- : Writing /Users/zzak/code/bugs/46292/dummy/public/assets/application-1fc700a7beaac60f463364fcba0edbe3921a0d3e0c4b828e7584a7ad7d574e59.js.gz
Switching to 7-0-stable version of Rails...
Precompiling assets using rails (7.0.8 5873d62)...