Regression in integration tests when providing as: :html

[inkstak]

Steps to reproduce

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem "rails", github: "rails/rails", branch: "main"
end

require "action_controller/railtie"

class TestApp < Rails::Application
  config.hosts << "www.example.com"
  routes.draw do
    resources :characters
  end

  config.root   = __dir__
  config.logger = Logger.new($stdout)
  Rails.logger  = config.logger
end

class CharactersController < ActionController::Base
  def create
    render head: :no_content
  end
end

require "minitest/autorun"
require "rack/test"

class BugTest < ActionDispatch::IntegrationTest
  def test_params_without_content_type
    post "/characters", params: { name: "Muad'Dib" }
  end

  def test_params_with_htm_content_type
    post "/users", as: :html, params: { name: "Muad'Dib" }
  end

  private

  def app
    Rails.application
  end
end

Expected behavior

In Rails 7.0, the second test works.

Actual behavior

In Rails 7.1, the second test will raise an error :

BugTest#test_params_with_htm_content_type:
NoMethodError: undefined method `to_html' for {:name=>"Muad'Dib"}:Hash
    .../actionpack/lib/action_dispatch/testing/request_encoder.rb:39:in `encode_params'
    .../actionpack/lib/action_dispatch/testing/integration.rb:251:in `process'
    .../actionpack/lib/action_dispatch/testing/integration.rb:22:in `post'
    .../actionpack/lib/action_dispatch/testing/integration.rb:379:in `post'
    issue_bug.rb:39:in `test_params_with_htm_content_type'

System configuration

Rails version:

Rails 7.1.2 & main

Ruby version:

Ruby 3.2.2

[inkstak]

The issue comes from this new line in action_dispatch/testing/request_encoder.rb

+   register_encoder :html, response_parser: -> body { Rails::Dom::Testing.html_document.parse(body) }

Without HTML encoder, the default encoderIdentityEncoder returned the params as passed.

Now that an HTML encoder has been registered, it tries to call :"to_#{@mime.symbol}" on params by default.

My first thought was to register a param_encoder :

ActionDispatch::IntegrationTest.register_encoder :html,
  response_parser: ->(body) { Rails::Dom::Testing.html_document.parse(body) },
  param_encoder: ->(params) { params }

But it doesn't seem to work : the request is now performed but params are not passed to the controller.

EDIT: The patch suggested is working. I'll try to pull a request asap.

[mguan2020]

Thank you very much @inkstak for the detailed explanations. I have attached a PR that implements your suggested change as well as the corresponding tests.

Please let me know if there is anything you would like me to fix.

[seanpdoyle]

In the case you've shared above, passing as: :html implies that the request parameters will come from a POST body encoded as text/html. That isn't entirely true -- browsers will encode POST parameters as either application/x-www-form-urlencoded or multipart/form-data.

In the short-term, you might be better off omitting the as: :html entirely and letting Action Dispatch's test harness determine the proper content type and parameter encoding.

In the long-term, the as: :html keyword argument might require some special treatment. I've opened #50390 as an alternative to #50375.

[inkstak]

In the case you've shared above, passing as: :html implies that the request parameters will come from a POST body encoded as text/html. That isn't entirely true -- browsers will encode POST parameters as either application/x-www-form-urlencoded or multipart/form-data.

@seanpdoyle I couldn't agree more. Using as: :html seems unnecessary when you're testing endpoints that are intended to be used in a browser.

I used to use it as RSpec metadata to explicitly test responses to unexpected formats against API endpoints.
Because Rails default format for */* is HTML, I used as: :html :

RSpec.describe "API::ResourcesController#create" do
  subject do
    post "/resources", as:, headers:, params:
  end

  let(:as) { |e| e.metadata[:as] }
  
  context "when requesting JSON", as: :json do
    it_behaves_like "it requires authentication"
    it_behaves_like "it responds successfully when signed in"
  end

  context "when requesting HTML", as: :html do
    it_behaves_like "it responds with not acceptable"
  end
end