#to_text doesn't handle <br> elements well.

[der-flo]

#to_text doesn't handle <br> elements explicitly, so phrases like "foo<br>bar" get sanitzed to "foobar". In our use-case a line break ("foo\nbar") is a more practical result.

So I wrote a custom scrubber:

class HTMLToPlainText
  class BreakElementScrubber < Loofah::Scrubber
    def scrub(node)
      return CONTINUE unless node.name == 'br'
 
      node.add_previous_sibling Nokogiri::XML::Text.new("\n", node.document)
      node.remove
    end
  end

  def call(html_string)
    Loofah::HTML::Document
      .parse(html_string)
      .scrub!(:prune)
      .scrub!(BreakElementScrubber.new)
      .to_text(encode_special_chars: false)
      .strip
  end
end

Is this something valuable which should find it's way in the project via a pull request? If yes, could a maintainer give me implementation hints etc.? I'm willing to help.

[flavorjones]

Hi, @der-flo! Thanks for pointing this out, that seems like a really good feature to add. I don't think we need a new scrubber for it, though, I think we should modify the current behavior of #to_text (or, really, the behavior of the underlying scrubber, Loofah::Scrubbers::NewlineBlockElements.

That scrubber currently adds newlines for "block" elements in HTML4 and HTML5 plus a loose set of elements that can contain block elements. But it doesn't pay attention to the element br which is the direct analog of a newline in HTML.

I'll write up a PR for this now, it should be a very small change.

[flavorjones]

Loofah 2.14.0 has been released with #to_text inserting newlines for <br> elements. I hope this meets your needs!

[der-flo]

Your solution works fine, thanks!

[flavorjones]

Happy to help!