Frozen string literal changes #967

dillonwelch · 2017-10-26T04:22:20Z

I was able to get frozen_string_literal: true turned on in two files. Unfortunately I wasn't able to get them on in the rest, the majority of which were because of various calls to rstrip! and tr! which are more performant in micro-benchmarks and in memory usage. I made other changes in these files to string builder logic such that they no longer mutate strings, based on this StackOverflow post.

String mutations preventing frozen_string_literal: true
lib/haml/compiler.rb - rstrip!
lib/haml/filters - rstrip!
lib/haml/template_engine.rb - force_encoding, tr!
lib/haml/parser.rb - rstrip!

I wanted to keep the commit history for discussion about the various changes. Once everything is approved I can squash the commits and add a changelog entry.

Result of ruby benchmark.rb

                                         Haml |     ERB |   Erubi |
-------------------------------------------------------------------
Cached                                  0.088 |   0.073 |   0.051 |
ActionView                             15.558 |         |  10.493 |
ActionView with deep partials          41.512 |         |  37.338 |

Current master

                                         Haml |     ERB |   Erubi |
-------------------------------------------------------------------
Cached                                  0.085 |   0.073 |   0.070 |
ActionView                             18.001 |         |  10.954 |
ActionView with deep partials          46.022 |         |  34.406 |

dillonwelch · 2017-10-26T04:23:34Z

lib/haml/engine.rb

-             @temple_engine.precompiled_with_ambles(local_names) << "}\n", scope, @options.filename, @options.line)
+        str = @temple_engine.precompiled_with_ambles(local_names)
+        eval(
+          "Proc.new { |*_haml_locals| _haml_locals = _haml_locals[0] || {}; #{str}}\n",


This change is faster on micro-benchmarking as well as saving memory (2 strings per method call).

# frozen_string_literal: false begin require "bundler/inline" rescue LoadError => e $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" raise e end gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" gem "rails" end def allocate_count GC.disable before = ObjectSpace.count_objects yield after = ObjectSpace.count_objects after.each { |k,v| after[k] = v - before[k] } after[:T_HASH] -= 1 # probe effect - we created the before hash. GC.enable result = after.reject { |k,v| v == 0 } GC.start result end def master_version "hi there" << "cool" << "yeah" end def fast_version "#{'hi there'}#{'cool'}#{'yeah'}" end puts "master_version" puts allocate_count { 1000.times { master_version } } puts "fast_version" puts allocate_count { 1000.times { fast_version } } Benchmark.ips do |x| x.report("master_version") { master_version } x.report("fast_version") { fast_version } x.compare! end

master_version {:FREE=>-2846, :T_STRING=>3052} fast_version {:FREE=>-1001, :T_STRING=>1000} Warming up -------------------------------------- master_version 120.220k i/100ms fast_version 181.848k i/100ms Calculating ------------------------------------- master_version 2.527M (±14.8%) i/s - 12.383M in 5.039857s fast_version 6.634M (±17.6%) i/s - 32.005M in 5.002381s Comparison: fast_version: 6633843.3 i/s master_version: 2526801.8 i/s - 2.63x slower

dillonwelch · 2017-10-26T04:25:02Z

lib/haml/helpers.rb

-        s << '|' unless s.empty?
-        s << Regexp.escape(t)
-      end
+      tags = tags.map { |tag| Regexp.escape(tag) }.join('|')


Doesn't impact performance or memory either way but was necessary for the frozen_string_literal: true change

# frozen_string_literal: false begin require "bundler/inline" rescue LoadError => e $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" raise e end gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" gem "rails" end def allocate_count GC.disable before = ObjectSpace.count_objects yield after = ObjectSpace.count_objects after.each { |k,v| after[k] = v - before[k] } after[:T_HASH] -= 1 # probe effect - we created the before hash. GC.enable result = after.reject { |k,v| v == 0 } GC.start result end @tags = %w(hi there I am a list of tags) def master_version @tags.each_with_object('') do |t, s| s << '|'.freeze unless s.empty? s << Regexp.escape(t) end end def fast_version @tags.map { |tag| Regexp.escape(tag) }.join('|'.freeze) end puts "master_version" puts allocate_count { 1000.times { master_version } } puts "fast_version" puts allocate_count { 1000.times { fast_version } } Benchmark.ips do |x| x.report("master_version") { master_version } x.report("fast_version") { fast_version } x.compare! end

master_version {:FREE=>-9790, :T_STRING=>9052, :T_IMEMO=>1001} fast_version {:FREE=>-10001, :T_STRING=>9000, :T_ARRAY=>1000} Warming up -------------------------------------- master_version 13.960k i/100ms fast_version 16.597k i/100ms Calculating ------------------------------------- master_version 200.727k (±15.4%) i/s - 991.160k in 5.057382s fast_version 206.033k (±11.7%) i/s - 1.029M in 5.072112s Comparison: fast_version: 206033.4 i/s master_version: 200727.0 i/s - same-ish: difference falls within error

dillonwelch · 2017-10-26T04:29:12Z

lib/haml/parser.rb

@@ -698,7 +698,7 @@ def parse_new_attributes(text)
      end

      static_attributes = {}
-      dynamic_attributes = "{"
+      dynamic_attributes = ["{"]


Doesn't impact performance or memory either way but is necessary for future future_string_literal: true changes

# frozen_string_literal: false begin require "bundler/inline" rescue LoadError => e $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" raise e end gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" gem "rails" end def allocate_count GC.disable before = ObjectSpace.count_objects yield after = ObjectSpace.count_objects after.each { |k,v| after[k] = v - before[k] } after[:T_HASH] -= 1 # probe effect - we created the before hash. GC.enable result = after.reject { |k,v| v == 0 } GC.start result end def master_version dynamic_attributes = '{' if rand(2) == 1 dynamic_attributes << 'hi there'.freeze end dynamic_attributes << "}".freeze dynamic_attributes = nil if dynamic_attributes == "{}".freeze end def fast_version dynamic_attributes = ['{'.freeze] if rand(2) == 1 dynamic_attributes << 'hi there'.freeze end dynamic_attributes << "}".freeze dynamic_attributes = nil if dynamic_attributes.first == "{}".freeze && dynamic_attributes.length == 1 end puts "master_version" puts allocate_count { 1000.times { master_version } } puts "fast_version" puts allocate_count { 1000.times { fast_version } } Benchmark.ips do |x| x.report("master_version") { master_version } x.report("fast_version") { fast_version } x.compare! end

master_version {:FREE=>-693, :T_STRING=>1052, :T_IMEMO=>1} fast_version {:FREE=>-1001, :T_ARRAY=>1000} Warming up -------------------------------------- master_version 111.400k i/100ms fast_version 122.259k i/100ms Calculating ------------------------------------- master_version 1.976M (±17.1%) i/s - 9.580M in 5.056250s fast_version 1.966M (±25.6%) i/s - 8.925M in 5.017136s Comparison: master_version: 1975829.5 i/s fast_version: 1965749.8 i/s - same-ish: difference falls within error

Thanks for your effort of gradual changes and to put benchmark, but please exclude intermediate changes that have no benefit. It makes hard to read commit log to know why this change was necessary, and this change should be committed with # frozen_string_literal: true.

And there's no need to change this to Array buffer for introducing # frozen_string_literal: true.

It uglifies code after this, so please use "{".dup instead.

dillonwelch · 2017-10-26T04:45:42Z

lib/haml/helpers.rb

@@ -200,8 +197,8 @@ def preserve(input = nil, &block)
    # @yield [item] A block which contains Haml code that goes within list items
    # @yieldparam item An element of `enum`
    def list_of(enum, opts={}, &block)
-      opts_attributes = opts.each_with_object('') {|(k, v), s| s << " #{k}='#{v}'"}
-      enum.each_with_object('') do |i, ret|
+      opts_attributes = opts.map { |k, v| " #{k}='#{v}'" }.join


Same as above

# frozen_string_literal: false begin require "bundler/inline" rescue LoadError => e $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" raise e end gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" gem "rails" end def allocate_count GC.disable before = ObjectSpace.count_objects yield after = ObjectSpace.count_objects after.each { |k,v| after[k] = v - before[k] } after[:T_HASH] -= 1 # probe effect - we created the before hash. GC.enable result = after.reject { |k,v| v == 0 } GC.start result end @strings = %w(hi there I am an array of strings) def master_version @strings.each_with_object('') do |string, builder| builder << "#{string} is cool " end end def fast_version @strings.map do |string| "#{string} is cool " end.join end puts "master_version" puts allocate_count { 1000.times { master_version } } puts "fast_version" puts allocate_count { 1000.times { fast_version } } Benchmark.ips do |x| x.report("master_version") { master_version } x.report("fast_version") { fast_version } x.compare! end

master_version {:FREE=>-9836, :T_STRING=>9052, :T_IMEMO=>1001} fast_version {:FREE=>-10001, :T_STRING=>9000, :T_ARRAY=>1000} Warming up -------------------------------------- master_version 23.988k i/100ms fast_version 29.611k i/100ms Calculating ------------------------------------- master_version 302.295k (± 8.2%) i/s - 1.511M in 5.034657s fast_version 359.883k (±12.2%) i/s - 1.777M in 5.015867s Comparison: fast_version: 359883.3 i/s master_version: 302295.1 i/s - same-ish: difference falls within error

dillonwelch · 2017-10-26T04:49:51Z

lib/haml/helpers.rb

-        ret << %Q!<li#{opts_attributes}>#{result}</li>!
-      end
+        %Q!<li#{opts_attributes}>#{result}</li>!
+      end.join("\n")


Same as above

# frozen_string_literal: false begin require "bundler/inline" rescue LoadError => e $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler" raise e end gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" gem "rails" end def allocate_count GC.disable before = ObjectSpace.count_objects yield after = ObjectSpace.count_objects after.each { |k,v| after[k] = v - before[k] } after[:T_HASH] -= 1 # probe effect - we created the before hash. GC.enable result = after.reject { |k,v| v == 0 } GC.start result end @strings = %w(hi there I am an array of strings) def master_version @strings.each_with_object('') do |string, builder| builder << "\n".freeze unless builder.empty? builder << "#{string} is cool " end end def fast_version @strings.map do |string| "#{string} is cool " end.join("\n".freeze) end puts "master_version" puts allocate_count { 1000.times { master_version } } puts "fast_version" puts allocate_count { 1000.times { fast_version } } Benchmark.ips do |x| x.report("master_version") { master_version } x.report("fast_version") { fast_version } x.compare! end

master_version {:FREE=>-9773, :T_STRING=>9052, :T_IMEMO=>1001} fast_version {:FREE=>-10001, :T_STRING=>9000, :T_ARRAY=>1000} Warming up -------------------------------------- master_version 23.523k i/100ms fast_version 18.124k i/100ms Calculating ------------------------------------- master_version 271.699k (± 9.1%) i/s - 1.364M in 5.074300s fast_version 280.265k (±13.2%) i/s - 1.377M in 5.012751s Comparison: fast_version: 280265.0 i/s master_version: 271698.9 i/s - same-ish: difference falls within error

k0kubun · 2017-10-26T06:35:48Z

lib/haml/parser.rb

@@ -737,8 +741,10 @@ def parse_new_attribute(scanner)
      end

      return name, [:static, content.first[1]] if content.size == 1
-      return name, [:dynamic,
-        %!"#{content.each_with_object('') {|(t, v), s| s << (t == :str ? inspect_obj(v)[1...-1] : "\#{#{v}}")}}"!]


Please don't introduce unnecessary changes. Obviously we can enable # frozen_string_literal: true if we change '' to ''.dup.

dillonwelch · 2017-10-28T00:14:57Z

New benchmark:

                                         Haml |     ERB |   Erubi |
-------------------------------------------------------------------
Cached                                  0.073 |   0.117 |   0.076 |
ActionView                             17.292 |         |  10.315 |
ActionView with deep partials          38.455 |         |  38.641 |

k0kubun · 2017-10-29T07:50:16Z

With ruby 2.5.0dev (2017-10-27 trunk 60479) [x86_64-darwin16],

before

                                         Haml |     
-----------------------------------------------
Cached                                  0.059 |         
ActionView                             11.794 |         
ActionView with deep partials          30.832 |

after

                                         Haml |    
-----------------------------------------------
Cached                                  0.061 |         
ActionView                             11.706 |         
ActionView with deep partials          31.120 |

Probably this PR doesn't approach this benchmark's hotspot.

But in general I like to have # frozen_string_literal: true to avoid future performance regression, and I feel changed code is more readable (except ''.dup part though). So I'm merging this. Thanks for your work!

dillonwelch · 2017-10-30T16:39:13Z

Excellent. Glad to help.

What's the general release/version bump schedule?

k0kubun · 2017-11-06T16:43:53Z

@amatsuda I'm in favor of releasing current one as new version.

Changes are not breaking but making things to be frozen might be dangerous. I think it's time to cut 5.1.0 release.

amatsuda · 2017-11-07T11:44:16Z

Hm... I can do 5.0.5 release. I don't think this is worth a minor version bump since we don't really have any new user-facing feature nor breaking change.

k0kubun · 2017-11-07T12:24:02Z

I see, it was not so strong opinion. 5.0.5 is okay for me.

xrl · 2019-09-30T20:06:41Z

I'm very late to the party, but I'm on rails 4.2 still. It would be nice to have this in a 5.0.5 release.

Update ruby-haml to 5.1.2. pkgsrc change: add "USE_LANGUAGES= # none". ## 5.1.2 Released on August 6, 2019 ([diff](haml/haml@v5.1.1...v5.1.2)). * Fix crash in some environments such as New Relic by unfreezing string literals for ParseNode#inspect. [#1016](haml/haml#1016) (thanks [Jalyna](https://github.com/jalyna)) ## 5.1.1 Released on May 25, 2019 ([diff](haml/haml@v5.1.0...v5.1.1)). * Fix NameError bug for that happens on ruby 2.6.1-2.6.3 + haml 5.1.0 + rails 4.2.x + erubi. (Akira Matsuda) ## 5.1.0 Released on May 16, 2019 ([diff](haml/haml@v5.0.4...v5.1.0)). * Rails 6 support [#1008](haml/haml#1008) (thanks [Seb Jacobs](https://github.com/sebjacobs)) * Add `escape_filter_interpolations` option for backwards compatibility with haml 4 defaults [#984](haml/haml#984) (thanks [Will Jordan](https://github.com/wjordan)) * Fix error on empty :javascript and :css filter blocks [#986](haml/haml#986) (thanks [Will Jordan](https://github.com/wjordan)) * Respect changes in Haml::Options.defaults in `Haml::TempleEngine` options (Takashi Kokubun) * Un-freeze TempleEngine precompiled string literals [#983](haml/haml#983) (thanks [Will Jordan](https://github.com/wjordan)) * Various performance/memory improvements [#965](haml/haml#965), [#966](haml/haml#966), [#963](haml/haml#963) (thanks [Dillon Welch](https://github.com/oniofchaos)) * Enable `frozen_string_literal` magic comment for all .rb files [#967](haml/haml#967) (thanks [Dillon Welch](https://github.com/oniofchaos))

dillonwelch added 4 commits October 25, 2017 19:48

Frozen string

bbea4b3

Revert changes that are slower

9d22438

Add object to map conversion for temple engine

285c988

Revert more frozen strings

d3447c7

dillonwelch commented Oct 26, 2017

View reviewed changes

dillonwelch added 2 commits October 25, 2017 21:52

Get rid of unnecessary extra array

57b1e7a

Revert this change because it's slower in benchmarks

b5a0d7b

k0kubun reviewed Oct 26, 2017

View reviewed changes

dillonwelch added 5 commits October 27, 2017 17:03

parser

a7abfae

true for filters

24d97ab

compiler

4cfb541

temple

f32a81d

finalize

17dd57a

k0kubun merged commit 1d446b3 into haml:master Oct 29, 2017

k0kubun mentioned this pull request Oct 29, 2017

frozen_string_literal: false #962

Closed

dillonwelch deleted the frozen-string-literal branch October 30, 2017 16:36

wjordan mentioned this pull request Feb 28, 2018

FrozenError - can't modify frozen String #982

Closed

k0kubun mentioned this pull request Mar 10, 2018

un-freeze TempleEngine precompiled string literals #983

Merged

aliismayilov mentioned this pull request Sep 10, 2020

Enable frozen string literal #1039

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Frozen string literal changes #967

Frozen string literal changes #967

dillonwelch commented Oct 26, 2017 •

edited

dillonwelch Oct 26, 2017 •

edited

dillonwelch Oct 26, 2017

dillonwelch Oct 26, 2017

k0kubun Oct 26, 2017

k0kubun Oct 26, 2017

dillonwelch Oct 26, 2017

dillonwelch Oct 26, 2017

k0kubun Oct 26, 2017

dillonwelch commented Oct 28, 2017

k0kubun commented Oct 29, 2017

dillonwelch commented Oct 30, 2017

k0kubun commented Nov 6, 2017

amatsuda commented Nov 7, 2017

k0kubun commented Nov 7, 2017

xrl commented Sep 30, 2019

Frozen string literal changes #967

Frozen string literal changes #967

Conversation

dillonwelch commented Oct 26, 2017 • edited

dillonwelch Oct 26, 2017 • edited

Choose a reason for hiding this comment

dillonwelch Oct 26, 2017

Choose a reason for hiding this comment

dillonwelch Oct 26, 2017

Choose a reason for hiding this comment

k0kubun Oct 26, 2017

Choose a reason for hiding this comment

k0kubun Oct 26, 2017

Choose a reason for hiding this comment

dillonwelch Oct 26, 2017

Choose a reason for hiding this comment

dillonwelch Oct 26, 2017

Choose a reason for hiding this comment

k0kubun Oct 26, 2017

Choose a reason for hiding this comment

dillonwelch commented Oct 28, 2017

k0kubun commented Oct 29, 2017

before

after

dillonwelch commented Oct 30, 2017

k0kubun commented Nov 6, 2017

amatsuda commented Nov 7, 2017

k0kubun commented Nov 7, 2017

xrl commented Sep 30, 2019

dillonwelch commented Oct 26, 2017 •

edited

dillonwelch Oct 26, 2017 •

edited