Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bundle:mem throws SystemStackError: stack level too deep #136

Closed
florianroesler opened this issue Jun 26, 2019 · 12 comments
Closed

bundle:mem throws SystemStackError: stack level too deep #136

florianroesler opened this issue Jun 26, 2019 · 12 comments

Comments

@florianroesler
Copy link

When using the recently released version 1.3.6 I get the following stacktrace upon executing bundle exec derailed bundle:mem:

/app # bundle exec  derailed bundle:mem 
bundler: failed to load command: derailed (/usr/local/bundle/bin/derailed)
SystemStackError: stack level too deep
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:45:in `class_eval'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:45:in `class_eval'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:45:in `block in <class:SaveOptions>'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:44:in `each'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:44:in `<class:SaveOptions>'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:6:in `<class:Node>'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:3:in `<module:XML>'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:2:in `<module:Nokogiri>'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node/save_options.rb:1:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml/node.rb:3:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri/xml.rb:5:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/nokogiri-1.8.3/lib/nokogiri.rb:36:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/loofah-2.2.2/lib/loofah.rb:3:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/rails-html-sanitizer-1.0.4/lib/rails-html-sanitizer.rb:2:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/sanitize_helper.rb:4:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/text_helper.rb:34:in `<module:TextHelper>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/text_helper.rb:31:in `<module:Helpers>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/text_helper.rb:8:in `<module:ActionView>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/text_helper.rb:6:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/form_tag_helper.rb:20:in `<module:FormTagHelper>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/form_tag_helper.rb:16:in `<module:Helpers>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/form_tag_helper.rb:10:in `<module:ActionView>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/form_tag_helper.rb:8:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers/form_helper.rb:6:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers.rb:54:in `<module:Helpers>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers.rb:6:in `<module:ActionView>'
  /usr/local/bundle/gems/actionview-5.2.0/lib/action_view/helpers.rb:5:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/sprockets-rails-3.2.1/lib/sprockets/rails/context.rb:1:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/sprockets-rails-3.2.1/lib/sprockets/railtie.rb:7:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/railties-5.2.0/lib/rails/all.rb:17:in `block in <top (required)>'
  /usr/local/bundle/gems/railties-5.2.0/lib/rails/all.rb:15:in `each'
  /usr/local/bundle/gems/railties-5.2.0/lib/rails/all.rb:15:in `<top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/bin/derailed:85:in `setup_bundler!'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/bin/derailed:66:in `block in <class:DerailedBenchmarkCLI>'
  /usr/local/bundle/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
  /usr/local/bundle/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
  /usr/local/bundle/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
  /usr/local/bundle/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
  /usr/local/bundle/gems/derailed_benchmarks-1.3.5/bin/derailed:92:in `<top (required)>'
  /usr/local/bundle/bin/derailed:23:in `load'
  /usr/local/bundle/bin/derailed:23:in `<top (required)>'

When installing 1.3.5, everything is fine again. I am running the command in an Alpine based Docker container with Rails 5.2. Let me know if I can support with additional information.
@benoittgt
Copy link
Contributor

Hello @florianroesler

Thanks for your issue. Do you think you can provide a way to reproduce this behavior?

Cheers 👋

@florianroesler
Copy link
Author

I tried some stuff again and it seems weirder than I thought. Actually, both versions (1.3.5 and 1.3.6) work sometimes but also fail sometimes with the above trace. So my initial suspicion was just based on pure luck that version 1.3.6 failed and 1.3.5 worked for the few runs I did.

I will keep investigating and try to find a reproducible example! Have you seen similar behaviour before, where running the memory investigation would fail and succeed when executed multiple times?

@benoittgt
Copy link
Contributor

Have you seen similar behaviour before, where running the memory investigation would fail and succeed when executed multiple times?

Sadly not.

@schneems
Copy link
Member

Can you give me a repro example that reproduces the behavior?

https://www.codetriage.com/example_app

@florianroesler
Copy link
Author

Will do as soon as I find the time :)

@florianroesler
Copy link
Author

There you go: https://github.com/florianroesler/derailed_reproduction

The README gives an action plan to reproduce the issue. I used Docker and docker-compose so that the execution environment should be the same for you.

Let me know if you need further assistance.

@schneems
Copy link
Member

schneems commented Jun 28, 2019 via email

@schneems
Copy link
Member

schneems commented Jul 4, 2019

I tried to repro with this setup but could not.

I've got it running in a loop:

while bundle exec derailed bundle:mem; do :; done

Tried on docker and on a mac.

Can you try it out on another machine and see if you can get it to reproduce there?

Performance

I've not triggered the behavior yet. In a strange note, it runs MUCH faster on docker than on my mac

# mac
$ time bundle exec derailed bundle:mem
real	0m15.535s

# docker
$ time bundle exec derailed bundle:mem
real	0m 1.12s

Not sure why, I thought it might be the number of gems installed, but they seem about the same:

$ gem list | wc -l
     132
/app # gem list | wc -l
102

Doesn't look like the issue is bundler version or rubygems version. Pretty bizarre.

@florianroesler
Copy link
Author

Ok, I was really clueless on why you had different results than me. Also checked with a coworker and freshly set up the project, he also couldn't reproduce the error.
After further investigation I found that the underlying image for my Dockerfile (ruby:2.5.1-alpine) had not been updated on my machine in 13 months so I updated it ét voila -- the error was not there anymore.

For the reproduction of the error I have now pinned the image version of ruby:2.5.1-alpine to that very digest that fails for me. Just check out the newest version of the reproduction repository and rebuild the image using docker-compose up --build. I checked with my coworker that it would fail on his machine (Mac OS) as well.

@schneems
Copy link
Member

schneems commented Jul 4, 2019

Thanks for the container and repro.

Is there a way to diff docker images? I don’t want to invest a ton of time here if the issue was a bug in the container instead of in the library. I.e. I’m not sure I would be able to do anything about it in derailed also this is the only case I’ve ever seen of this happening.

@florianroesler
Copy link
Author

Understandable, you shouldn't invest your time here. I am still wondering what underlying differences in the images could lead to a SystemStackError when executing derailed.

I will close this issue. I guess if anyone else ever encounters a similar issue, we can pick this up again. Thank you very much for your assistance!

@schneems schneems mentioned this issue Jul 4, 2019
Closed
@schneems
Copy link
Member

schneems commented Jul 4, 2019

Here's some investigation on why it's so slow on mac zombocom/get_process_mem#31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@schneems @benoittgt @florianroesler