Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

File traversal is recursive, which could result in SystemStackError #1375

Open
fsateler opened this issue Mar 17, 2021 · 2 comments · May be fixed by #1484
Open

File traversal is recursive, which could result in SystemStackError #1375

fsateler opened this issue Mar 17, 2021 · 2 comments · May be fixed by #1484

Comments

@fsateler
Copy link

fsateler commented Mar 17, 2021 •
edited

On a large project, it is easy to see lots of (debug) messages like this:

[debug]: Missing object Item in file `app/forms/item/assign_form.rb', moving it to the back of the line.

These are caused by idioms of the type Item::AssignForm, as opposed to

module Item
  class AssignForm

Grepping for that phrase, yields this code in ensure_loaded!:

yard/lib/yard/handlers/base.rb

Lines 566 to 570 in 3590066

while object.is_a?(Proxy)
raise NamespaceMissingError, object if retries > max_retries
log.debug "Missing object #{object} in file `#{parser.file}', moving it to the back of the line."
parser.parse_remaining_files
retries += 1

In particular, parser.parse_remaining_files causes a recursion:

image

On sufficiently large projects, this causes a SystemStackError.

Steps to reproduce

% seq 1 10000 | while read i ; do
echo "module Foo::My${i} ; end" > my_${i}.rb
done
% bundle exec yard doc --debug '*.rb'

Actual Output

<snip>
[debug]: Parsing my_79.rb
[debug]: Missing object Foo in file `my_79.rb', moving it to the back of the line.
[debug]: Parsing my_80.rb
[debug]: Missing object Foo in file `my_80.rb', moving it to the back of the line.
[debug]: Parsing my_81.rb
[debug]: Missing object Foo in file `my_81.rb', moving it to the back of the line.
[debug]: Parsing my_82.rb
[debug]: Missing object Foo in file `my_82.rb', moving it to the back of the line.
[debug]: Parsing my_83.rb
<snip>
[debug]: Missing object Foo in file `my_556.rb', moving it to the back of the line.
[debug]: Parsing my_557.rb
[debug]: Missing object Foo in file `my_557.rb', moving it to the back of the line.
[debug]: Parsing my_558.rb
[debug]: Missing object Foo in file `my_558.rb', moving it to the back of the line.
[debug]: Parsing my_559.rb
[debug]: Missing object Foo in file `my_559.rb', moving it to the back of the line.
[debug]: Parsing my_560.rb
<snip>
bundler: failed to load command: yard (<home>/<gems>/ruby/2.6.0/bin/yard)
Traceback (most recent call last):
        10483: from <home>/.rbenv/versions/2.6.6/bin/bundle:23:in `<main>'
        10482: from <home>/.rbenv/versions/2.6.6/bin/bundle:23:in `load'
        10481: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/exe/bundle:37:in `<top (required)>'
        10480: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/lib/bundler/friendly_errors.rb:130:in `with_friendly_errors'
        10479: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/exe/bundle:49:in `block in <top (required)>'
        10478: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/lib/bundler/cli.rb:24:in `start'
        10477: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
        10476: from <home>/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/bundler-2.2.11/lib/bundler/cli.rb:30:in `dispatch'
         ... 10471 levels...
            4: from <home>/<gems>/ruby/2.6.0/gems/yard-0.9.25/lib/yard/parser/ruby/ruby_parser.rb:237:in `visit_event'
            3: from <home>/<gems>/ruby/2.6.0/gems/yard-0.9.25/lib/yard/parser/ruby/ast_node.rb:64:in `source_range'
            2: from <home>/<gems>/ruby/2.6.0/gems/yard-0.9.25/lib/yard/parser/ruby/ast_node.rb:345:in `reset_line_info'
            1: from <home>/<gems>/ruby/2.6.0/gems/yard-0.9.25/lib/yard/parser/ruby/ast_node.rb:200:in `children'
<home>/<gems>/ruby/2.6.0/gems/yard-0.9.25/lib/yard/parser/ruby/ast_node.rb:200:in `select': stack level too deep (SystemStackError)

Expected Output

Not a crash.

Environment details:

  • OS: Linux
  • Ruby version (ruby -v): ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
  • YARD version (yard -v): yard 0.9.25
  • Relevant software dependency/versions:

I have read the Contributing Guide.
@Skipants Skipants linked a pull request Feb 27, 2023 that will close this issue
Open
4 tasks
@QuentinLemCode
Copy link

QuentinLemCode commented May 11, 2023 •
edited

+1 on this issue
Can you merge the linked PR please ?
It successfully fix the issue on my project

@lsegal
Copy link
Owner

lsegal commented May 11, 2023

@QuentinLemCode the temporary workaround here is to use the module Item; class AssignForm syntax, or simply create an app/forms/item.rb in which you define the outer module:

# app/forms/item.rb:
module Item; end

This would be fully compatible with Rails / Zeitwerk naming conventions and should not create any issues with your existing codebase except for the few extra files.

You could also create a single lib/extras/modules.rb file that contains all of these toplevel modules and included at the top of your .yardopts (given the naming, it would not be Zeitwerk compatible and thus never loaded by Rails, only YARD):

# .yardopts
lib/extras/modules.rb
lib
app

This would only need to be done for widely used namespaces, and only if the first suggestion is not possible.

FWIW, rubocop can auto-fix/auto-format the class Item::AssignForm syntax for you, so there's really no "good" reason to avoid this option for codebases that require it; it is also the default style.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix stack overflow for large projects Skipants/yard
3 participants
@lsegal @fsateler @QuentinLemCode