Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Name the async_cb_thread for easier debugging #883

Merged
merged 6 commits into from Feb 28, 2021

Conversation

ivoanjo
Copy link
Contributor

@ivoanjo ivoanjo commented Feb 19, 2021

I found this useful while reviewing DataDog/dd-trace-rb#1371 -- we were searching for specs that leaked threads, and saw a "mysterious" thread with no stack that we couldn't exactly point out what created it or why it existed.

By setting a name on the callback thread, it becomes immediately obvious when looking at Thread.list what this thread is and where it comes from.

(Thread naming has been around since Ruby 2.3 which means we can do it safely for every supported version)

I found this useful while reviewing DataDog/dd-trace-rb#1371 -- we were
searching for specs that leaked threads, and saw a "mysterious" thread
with no stack that we couldn't exactly point out what created it or
why it existed.

By setting a name on the callback thread, it becomes immediately
obvious when looking at `Thread.list` what this thread is and where
it comes from.

(Thread naming has been around
[since Ruby 2.3](https://github.com/ruby/ruby/blob/v2_3_0/NEWS) which
means we can do it safely for every supported version)
I'll submit this separately.
@larskanis
Copy link
Member

larskanis commented Feb 19, 2021

To name the thread is a great idea! Two thoughts:

  • It's MRI specific, so that JRuby and TruffleRuby needs to be excluded.
  • I think it's useful to name the callback thread as well. Maybe as "FFI::Function Callback"?

I'll take a look at it separately as well.
@ivoanjo
Copy link
Contributor Author

ivoanjo commented Feb 19, 2021

Thanks for the feedback!

It's MRI specific, so that JRuby and TruffleRuby needs to be excluded.

Yeap, already skipped both. It's weird that CI for JRuby seemed to pass even without my change, because locally I did see that it didn't work on JRuby.

I think it's useful to name the callback thread as well. Maybe as "FFI::Function Callback"?

Sounds good! I'll look into it -- I'm guessing it needs a bit of wrapping, because the Thread gets started with the callback directly, so I'll probably need to wrap the callback with the equivalent of a proc that first sets the name on the thread it runs on, and then goes on to run the callback.

@larskanis
Copy link
Member

It's MRI specific, so that JRuby and TruffleRuby needs to be excluded.

Yeap, already skipped both. It's weird that CI for JRuby seemed to pass even without my change, because locally I did see that it didn't work on JRuby.

Unfortunately github actions doesn't support allow_failure so far, so that we use continue-on-error as a workaround.

I think it's useful to name the callback thread as well. Maybe as "FFI::Function Callback"?

Sounds good! I'll look into it -- I'm guessing it needs a bit of wrapping, because the Thread gets started with the callback directly, so I'll probably need to wrap the callback with the equivalent of a proc that first sets the name on the thread it runs on, and then goes on to run the callback.

I think it's OK to set the thread name from the thread calling rb_thread_create() through the result of this function. The same way as for the dispatcher. I didn't find any information about thread safety of Thread#name= but at least the corresponding pthread function is labled thread save and the sample program sets the name from the main thread not from the created thread.

@ivoanjo
Copy link
Contributor Author

ivoanjo commented Feb 22, 2021

I think it's useful to name the callback thread as well. Maybe as "FFI::Function Callback"?

Done! I ended up using "Callback Runner" since it was a bit weird to have a "FFI::Function Callback" and a "FFI::Function Callback Dispatcher", but let me know if you still prefer your original suggestion :)

@ivoanjo
Copy link
Contributor Author

ivoanjo commented Feb 22, 2021

Hmmm I guess I have some debugging to do on windows. I'm guessing it may be due to the racy manner in which we set the name in one thread and then access it in the other.


thread_name = nil

LibTest.testAsyncCallback(proc { thread_name = Thread.current.name }, 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be race-free to store Thread.current here and call Thread#name below. The name is readable even if the thread has terminated.

This was probably failing due to thread naming being a racy operation;
by keeping a reference to the thread and checking it after the
call finishes, this will hopefully be enough for the name to become
available.
@ivoanjo
Copy link
Contributor Author

ivoanjo commented Feb 22, 2021

https://github.com/ffi/ffi/blob/master/spec/ffi/fixtures/FunctionTest.c#L128
Hah! Nice troll!

I can try to name the thread all day long! The "async" callback is not async on windows xD

Took me a bit to debug this ahahaha :)

@larskanis larskanis merged commit 1cce06d into ffi:master Feb 28, 2021
@ivoanjo
Copy link
Contributor Author

ivoanjo commented Feb 28, 2021

Thanks for picking up the last bit @larskanis! I admit I was not having much sucess poking at the win32 bits and fighting with _beginthreadex -- I had gotten so far as to make it work on Ruby 2.7 and 2.6, but was getting segmentation faults on older Rubies.

🙇 thanks for the help!

@ivoanjo ivoanjo deleted the name-dispatcher-thread branch February 28, 2021 19:07
@larskanis
Copy link
Member

No problem. Thank you for contributing this!

This was referenced Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants