Avoid relocation of strings by moving the content to heap memory #811
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
larskanis
force-pushed
the
avoid-string-reloc-v1
branch
3 times, most recently
from
4aafac0
to
12aa771
Compare
This is due to the fact that ruby-2.7+ relocates emdedded strings by GC.compact . See https://bugs.ruby-lang.org/issues/17023 for more description of the issue. Unfortunately there's no good way to pin string values to a fixed memory location in the ruby C API. The only way seems to be rb_gc_register_address(), but this function would keep a reference to the string object forever, so that it will never be freed. To work around this issue the string capacity is expanded, so that the string content is moved from embedded string memory to ordinary heap memory. The underlying string buffer of such strings is never relocated at GC.compact . There are some issues: 1. String expanding fails with frozen strings, but frozen strings are relocated as well. 2. All strings smaller than 24 characters are copied when passed to the C function. This increases call time by up to 70% although this copying isn't required in almost all cases. It's only needed when GC.compact is used and the string in question is accessed after GC.compact. 3. The string object is modified in a way that changes it's RSTRING_PTR(). That isn't visible in ruby, but could be noticed in conjunction with other C extensions.
larskanis
force-pushed
the
avoid-string-reloc-v1
branch
from
12aa771
to
d301e55
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is due to the fact that Ruby-2.7+ relocates emdedded strings by
GC.compact. See https://bugs.ruby-lang.org/issues/17023 for more description of the issue.Unfortunately there's no good way to pin string values to a fixed memory location in the ruby C API. The only way seems to be
rb_gc_register_address(), but this function would keep a reference to the string object forever, so that it will never be freed.To work around this issue the string capacity is expanded, so that the string content is moved from embedded string memory to ordinary heap memory. The underlying string buffer of such strings is never relocated at
GC.compact.There are some issues:
String expanding fails with frozen strings, but frozen strings are relocated as well.
All strings smaller than 24 characters are copied when passed to the C function. This increases call time by up to 70% although this copying isn't required in almost all cases. It's only needed when
GC.compactis used and the string in question is accessed afterGC.compact.The string object is modified in a way that changes it's
RSTRING_PTR(). That isn't visible in ruby, but could be noticed in conjunction with other C extensions.