Implement callback returning type :string

[larskanis]

Callbacks returning a :string type were not supported so far. It was possible to define such a callback, but the value returned was NULL in any case. This implementation follows the semantics for passing :string parameters to functions as implemented in Call.c. That means in particular usage of StringValueCStr(), so that no zero bytes are allowed in the string.

Special care has to be taken regarding lifetime of the returned text pointer. The C pointer to the string is only valid until the next GC triggering function is called, unless the String object is hold in Ruby outside of the callback block. Even when the object is stored outside of the Ruby block, the text pointer can move when GC.compact is called. So :string returning callbacks can safely be used only, when the referenced text is processed immediately by the calling C function.

Return type :string is already implemented in TruffleRuby, but fails in JRuby and raises a

TypeError: invalid callback return type: STRING

Resolves #751

• Update the Wiki accordingly

@eregon, @headius What do you think about this addition? Should we add :string returning callbacks in JRuby or deny them on all implementations?

[eregon]

I think this is good, and if C calls a Ruby callback and expects a char* I would indeed expect in most causes the C side uses it immediately (it's not something the C side allocated, so the C side must anyway not rely on it to be alive forever).

[larskanis]

@eregon How long is a char* string returned by a callback alive in TruffleRuby? To what point in time must the string pointer be considered as invalid? Could there be any relocation even when the string is still bound in Ruby space (similar to GC.compact).

[eregon]

@larskanis It's similar to what you described above, as soon as the Ruby String is no longer referenced then the native char* pointer can be freed.

The char* pointer itself never moves on TruffleRuby (basically a malloc()) as long as the String is not reallocated (by appending or removing characters from the String).
I think on MRI it's also malloc() if the string is long enough (> 23 chars), but if not it's embedded directly in the struct RString and then I guess it can be moved along the Ruby String object by GC.compact as you said above.

[eregon]

Actually, the current semantics on TruffleRuby are different than I thought (they're not using RSTRING_PTR(), they're using the Truffle NFI type).
@rschatz who wrote Truffle NFI clarified that.
For a callback returning :string, strdup() will be called on it, which means the C code receiving the char* should then free() it.
This avoids any lifetime issue, but requires the caller to free() it.

This is documented here: https://github.com/oracle/graal/blob/master/truffle/docs/NFI.md#string

[larskanis]

Closing this in favor of #782 .

The validity of the char* returned by a callback with string type is not well enough defined. It also seems dependent on the ruby implementation. Therefore we now reject :string return type for callbacks.

@eregon Thank you for investigating the Truffleruby conversion of strings! I also inspected the Truffleruby's FFI code a bit, but wasn't sure about how and when the memory is freed.