Add PyString::intern to enable access to Python's built-in string interning.

[adamreichold]

I think the API is somewhat unfortunate but I am not sure how to get around the requirement for a null-terminated string without the cost of the allocation that this API is supposed to avoid in the first place. At least not without a PyUnicode_InternFromStringAndSize.

[birkenfeld]

I'd at least add a version that takes &str. This is so inconvenient that it's going to hurt uptake, and there is an allocation anyway in the Python function.

[adamreichold]

I'd at least add a version that takes &str. This is so inconvenient that it's going to hurt uptake

Do you mean a function taking &str and only checking that it is null-terminated?

and there is an allocation anyway in the Python function.

While that is true for the current implementation, I think this is actually a performance bug as that implementation should probably use PyDict_{Get,Set}ItemString instead of immediately allocating a PyUnicode object.

Or put differently, if this is guaranteed to allocate a temporary PyUnicode object for every call, then this will actually be useless in the context of #2266 and we would probably be better off exposing PyDict_{Get,Set}ItemString ourselves.

[birkenfeld]

I'd at least add a version that takes &str. This is so inconvenient that it's going to hurt uptake
Do you mean a function taking &str and only checking that it is null-terminated?

Either that, or one creating the intermediary CString by itself. It could even do both, by trying first to create a CStr and making a new null-terminated string if necessary. Then you can easily pass "abc\0" for speed, or any user given str for convenience.

[birkenfeld]

Sorry I wasn't thinking properly about the "benchmark dict access with strings" use case.

In general, having an API for interning is still useful for other cases, so I'd like this to go forward independently.

[adamreichold]

I'd at least add a version that takes &str. This is so inconvenient that it's going to hurt uptake
Do you mean a function taking &str and only checking that it is null-terminated?

Either that, or one creating the intermediary CString by itself. It could even do both, by trying first to create a CStr and making a new null-terminated string if necessary. Then you can easily pass "abc\0" for speed, or any user given str for convenience.

Added a separate commit providing a more convenient API using CStr::from_bytes_with_nul as a fast path.

[birkenfeld]

Thanks! BTW, I can never figure out what people mean by the "eyes" reaction, what's your take on that?

[adamreichold]

Thanks! BTW, I can never figure out what people mean by the "eyes" reaction, what's your take on that?

The meaning I wanted to convey was "I am looking into that", so when I am currently thinking about/working on something but do not have any positive or negative results to share yet.

[adamreichold]

In general, having an API for interning is still useful for other cases, so I'd like this to go forward independently.

@mejrs @birkenfeld I rebooted this into a wrapper that is purely targeted at memory efficiency as PyDict_{Set,Get}ItemString actually have the same deficiency as PyUnicode_InternFromString.

[birkenfeld]

LGTM.

[mejrs]

Thanks!