Issue #19315 has been updated by rhenium (Kazuki Yamaguchi). kou (Kouhei Sutou) wrote in #note-27:
I think that the current `RSTRING_PTR()` for non frozen `String` also has similar situation. If arbitrary Ruby code changes the target `String`, pointer returned by `RSTRING_PTR()` may be invalid:
```c char *ptr = RSTRING_PTR(str); // Run "str << large_str" in Ruby // ptr may be invalid here. ```
So I don't feel that this is a new problem of new `RSTRING_RAW_PTR()` API.
That's a fair point. My concern is that such code is probably common in extensions, and currently it often happens to work fine unless a user intentionally tries to trigger the bug. If we go down the route of having `RSTRING_PTR()` lazily allocate a NUL-terminated buffer in it, that is a breaking change on its own because it introduces the possibility of GC and longjmp where they previously weren't possible. This will force some extensions to migrate away from `RSTRING_PTR()`. I think this may be acceptable, but when migrating to `RSTRING_RAW_PTR()`, users should ideally only need to worry about the absence of the NUL terminator. kou (Kouhei Sutou) wrote in #note-32:
Dan0042 (Daniel DeLorme) wrote in #note-29:
`RSTRING_CSTR()` doesn't check whether `\0` is included in the target content or not.
That is a very good point. In that case, perhaps the new NUL-terminated macro should instead alias or behave similarly to `StringValueCStr()`. Otherwise, we risk allowing a NUL-terminated string that contains embedded NUL bytes in the middle. I struggle to see a valid use case for that scenario; it feels like an invitation for bugs and undefined behavior.
I think that we don't need a new macro for the `CStr` behavior. We can just use existing `StringValueCStr()`.
On a related note, while `StringValueCStr()` can work here, I've sometimes wished for a function to ensure a C string without the `StringValue` part (implicit type conversion by `#to_str`) for clarity. Something like: ```c const char *rb_str_cstr(VALUE caller_must_ensure_t_string); ``` ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315#change-117502 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth it with lazy substring and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://bugs.ruby-lang.org/issues/18598#note-3 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/