Issue #19315 has been updated by himura467 (Akito Shitara). Following the direction suggested by @Eregon in #note-17, I created a PoC implementation for this issue: https://github.com/ruby/ruby/pull/17045 While maintaining backward compatibility with `RSTRING_PTR()` and `RSTRING_END()`, there are two design points to consider: ### 1. Handling `RSTRING_PTR()` without the GVL To ensure null-termination for shared middle substrings, `RSTRING_PTR()` now calls `str_make_independent_expand`. Since this function requires the GVL, I added a separate code path for when `RSTRING_PTR()` is called without it. However, it is unclear if `RSTRING_PTR()` is ever called without the GVL in practice. I would appreciate feedback on whether this edge-case handling is actually necessary or redundant. ### 2. Naming the new API To prevent breaking existing code that uses `RSTRING_PTR()` and `RSTRING_END()` together, I think `RSTRING_END()` must now guarantee null-termination in the same way as `RSTRING_PTR()`. Otherwise, they could point to different buffers after unsharing. While the name `RSTRING_START()` would pair well with `RSTRING_END()`, the new API and `RSTRING_END()` cannot safely be mixed because `RSTRING_END()` now also triggers unsharing. Therefore, I used the name `RSTRING_RAW_PTR()` as proposed in #note-16. ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315#change-117452 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth it with lazy substring and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://bugs.ruby-lang.org/issues/18598#note-3 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/