Issue #19315 has been updated by kou (Kouhei Sutou). Dan0042 (Daniel DeLorme) wrote in #note-37:
I think that we don't need to introduce `RSTRING_RAW_END()` or something. We can just recommend `RSTRING_RAW_PTR() + RSTRING_LEN()` instead of `RSTRING_END()`.
Then what was the original purpose of introducing `RSTRING_END()` in the first place? Why not always `RSTRING_PTR() + RSTRING_LEN()` ? I imagine it was introduced to provide a better developer experience (DX) and cleaner code, so going back to only `ptr + len` feels like going back on that API decision.
I'm not sure the original purpose but `RSTRING_END()` was introduced by Matz by commit:a25fbe3b3e531bbe479f344af24eaf9d2eeae6ea . Matz may share the original purpose. I don't object that we add `RSTRING_RAW_END()` or something.
I am highly sympathetic to the argument for backward compatibility, but it must be weighted against other factors. In order to migrate to the new API while maintaining compatibility for `RSTRING_PTR() + RSTRING_END()` we must change from `char *beg = RSTRING_PTR(str), *end = RSTRING_END(str);` to `char *beg = RSTRING_RAW_PTR(str), *end = beg + RSTRING_LEN(str);` and any misuses of `RSTRING_PTR()` to `StringValueCStr()` and as a result be left with no `RSTRING_PTR()` or `RSTRING_END()` anywhere, which is ironic given the effort to preserve their compatibility.
If it happens, it's a success story. It means that there are no problems by this optimization. We can deprecate/remove `RSTRING_PTR()`/`RSTRING_END()` after it. We may reuse them like we did for `Data`.
That feels very awkward to me, to the point that a clean break might be better. Important to note: because this only impacts C extensions and not Ruby code, compatibility issues are caught immediately at compile time, making them straightforward (although annoying) for maintainers to identify and fix.
I concern about gems that don't have any active maintainers. They may not be fixed soon. Users of them may need to migrate to other gems or re-implement features provided by them. I'm not sure whether it justifies the `RSTRING_PTR()` (that is a widely used API) breaking change or not. ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315#change-117522 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth it with lazy substring and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://bugs.ruby-lang.org/issues/18598#note-3 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/