Issue #19315 has been updated by himura467 (Akito Shitara). The branch benchmarked in #note-52 also included a change that increases the embeddable size limit in `str_subseq()`, in addition to `SHARABLE_MIDDLE_SUBSTRING`. To isolate the two, I re-ran ruby-bench with the embed-size change reverted from the branch. The `fluentd` improvement seen in #note-52 disappears with the revert (master/branch ratio 1.005), while the embed-size change alone reproduces it (ratio roughly 1.12). So the win observed on ruby-bench appears to be attributable to the larger embed threshold, not to `SHARABLE_MIDDLE_SUBSTRING` itself. I've filed the embed size change separately as [Feature #22186], with the full benchmark results attached there. It is a small self-contained fix that doesn't touch the `RSTRING_PTR()` null-termination contract and doesn't keep parent strings alive, so it avoids the compatibility and memory-retention concerns discussed here. How about pursuing #22186 first? This proposal would still help for slices too large to embed, as the micro-benchmarks show, so I think the two are complementary rather than competing. ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315#change-117945 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth it with lazy substring and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://bugs.ruby-lang.org/issues/18598#note-3 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. ---Files-------------------------------- ruby-bench-ratio.png (230 KB) -- https://bugs.ruby-lang.org/