Issue #22274 has been updated by Eregon (Benoit Daloze). Thank you for creating this issue to discuss it. I have several points in no particular order. * There is a [lot of work](https://github.com/ruby/ruby/pulls?q=is%3Apr+IO%3A%3ABuffer) on `IO::Buffer` recently, this is great. But it might also mean some new bugs, maybe we should wait some time so it gets "battle tested" (= in a release 4.1.0, and some time for it to be tested in production for various apps) before declaring it stable? * https://github.com/ruby/ruby/pull/18483 is a large breaking change, it swaps the order of arguments from `length, offset` to `offset, length`. That I think could break many existing usages of IO::Buffer in subtle ways. Which makes me think maybe there is a possibility this needs to be reverted, in which case having it marked stable looks a bit strange. I do hope it doesn't need to be reverted though, or at least if it is that it happens before the 4.1.0 release. I think we have never seen a core API changing positional arguments order before (or at least not before several deprecation/migration phases in between), but since `IO::Buffer` has been experimental so far maybe it's OK? * From my implementation of IO::Buffer in TruffleRuby I recall two things: `#slice` returns a "sliced IO::Buffer" which is pretty tricky to implement as e.g. just copying the `char*` in the slice is not OK, as the parent buffer might be resized, free'd, etc. So I think every slice needs to always go through the parent buffer, and can't assume the parent buffer didn't change. Also slices are writable, which limits a lot of that can be implemented. Currently CRuby does not go through the parent but captures the raw pointer and validates it, which affects semantics, see below. * Error messages seems inconsistent with other core API, e.g. `ArgumentError: Size can't be negative!` vs `ArgumentError: negative array size`. I think consistency is valuable there (otherwise I think it won't "feel" like a core API). I asked Claude to compare the TruffleRuby and CRuby master implementations and find potential issues or behavior that could be clarified/improved: --- I went through the current `master` implementation while checking what TruffleRuby needs to match for stabilization, and ran a differential + concurrency probe against a fresh `master` build (`4.1.0dev 2026-09-08`). A few things seem worth resolving before freezing the interface. ## 1. `set_string` is not lock-protected and can segfault under concurrent `resize` `IO::Buffer#set_string` with a payload ≥ `IO_BUFFER_BLOCKING_SIZE` (1 MiB) segfaults on `master` when another thread resizes the same buffer concurrently: ```ruby Warning[:experimental] = false big = 4 * 1024 * 1024 60.times do b = IO::Buffer.new(big) src = "Z" * (big / 2) t = Thread.new { sleep(rand * 0.0008); b.resize(64 * 1024) rescue nil } b.set_string(src) # [BUG] Segmentation fault in io_buffer_memmove_blocking t.join end ``` ``` -- C level backtrace ------------------ __memcpy_avx_unaligned_erms io_buffer_memmove_blocking io_buffer.c:3011 rb_nogvl thread.c:1820 io_buffer_copy_from io_buffer.c:3063 io_buffer_set_string io_buffer.c:3312 ``` The cause is an asymmetry with `copy`. Both go through `io_buffer_copy_from` → `io_buffer_memmove`, which **releases the GVL** for copies ≥ 1 MiB. `copy` was hardened to lock both buffers (`rb_io_buffer_locked_for_reading` / `_for_writing`, #18489), so a concurrent `resize` hits a locked buffer and cannot `realloc`. `set_string` (`io_buffer_set_string` → `io_buffer_copy_from`) takes **no** such lock, so the resize reallocates the base out from under the in-flight `memmove` → use-after-free. Running each write path against the same race in isolation: | method | result | why | |--------------|-------------|-----| | `set_string` | **SIGSEGV** | releases GVL, does not lock destination | | `copy` | ok | locks both sides | | `clear` | ok | keeps GVL (plain `memset`) | | `read` | ok | locks via `io_buffer_blocking_region` | The fix is presumably to give `set_string` the same locked scope `copy` already uses. ## 2. Slice validity is address-based — two asymmetric edge cases worth ratifying A slice captures an absolute pointer (`source->base + offset`) at creation and re-validates on every access by checking the pointer still falls within the source's current range (`io_buffer_validate_slice`). This correctly catches a freed/transferred source (base → `NULL` → `InvalidatedError`) and a source shrunk past the slice. But because validation is address-containment rather than offset-based, two cases are asymmetric, and I'd suggest confirming they're intended before the interface is frozen: 1. A resize that **moves** the allocation (`realloc`/`mremap` relocating) invalidates the slice **even when `offset + length` would still logically fit**. Reproduced for both `realloc` (grow to 1 MiB) and `mremap` (mapped buffer grown ~8 MiB): slice `valid?` becomes `false` every time. 2. If the source is later reallocated back over the slice's old address (ABA), the slice silently becomes **valid again but now points at semantically-unrelated bytes**. Reproduced: `free` a 64-byte buffer, `resize(64)` back → the stale slice resurrected and read the new contents in 8/10 runs. Memory-safe (still inside a live allocation), but returns wrong data. Case 2 is currently documented as intended (cc706f3956, 4d5ae42629). It's sharp enough to be worth an explicit decision, and relevant to this ticket's request for feedback from other implementations: **an offset-based slice design (the natural fit for TruffleRuby and JRuby, where the backing store can move under a managed GC) cannot reproduce the address-reuse revalidation semantics at all**, and would keep case 1's slices valid rather than invalidating them. If the spec is defined in terms of observable `valid?` results across resize, we'd want it worded so an offset-based implementation can conform. ## 3. Minor points - **`freeze` only guards lifecycle, not contents.** A frozen `IO::Buffer` still accepts `set_value` / `set_string` (only `free` / `resize` / `transfer` raise `FrozenError`). This is surprising for a frozen object and worth either documenting explicitly or reconsidering before stable. - **`initialize` is re-callable and leaks.** `io_buffer_initialize` overwrites `base`/`size`/`flags` without releasing the previous allocation. Re-invoking it on a mapped buffer silently leaks the mapping (`mapped?` flips to `false`, no `munmap`); on an internal buffer it leaks the old `malloc`. Doing it inside `#locked` also zeroes `lock_count`, so the ensuing unlock underflows and raises the misleading `LockedError: "Buffer not locked!"`. It'd be safer for `initialize` to release the old buffer first, or to refuse re-initialization. Happy to file (1) as a separate bug if that's easier to track. --- I think semantically it would be best to use "parent buffer + offset" for slices on CRuby too (cleaner semantics, easier to understand and document, also harder to misuse). Absolute pointers are not available on JRuby at all, and on TruffleRuby for Ruby Strings living in the managed heap (`byte[]`). ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-118864 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/

Issue #22274 has been updated by ioquatix (Samuel Williams). Also re frozen, I think frozen should be equivalent to locked, which can be a fast path for some operations. I don't think a frozen buffer is immutable, as that's a totally different issue (read only at the OS level) and I'm not even sure something Ruby can enforce e.g. a shared mapped buffer can change even if read only if another process has a read/write view of it. ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-118867 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/

Issue #22274 has been updated by matz (Yukihiro Matsumoto). Two points on the semantics. **Slices.** I agree with Eregon that a slice should track a logical `[offset, length)` range of its source, not an absolute address. With address-based validation, a reallocation that lands overlapping but shifted from the old one leaves the slice `valid?` while it refers to a different logical region, and a write through it silently damages live data. That is worse than an invalidated slice. The offset-based design cannot hit this case, it costs the same on every access, and it is the model that JRuby and TruffleRuby can implement. Please specify it rather than leaving it implementation-defined. **Freeze.** A frozen `IO::Buffer` that still accepts `set_string` is surprising. In #22291 I took freezing to mean that the object's own state is immutable, and I would like to keep that meaning. I understand a shared mapping can change from outside, but that is the same as a frozen object holding a reference to a mutable one, and it does not make freezing mean something else. If what you want is a fast path for the locked state, please use a separate name for it instead of `freeze`. Matz. ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-118971 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/

Issue #22274 has been updated by himura467 (Akito Shitara). ioquatix (Samuel Williams) wrote in #note-6:

For the purpose of sharing `IO::Buffer`, `freeze`ing this state is sufficient.

Even on a frozen buffer, `#locked` updates `lock_count` with a non-atomic read-modify-write, so two Ractors calling `#locked` on the same buffer would cause a data race on it. It seems that freezing alone is not sufficient for sharing. ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-119124 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/

Issue #22274 has been updated by ioquatix (Samuel Williams).

Even on a frozen buffer, #locked updates lock_count with a non-atomic read-modify-write, so two Ractors calling #locked on the same buffer would cause a data race on it. It seems that freezing alone is not sufficient for sharing.

As per my design, a buffer that is frozen does not need to update or touch locked_count (in fact, only a buffer with locked_count == 0 can be frozen). Once frozen, all operations become safe, since the buffer cannot be resized, reallocated, freed, etc.

slice.set_string("x")

---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-119126 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/

Issue #22274 has been updated by himura467 (Akito Shitara). ioquatix (Samuel Williams) wrote in #note-10:

As per my design, a buffer that is frozen does not need to update or touch `locked_count` (in fact, only a buffer with `locked_count == 0` can be frozen). Once frozen, all operations (that would otherwise require `locked?`) become safe, since the buffer cannot be resized, reallocated, freed, etc.

Thanks, that makes sense. At the moment a locked buffer can be frozen, and `#locked` on a frozen buffer still increments `lock_count`, so I assume both will be handled as part of the implementation. ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-119128 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/Ruby... * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/