- ruby-core - ml.ruby-lang.org

[ruby-core:126187] [Ruby Bug#22216] Regexp backref and IO lastline are incompatible with Ractor
by headius (Charles Nutter) 07 Aug '26

07 Aug '26

Issue #22216 has been reported by headius (Charles Nutter). ---------------------------------------- Bug #22216: Regexp backref and IO lastline are incompatible with Ractor https://bugs.ruby-lang.org/issues/22216 * Author: headius (Charles Nutter) * Status: Open * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ## Problem Several Regexp-matching methods currently write (and sometimes read) the implicit "backref" `$~` variable in the local frame (and related variables like `$'`). Several IO methods read or write the "last line" `$_` variable in the same way. In both cases, the result is a mutable object, which makes these variables already problematic for ractors. Making matters worse, the frame might be shared if a proc is captured and used across threads or ractors, and there's no static way to inspect a piece of code to know if it expects to read or write these variables. Where procs can be rejected by a proc for accessing captured state, there's no such check possible for these variables. All of these facts make the backref and lastline variables fundamentally incompatible with Ractor. ## Possible remedies A wholesale removal of these variables would solve the problem, but there's a lot of code that depends on them... much of that code without even realizing it, since they might not access the variables directly. In some cases, the dependencies are internal and part of the behavior of core methods. Hard errors when using methods that read or write these variables would avoid introducing threading problems into a Ractor, but would also break a large number of commonly-used methods. There have been experiments to make these variables both frame and thread-local, but they have never been made standard. Updates to backref and lastline are visible across threads and already can lead to concurrency issues even on CRuby. Deprecating the implicit behavior and making it opt-in (or opt-out?), perhaps with keyword arguments or file pragmas, might be a halfway measure. It would probably not be an easy transition. I don't know the right path forward, but I believe this issue needs to be discussed. ## JRuby perspective We continue to mimic CRuby behavior, which has led to our users occasionally running into issues when a proc accesses these variables across threads. Our recommendation: "don't do that". We also have had our frustrations optimizing around these variables, since they implicitly require access across calls. Because we cannot statically detect when they will be used, we essentially treat all method names that might *potentially* access them as deopt triggers. It's not ideal. I'd like to hear ideas for how to make these variables less "magic", less implicit and easier to deal with across calls (and across threads/ractors). -- https://bugs.ruby-lang.org/

6 19

[ruby-core:124311] [Ruby Feature#21795] Methods for retrieving ASTs
by kddnewton (Kevin Newton) 07 Aug '26

07 Aug '26

Issue #21795 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #21795: Methods for retrieving ASTs https://bugs.ruby-lang.org/issues/21795 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I would like to propose a handful of methods for retrieving ASTs from various objects that correspond to locations in code. This includes: * Proc#ast * Method#ast * UnboundMethod#ast * Thread::Backtrace::Location#ast * TracePoint#ast (on call/return events) The purpose of this is to make tooling easier to write and maintain. Specifically, this would be able to be used in irb, power_assert, error_highlight, and various other tools both in core and not that make use of source code. There have been many previous discussions of retrieving node_id, source_location, source, etc. All of these use cases are covered by returning the AST for some entity. In this case node_id becomes an implementation detail, invisible to the user. Source location can be derived from the information on the AST itself. Similarly, source can be derived from the AST. Internally, I do not think we have to store any more information than we already do (since we have node_id for the first four of these, it becomes rather trivial). For TracePoint we can have a larger discussion about it, but I think it should not be too much work. In terms of implementation, the only caveat I would put is that if the ISEQ were compiled through the old parser/compiler, this should return `nil`, as the node ids do not match up and we do not want to further propagate the RubyVM::AST API. The reason I am opening up this ticket with 5 different methods requested in it is to get approval first for the direction, then I can open individual tickets or just PRs for each method. I believe this feature would ease the maintenance burden of many core libraries, and unify otherwise disparate efforts to achieve the same thing. -- https://bugs.ruby-lang.org/

6 31

[ruby-core:124714] [Ruby Bug#21870] Regexp: Warnings when using multiple non-overlapping \p{...} classes
by jneen (Jeanine Adkisson) 06 Aug '26

06 Aug '26

Issue #21870 has been reported by jneen (Jeanine Adkisson). ---------------------------------------- Bug #21870: Regexp: Warnings when using multiple non-overlapping \p{...} classes https://bugs.ruby-lang.org/issues/21870 * Author: jneen (Jeanine Adkisson) * Status: Open * ruby -v: 4.0.0, 4.0.1, earlier versions to a lesser extent * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ```ruby $VERBOSE = true # warning: character class has duplicated range: /[\p{Word}\p{S}]/ regex = /[\p{Word}\p{S}]/ ``` As far as I can tell this is a perfectly valid and non-overlapping set of unicode properties, but I am still being spammed with warnings. Using `/(\p{Word}|\p{S})/` is kind of a workaround, but it is slower. -- https://bugs.ruby-lang.org/

9 23

[ruby-core:124418] [Ruby Feature#21826] Deprecating RubyVM::AbstractSyntaxTree
by Eregon (Benoit Daloze) 06 Aug '26

06 Aug '26

Issue #21826 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #21826: Deprecating RubyVM::AbstractSyntaxTree https://bugs.ruby-lang.org/issues/21826 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- I think it is time to deprecate `RubyVM::AbstractSyntaxTree`. [Matz has agreed that going forward the official parser API for Ruby will be the Prism API](https://railsatscale.com/2024-04-16-prism-in-2024/) so it's clear the official Ruby parsing API is the Prism API. `RubyVM::AbstractSyntaxTree` is CRuby-specific and does not work on any other Ruby implementation, the API is unstable, inconvenient, etc: > This module is experimental and its API is not stable, therefore it > might change without notice. As examples, the order of children nodes is > not guaranteed, the number of children nodes might change, there is no > way to access children nodes by name, etc. Because it's under `RubyVM`, it also has the following documented caveats: > This module is for very limited purposes, such as debugging, prototyping, and research. > Normal users must not use it. This module is not portable between Ruby implementations. Given these caveats I think it could possibly be fair enough to remove it without deprecation, however I see no urgency to remove it straight away so I think having it deprecated for one release is nicer for the few gems using it. The advantages of deprecating it are: * Make it clear that this API should not be used in new code * Encourage the few usages to migrate to the Prism API, which is stable, officially supported, designed with many people to be usable and convenient for every parsing/tooling usage, etc. * Improve portability of Ruby code between Ruby implementations So, let's deprecate `RubyVM::AbstractSyntaxTree` in 4.1, and remove it in 4.2? -- https://bugs.ruby-lang.org/

5 6

[ruby-core:125966] [Ruby Feature#22186] Increase the embeddable size limit for substrings created by `str_subseq()`
by himura467 (Akito Shitara) 06 Aug '26

06 Aug '26

Issue #22186 has been reported by himura467 (Akito Shitara). ---------------------------------------- Feature #22186: Increase the embeddable size limit for substrings created by `str_subseq()` https://bugs.ruby-lang.org/issues/22186 * Author: himura467 (Akito Shitara) * Status: Open ---------------------------------------- ## Summary `str_subseq()` unnecessarily limits how large a sharable middle substring can be while still being embedded into the new `RString`. Substrings roughly 24 to 1000 bytes long (on a 64-bit build) always take the shared path, even though variable width allocation (VWA, Feature #18239) already supports embedding objects of that size. I propose sizing the allocation to the actual substring length via `STR_EMBEDDABLE_P()` / `str_alloc_embed()`, the same pattern used by other allocation sites in `string.c`, instead of allocating through `str_alloc_heap()` and checking whether the substring fits in the smallest heap slot class. ## Background For a sharable substring (per `SHARABLE_SUBSTRING_P`), `str_subseq()` either embeds the bytes directly into the new `RString`, or shares the parent's heap buffer, which keeps the parent alive. Since [commit 132f097149](https://github.com/ruby/ruby/commit/132f097149af36cb77308d9fe1c…, the decision has been: ```c str2 = str_alloc_heap(rb_cString); if (str_embed_capa(str2) >= len + termlen) { // embed } else { // share } ``` `str_alloc_heap()` always allocates the default `sizeof(struct RString)` slot regardless of `len`, so `str_embed_capa(str2)` is capped at roughly 23 bytes. Meanwhile the default GC's size pools go up to 1024 bytes on 64-bit builds, and other allocation sites (`str_enc_new` behind `rb_str_new`, `rb_str_buf_new`, `str_new_frozen_buffer`, `rb_str_times`, ...) already pick the right-sized slot via `STR_EMBEDDABLE_P()` / `str_alloc_embed()`. ## Proposed change Draft PR: https://github.com/ruby/ruby/pull/17723 Check `STR_EMBEDDABLE_P(len, termlen)` up front and allocate with `str_alloc_embed()`; only allocate the `STR_NOEMBED` heap object on the sharing branch. Behavior is unchanged for substrings that aren't sharable or that reach the end of the parent's buffer. Only the threshold at which a sharable middle substring switches from embedding to sharing grows. ## Benchmark results Using [ruby-bench](https://github.com/ruby/ruby-bench), this branch vs current master (`arm64-darwin25`, `+PRISM`). Top 3 and bottom 3 by ratio (master/branch, higher is better): | bench | master (ms) | current (ms) | ratio | | ---- | ---- | ---- | ----- | | fluentd | 258.6 | 230.8 | 1.120 | | ruby-json | 154.1 | 150.2 | 1.026 | | liquid-c | 31.2 | 30.6 | 1.022 | | graphql-native | 172.0 | 174.9 | 0.983 | | activerecord | 148.7 | 153.0 | 0.972 | | psych-load | 1244.9 | 1291.4 | 0.964 | Note that most except fluentd varies between runs. The fluentd improvement reproduces across runs, while the entries in the bottom 3 do not. So the few 2-4% slowdowns may be noise, but I'd like to verify them with more runs and machines before treating this as risk-free. ## Relation to Feature #19315 #19315 (sharable middle substrings) addresses the same underlying problem: middle substrings copy unnecessarily. The branch benchmarked in https://bugs.ruby-lang.org/issues/19315#note-52 actually included this embed size change as one of its commits, in addition to enabling `SHARABLE_MIDDLE_SUBSTRING`. To isolate the two, I benchmarked a variant of that branch with only the embed size change reverted ([rstring-raw-ptr-reverted](https://github.com/himura467/ruby/tree/rstring-ra…) alongside this proposal, against the same master: | bench | master (ms) | this proposal (ms) | #19315 without this change (ms) | | ---- | ---- | ---- | ---- | | fluentd | 256.9 | 207.2 (x1.240) | 255.6 (x1.005) | The clear `fluentd` improvement seen in note-52 reproduces with this change alone, and disappears when this change is reverted from the #19315 branch. So the practical win observed on ruby-bench so far is attributable to the larger embed threshold rather than to `SHARABLE_MIDDLE_SUBSTRING` itself. Beyond that, this change is much narrower: - A self-contained fix to one function, reusing the VWA embedding infrastructure already battle-tested elsewhere in `string.c`. - No API/ABI implications; `RSTRING_PTR()`'s `\0`-termination guarantee is untouched. - No memory-retention risk, since embedding copies the bytes instead of referencing the parent. The two are orthogonal (#19315 still helps for slices too large to embed, as its micro-benchmarks show), but this smaller fix seems worth landing first on its own merits. ---Files-------------------------------- output_001.txt (7.76 KB) output_001-ratio-current.png (184 KB) output_002.txt (14.9 KB) -- https://bugs.ruby-lang.org/

3 15

[ruby-core:126279] [Ruby Feature#22231] Add `IO::Buffer#index`
by himura467 (Akito Shitara) 06 Aug '26

06 Aug '26

Issue #22231 has been reported by himura467 (Akito Shitara). ---------------------------------------- Feature #22231: Add `IO::Buffer#index` https://bugs.ruby-lang.org/issues/22231 * Author: himura467 (Akito Shitara) * Status: Open ---------------------------------------- ## Use case `IO::Buffer` has no search primitive: no `#index`, no delimiter scan, no substring search. This matters for servers that hand request data to the application as zero-copy `IO::Buffer` views over their own socket read buffer. The zero-copy path ends at the first delimiter scan, such as splitting a query string on `&`, finding a multipart boundary, or scanning a header value, which is where request processing usually begins. The same applies to any protocol parser built on `IO::Buffer`: line framing, chunked transfer encoding, netstrings, length-prefixed records. All of them must find a delimiter before they can decide what to slice. Today there are two ways to do it, and both give up what the buffer was for: ```ruby buffer.get_string.index("&") # copies the whole region to find one offset i = 0 # no copy, but a method call per byte i += 1 while i < buffer.size && buffer.get_value(:U8, i) != 0x26 ``` Searching a 64 KiB body by copying it out is 3.7x slower than searching it in place, and allocates 66 KB per call. The byte loop allocates nothing but is roughly 1570x slower. ## Specification ```ruby buffer.index(object, offset = 0, length = size - offset) # => Integer or nil ``` `object` may be an `Integer` byte value (`0..255`), a `String`, or another `IO::Buffer`. ```ruby buffer = IO::Buffer.for("Hello World") buffer.index("World") # => 6 buffer.index("o".ord) # => 4 buffer.index(IO::Buffer.for("World")) # => 6 buffer.index("!") # => nil buffer.index("o", 5) # => 7 (absolute, not relative to offset) buffer.slice(6, 5).index("o") # => 1 (relative to the slice) ``` - Searching is byte-oriented; a `String`'s encoding is ignored. - An empty `object` matches at `offset`, as with `String#index`. - An `object` longer than the range returns `nil`. - An out-of-range `offset` or `length` raises `ArgumentError` (see below). Single-byte values use `memchr`. Longer values reuse `rb_memsearch`, the portable substring search that already backs `String#index`. ## Open questions ### Out-of-range `offset`/`length` raises, where `String#index` returns `nil`. All ten existing `(offset, length)` methods in `io_buffer.c` route through `io_buffer_validate_range` and raise, and the class never clamps. `String#index` also has no `length` parameter, so matching it does not settle what `length` should do; the Ruby-wide convention there is to clamp, as in `"hello".byteslice(0, 1000)`, which is what `IO::Buffer` declines to do elsewhere. I chose consistency within the class, and am happy to switch if the `String` reading is preferred. ### An `Integer` outside `0..255` raises, where `#clear` masks it. `buffer.clear(256)` fills with `0`, and `String#setbyte` masks too. Masking a value being searched for seems worse than masking one being written, since `index(256)` would quietly search for `\x00` and could return a match. ## Follow-ups `#rindex` for a reverse scan, and an `#each_until(delimiter)` framing iterator yielding successive delimited slices, in the same spirit as the vectorizable `#and!` / `#or!` / `#bit_count` family. Both build on `#index`, so this ticket is scoped to it. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:126315] [Ruby Feature#14394] Class.descendants
by shugo (Shugo Maeda) 06 Aug '26

06 Aug '26

Issue #14394 has been updated by shugo (Shugo Maeda). Status changed from Assigned to Closed Module#descendants was added instead of Class#descendants, so I'm closing this issue. See #9779 for details. ---------------------------------------- Feature #14394: Class.descendants https://bugs.ruby-lang.org/issues/14394#change-118425 * Author: ridiculous (Ryan Buckley) * Status: Closed * Assignee: ko1 (Koichi Sasada) ---------------------------------------- There have been numerous implementations of the method Class.descendants by various gems. However, I can't help but think that this ability should be included in the Ruby language itself. Especially since Ruby already offers the counterpart method Class.ancestors. Would it possible to add a `descendants` class method? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:126268] [Ruby Misc#22230] Should Ruby specs define byte oriented read behavior when the character buffer is not empty?
by javanthropus (Jeremy Bopp) 06 Aug '26

06 Aug '26

Issue #22230 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Misc #22230: Should Ruby specs define byte oriented read behavior when the character buffer is not empty? https://bugs.ruby-lang.org/issues/22230 * Author: javanthropus (Jeremy Bopp) * Status: Open ---------------------------------------- In https://github.com/ruby/spec/pull/1384, there was a discussion regarding whether or not raising an IOError when performing a byte oriented read operation while the character buffer is nonempty should be part of the Ruby spec. @eregon asked me to remove the new tests I was requesting to add and also remove existing tests that would define this behavior since it was unclear if the behavior in CRuby was an intentional design decision or a consequence of its specific implementation. I intend to remove the tests in the above MR via a single commit that is easy to revert pending the outcome of this ticket as requested. In all cases I tested, CRuby raises IOError for all byte oriented read operations whenever the character buffer is nonempty. `IO.getc` is used in all these cases to ensure that the character buffer has data in it, but it was argued that other implementations could instead make `getc` reverse the encoding conversion (if one is in play) and put the bytes into the byte buffer instead, eliminating the character buffer entirely. If that conversion would fail, `getc` could raise a conversion error of some kind and leave the buffer unchanged. It's unclear to me though if there are other ways that content could be left in the character buffer during normal read operations which would be harder to avoid while preserving performance and semantics. CRuby's implementation appears to be consistent, so I don't think these tests are attempting to enforce buggy behavior. Should this behavior be codified as part of the spec, or should other implementations be free to behave differently? -- https://bugs.ruby-lang.org/

3 2

[ruby-core:126238] [Ruby Feature#22227] Per-Ractor GC: collect each Ractor's heap locally, stop the world only when needed
by ko1 (Koichi Sasada) 06 Aug '26

06 Aug '26

Issue #22227 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #22227: Per-Ractor GC: collect each Ractor's heap locally, stop the world only when needed https://bugs.ruby-lang.org/issues/22227 * Author: ko1 (Koichi Sasada) * Status: Open ---------------------------------------- Companion PR: https://github.com/ruby/ruby/pull/18194 --- ## Abstract This proposal makes the default GC operate per Ractor. Each Ractor owns an objspace (a private heap) and collects it on its own thread without stopping any other Ractor ("local GC"). A stop-the-world collection over all objspaces ("global GC") runs only when something genuinely global has to be reclaimed: shareable objects, memory retained across Ractor boundaries, and the heaps of dead Ractors. With this change, allocation-heavy Ractor programs scale like forked processes, and Ractor communication can even beat interprocess communication; end-to-end single-Ractor workloads stay within 2-4% of current performance (focused allocation microbenchmarks move a few percent in both directions, see section 4.4). ## 1. Motivation Today all Ractors allocate from one shared objspace, and every GC stops every Ractor. This has two consequences: 1. GC pause time grows with the total heap of all Ractors, while the pause still stops everyone. N allocating Ractors trigger GCs N times as often, and each GC scans N heaps. 2. The synchronization needed to stop N mutators at every GC dominates as N grows. In practice, a program that parses JSON in 16 Ractors spends 10x the wall clock of its single-Ractor version (for the same per-worker work), while the identical code in 16 forked processes stays at the hardware's parallel limit of about 3.3x. Ractors currently lose to fork exactly on the workloads they were designed for. The root cause is architectural, not incremental: CRuby's current shared objspace couples every Ractor to the same stop-the-world GC. This proposal splits the heap. ## 2. Design overview Terminology used below: * **objspace**: one Ractor's private heap: its pages, freelists, mark stack, generational state and counters. * **local GC**: a collection of a single objspace, run by its owner thread, with no VM lock and no barrier. Other Ractors keep running. * **global GC**: a stop-the-world mark/sweep over all objspaces as one heap. Heap traversal is exact; machine-stack roots remain conservative. * **shareable object**: `Ractor.shareable?(obj) == true` (immutable values, Class/Module, shareable Procs, and so on). * **shref** ("shareable reference"): an unshareable object directly referenced by a shareable one. Details are given in 2.1 and 2.3. ### 2.1 The containment invariant The correctness of a local GC rests on one invariant: > An unshareable object is reachable only from its owner Ractor, except > through edges that the GC has been explicitly told about (shrefs and > in-flight messages). The invariant has an equally important flip side: a **shareable object can be referenced from any Ractor at any time, and nothing records who references it**. A local GC therefore never collects a shareable object, and never even traverses one; it only marks them all as live (2.3). Only a global GC, whose unified mark sees every Ractor's references, can prove a shareable object dead and reclaim it. In other words: unshareable objects are the local GC's responsibility, shareable objects are exclusively the global GC's. Ruby-level Ractor semantics already forbid directly sharing unshareable objects. CRuby, however, has a small number of implementation-level edges that cross the boundary, and the GC work is to make every one of them visible to the collector: * A shareable object can hold an implementation-level reference to an unshareable one: for example, writing through a shareable Proc's environment stores an unshareable value into a shareable env, and a shareable object's hidden instance variables (its fields imemo) can stay unshareable. The **write barrier** detects every such store and records the target as a **shref**. The transition that makes an object shareable likewise registers any pre-existing outgoing edges of this kind (for example, the hidden field values that `Ractor.make_shareable`'s traversal does not reach are shref-recorded at promotion time). * A message in flight (sent, not yet received) lives in the sender's objspace while being reachable only from queue structures. The send path **pins** the payload in the sender's objspace until the receiver materializes it (2.6). Given the invariant, marking one objspace's roots reaches everything a local GC is allowed to free, so a local GC requires no process-wide barrier and never stops another Ractor; its synchronization is confined to leaf mutexes and atomic bitmap operations on shared metadata (2.3). `GC.verify_internal_consistency` walks the heap and checks the invariant edge by edge (builds with `RGENGC_CHECK_MODE` also run it around every collection); it caught most of the bugs found during development. ### 2.2 Heap layout and ownership Every heap page records its owning objspace, so any object's owner is one load away: `GET_HEAP_OBJSPACE(obj)` reads the page header. There is no per-object tag; ownership is a property of the page. Page bodies come from a process-wide **page pool**: large mmap arenas carved into page-sized bodies and recycled through a freelist under a dedicated mutex. Ractors therefore exchange pages (allocation, death, inheritance) without mmap/munmap churn, and without serializing on the kernel's mmap lock. A process-wide **page index** (all pages of all objspaces, sorted by body address) supports the conservative machine-stack scan of a global GC and ownership tests; only stop-the-world code reads it, so it needs no reader lock. Allocation is a bump-pointer fast path (a cursor advancing through free regions) into per-objspace, per-size-class heaps. The upstream per-Ractor newobj cache became unnecessary (the objspace itself is per Ractor), which makes the allocation fast path about 9% faster. ### 2.3 What a local GC does A local GC is the classic incremental/generational collector of CRuby, restricted to one objspace, plus three rules: 1. **Never free a shareable object.** At the end of marking (`gc_marks_finish`), a dedicated pass (`pinned_roots_mark`) marks every shareable object on the objspace's pages that ordinary marking did not reach: mark bit only, no traversal. Another Ractor may hold the only reference to it; only a global GC can prove a shareable object dead. The count of objects kept alive this way is also a useful retention metric (an upper bound on garbage only a global GC can reclaim). 2. **Treat shrefs as roots.** A shref is marked and traversed like a remembered old-to-young target: the shareable parent is never traversed by a local GC (rule 1), so without this its unshareable children would look unreachable. 3. **Never touch a foreign object's GC state.** Marking stops at any object owned by another objspace ("foreign objects are live leaves"): its mark/age/remembered bits belong to its owner, and writing them would race the owner's local GC. Because objects can become shareable at any time between collections, rule 1 scans the shareable/shref bitmaps in every mark cycle rather than trying to maintain a pin set across sweeps. Generational collection stays per objspace. The remembered-set bitmap has exactly one class of cross-thread writers (another Ractor's write barrier remembering a shareable object), and that path uses an atomic bitmap set; everything else is single-writer under the owner's GVL. **Locking model.** A local GC takes neither the VM lock nor any barrier; it never stops or waits for another Ractor. The main Ractor's local GC is the one exception in two bounded windows: it walks VM-global roots and weak tables (`rb_vm_mark`, freeing entries of fstring/symbol/ci tables), which other Ractors rewrite under the VM lock, so it takes a no-barrier VM lock just for those stretches; and when a JIT is enabled, marking reaches JIT payloads of shareable iseqs, which must exclude concurrent compilation. One rule is absolute: a GC must never take the barrier VM lock mid-collection, because the waiter would join a pending global barrier and expose its half-collected heap to the global GC. Shared structures that GC paths touch (page pool, the generic-fields table, registered globals) use their own leaf mutexes instead. **Triggers.** Every local GC entry first asks whether a *global* cycle is needed instead (see 2.4); otherwise the usual malloc/allocation heuristics drive local minor/major cycles per objspace. ### 2.4 What a global GC does A global GC stops the world using the existing barrier mechanism (a GC has no safepoints, so the barrier implicitly waits for every in-flight local GC to finish). Then, with the driver thread executing every phase: 1. **Settle**: finish every objspace's lazy sweep, so mark bits have a single meaning. 2. **Clear**: clear every objspace's mark bits, remembered sets, age metadata and all shref bits. (Missing any of these would leave stale bits and cause use-after-free; this step is why shref bits may only be cleared by a global GC.) 3. **Unified mark**: mark from every Ractor's roots (C-struct roots, machine stacks scanned conservatively against the global page index, VM globals) and traverse exactly, crossing objspace boundaries freely. The write barrier's shref information is re-derived as a byproduct: each shareable-to-unshareable edge encountered re-records the shref bit. In-flight message payloads are re-pinned here (their shref bits were cleared in step 2). 4. **Weak passes**: the generic-fields table (weak keys) is processed to a fixpoint: values (`fields_obj`) of live keys are marked, entries of dead keys are dropped, and since marking a value can make another key live, the pass repeats until no progress. VM-global weak tables (fstring, symbols, call caches and so on) are swept once, not once per objspace. 5. **Sweep**: sweep every objspace inside the barrier (not lazily). This is the only place shareable objects and cross-objspace cycles die. Emptied pages return to the page pool. 6. **Budget**: objspaces left with no free pages get the same heap-growth budget a local full mark would have granted (a global GC bypasses that bookkeeping), so the next allocation does not fail. 7. Shareable-object populations and trigger limits are recounted per objspace. Global GCs are triggered by: an explicit full `GC.start`; growth of the shareable-object population past a per-objspace limit; and accumulation of heap pages held by dead Ractors (see 2.5). Two Ractors deciding to run a global GC at the same time are serialized by the barrier; the second cycle is wasted work, not an error. **Compaction.** With multiple objspaces, `GC.compact` (and autocompact on full marks) runs inside a global GC as three passes over all objspaces: (a) move every objspace's movable objects, leaving forwarding; (b) update references everywhere, following forwarding across objspaces, plus the VM-global side once; (c) sweep and free emptied source pages. It must be three global passes rather than per-objspace work because a reference can point at a moved object in another objspace, and freeing a source page early would let another objspace's update read freed forwarding. A local (non-stop-the-world) compaction runs only while the process has a single objspace; objects that C structs of other Ractors point at (an in-flight payload, a terminated Ractor's pending value) are pinned so they never move even under global compaction. Incremental marking similarly runs only in single-objspace mode; the transition point (`Ractor.new` making the process multi-Ractor) settles any cycle in progress. ### 2.5 Ractor lifecycle and memory hand-off **Creation.** A Ractor's objspace is created before its thread starts, and the child's main Thread and root Fiber wrapper objects are allocated directly into it. Between those allocations and the child joining the Ractor set, the creator "covers" the child objspace through a dedicated slot so a concurrent global GC still enumerates it (an unenumerated objspace would keep stale mark bits). Every failure path of creation (isolation errors, thread-creation failure) hands the child objspace over for reclamation rather than leaking it. **Termination.** A terminating Ractor runs one last local GC with a by-then minimal root set on its own thread ("retire GC": roots are minimal by then, and this reclaims garbage that would otherwise be inherited), and its objspace becomes a **zombie**: registered in a VM-wide table, mutated by nobody, but enumerated by every global GC. Ractor death therefore does not stop the world. **Inheritance.** `Ractor#value` (and join) *absorbs* the dead Ractor's objspace into the caller's: pages are spliced wholesale into the caller's heaps (kept sorted by body address for the conservative scan), the finalizer table entries move over, malloc-accounting counters transfer, and the caller's next collection is forced to a full mark to rebuild generational state. This is why `Ractor#value` can return the value **by reference** with no copy: after the absorb it is an ordinary object of the caller. If a dead Ractor's object is itself collected without ever being joined, its orphan objspace is merged into the main Ractor at main's next safepoint (scheduled as a postponed job, because the discovery happens inside a sweep). At VM shutdown all remaining zombies merge into main so at-exit processing sees every object. Because a zombie's pages still hold shareable objects that other Ractors can reach, zombies participate in the global-GC trigger: accumulated zombie pages past a threshold start a global cycle (measured against what survived the previous one, so a zombie with much live data does not retrigger forever). ### 2.6 Message passing: copy `Ractor#send` (copy semantics) became a two-phase protocol: 1. **Snapshot (sender side).** The sender deep-copies the object graph inside its own objspace using a native copier for core types (String, Array, Hash, Struct, T_OBJECT and so on); types the native copier does not support fall back to a Marshal round trip. `#clone` and `initialize_copy` are never invoked, although Marshal-specific hooks (`marshal_dump`, `_dump`) run on the sender for fallback types and can raise there. Every node of the snapshot is **pinned** in the sender's objspace, and the pin cover is handed off without gaps: from the builder's capture list, to the basket (queue entry), to the receiver's materialization frame. A global GC re-pins from whichever holder is current (its clear pass drops all shref bits, see 2.4 step 2/3). 2. **Materialize (receiver side).** When the receiver takes the message out of the port, it rebuilds the graph inside its own objspace. The sender-resident snapshot becomes garbage once the copy completes. Materialization can raise (Marshal load hooks and autoload run user code; asynchronous interrupts can arrive) and can nest (`Ractor.receive` inside a load hook), so in-progress materializations form a per-EC chain of machine-stack frames that the GC uses as roots. Passing the sender-resident graph to the receiver by reference is never allowed: it would create an unshareable cross-objspace edge that neither local GC could handle. ### 2.7 Message passing: move `Ractor#send(obj, move: true)` serializes the payload graph into an off-heap, malloc-allocated structure called the **courier**. While a moved graph is in flight, there is *no heap object* representing it: neither side's GC can mark, sweep, move, or race with it. * A read-only **preflight** walks the graph first and raises every "can not move" error before anything is mutated (capture is destructive, so failing halfway would corrupt the graph beyond repair). * **Capture** turns each source object into a node in the courier's node array (identity and cycles are preserved through node ids and a by-address dedup table) and immediately neutralizes the source into a `Ractor::MovedObject` shell, without ever passing through an inconsistent state that a concurrent marker could observe. * Payload internals move by ownership transfer where possible: a String that owns its buffer hands the pointer over (zero copy); an IO carries its whole `rb_io_t` including the file descriptor; MatchData dumps its match registers into an onig-independent blob. Shareable references and immediates are carried as values; classes of the moved objects are carried as references (classes are shareable), so subclasses and singleton classes survive the move. * In-flight couriers are registered in a VM-global list; a global GC's root pass marks and pins the shareable references they carry (nothing else roots them in some windows). * **Rebuild** on the receiver allocates all shells first, then fills references (two passes break cycles), and inserts Hash entries last, in reverse id order, so user-defined `#hash` methods observe fully built keys. Rebuilding can raise; an unconsumed courier is freed by the queue teardown, closing descriptor leaks for moved IOs. ### 2.8 Generic instance variables Instance variables of non-T_OBJECT hosts (`str.instance_variable_set` and friends) live in one process-global table keyed weakly by host object, guarded by a dedicated leaf mutex. A local GC reads it per object while marking (it cannot wait for the VM lock there); the global GC processes it in the weak pass described above; per-object cleanup happens when a host is freed. An earlier design with per-Ractor tables plus a shared table was abandoned: entries would have had to migrate on every make-shareable, send and inheritance, and that boundary was a reliable source of bugs. ### 2.9 Modular GC API The GC implementation interface (`gc/gc_impl.h`) gains per-Ractor entry points: `multi_objspace_p`, `objspace_absorb`, `objspace_retire_gc`, `obj_foreign_p`, `during_global_gc_p`, `obj_became_shareable`, `pin_in_flight_message`, `each_objects_shareable`, `each_objects_foreign`, `heap_page_count`, `gc_rest`. An implementation that keeps a single objspace (MMTk) reports `multi_objspace_p() == false` and the VM gates the whole per-Ractor machinery out: every Ractor aliases the one objspace, retire/absorb/zombie handling become no-ops, and Ractor-related marking falls back to following everything from the wrapper objects. MMTk continues to pass its test suite unchanged. ## 3. Compatibility * `Ractor#value` becomes a one-shot ownership transfer: the first call hands the value over by reference together with the dead Ractor's heap, and a second call raises `Ractor::Error` ("The value was already taken"). Returning the same unshareable object again would hand out a reference to an object now owned by the first caller, recreating exactly the cross-Ractor sharing that Ractor semantics forbid; special-casing repeat calls from the same Ractor, or shareable values only, would make the API's behavior depend on the caller and the value's type, so the simple rule was chosen. (Previously repeated calls returned the same object.) * `GC.start` in a multi-Ractor process runs a stop-the-world global GC; automatic GC in a Ractor collects only its own heap. * `GC.disable`/`GC.enable` become per-Ractor holds on a process-wide switch: any Ractor's disable stops automatic GC everywhere, but another Ractor's `GC.enable` no longer silently overrides it. A Ractor's hold is released when it exits. * `ObjectSpace.each_object` enumerates the calling Ractor's own objects plus other live Ractors' shareable objects, never their unshareable ones. Objects of a terminated but not yet inherited Ractor are not enumerated until its heap is absorbed. (`ObjectSpace.dump_all` and `memsize_of_all` still cover every object in the process; they walk the other objspaces under a global barrier.) * `ObjectSpace.define_finalizer` on another Ractor's object raises `Ractor::IsolationError`; finalizers are registered, stored and run in the owner's objspace. * The `RUBY_INTERNAL_EVENT_FREEOBJ` tracepoint fires only for the main Ractor's objspace (it runs user callbacks from inside the sweep, which is unsafe in a lock-free non-main sweep; extending it is future work). * Invalidated call-cache entries are no longer pruned during GC marking; they are reclaimed at the next lookup of that method or when the class dies. Retention is bounded (one entry per class-and-method-id pair); the worst-case microbenchmark retains approximately 2.7 MB for 200,000 stale entries. ## 4. Performance Setup: 16 logical CPUs (8 cores, SMT), x86_64-linux, `-O3` release builds. `master` is the merge-base of the PR, `RLGC` is the branch. Every cell is wall-clock seconds until all N workers finish (lower is better). Because the machine's clock state drifts on multi-minute scales, each RLGC/master pair was measured interleaved back to back, and each number is the minimum over at least 10 runs spread across several sessions. All three programs are weak-scaling benchmarks: every worker runs the full workload independently, so with perfect parallelism the wall clock stays flat as N grows. The ratio tables divide every cell by master-Ractor N=1, i.e. "how many times the wall time of current Ruby running one Ractor"; a perfectly scaling configuration keeps its N=1 ratio at every N. "Ractor" rows run the N workers as Ractors, "fork" rows run the identical code in forked processes: the no-shared-GC baseline that Ractors should match. Scripts are about 40 lines each and attached to the PR. ### 4.1 binary-trees (D=16; allocation/GC-heavy) The classic benchmarks-game workload: each worker builds and checksums millions of short-lived binary trees (3-element arrays). Allocation and GC dominate; objects die young. seconds: | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 3.83 | 4.01 | 4.48 | 5.76 | 10.26 | | master Ractor | 3.65 | 4.18 | 5.32 | 8.18 | 14.90 | | RLGC fork | 3.75 | 3.99 | 4.48 | 6.56 | 10.22 | | master fork | 3.66 | 3.90 | 4.40 | 6.40 | 10.03 | ratio (x master-Ractor N=1): | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 1.05 | 1.10 | 1.23 | 1.58 | **2.81** | | master Ractor | 1.00 | 1.15 | 1.46 | 2.24 | 4.08 | | RLGC fork | 1.03 | 1.09 | 1.23 | 1.80 | 2.80 | | master fork | 1.00 | 1.07 | 1.21 | 1.75 | 2.75 | RLGC Ractors scale exactly like forked processes (2.81 vs 2.75-2.80 at N=16); master Ractors are 1.45x behind at N=16. For calibration, a zero-allocation pure-integer loop on this machine goes from 5.9s (N=1) to 10.4s (N=16), so RLGC's 10.26s at N=16 is the hardware's SMT ceiling, not a GC limit. ### 4.2 JSON parse (embarrassingly parallel) Each worker parses its own copy of the same ~30 KB JSON document (200 nested user records) 8000 times and discards each result: pure parse-allocate-discard churn, no data crosses workers. This isolates how allocation and GC scale. seconds: | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 1.59 | 1.70 | 1.89 | 2.61 | 4.93 | | master Ractor | 1.49 | 2.06 | 2.63 | 5.24 | 15.10 | | RLGC fork | 1.50 | 1.73 | 1.88 | 2.53 | 4.87 | | master fork | 1.48 | 1.64 | 1.81 | 2.48 | 4.94 | ratio (x master-Ractor N=1): | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 1.07 | 1.14 | 1.27 | 1.76 | **3.32** | | master Ractor | 1.00 | 1.39 | 1.77 | 3.53 | 10.17 | | RLGC fork | 1.01 | 1.17 | 1.27 | 1.71 | 3.28 | | master fork | 0.99 | 1.10 | 1.22 | 1.67 | 3.33 | master Ractors fall off almost immediately (1.39x already at N=2: every Ractor's allocation churn stops all the others) and end 10x over the baseline at N=16; RLGC matches the fork rows within noise the whole way and finishes 3.1x faster than master Ractors. ### 4.3 JSON producer/consumer (Ractor move vs process IPC) The "coordinator hands work to parsers" pattern: the main Ractor/process produces 20000xN JSON strings (~6 KB each) and distributes them round-robin to N workers, which parse and discard them. The Ractor version transfers each string with `send(move: true)`; the fork version writes it down a pipe (a kernel copy). This measures the data-transfer path, not just parallel GC. seconds: | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 0.77 | 0.89 | 1.05 | 1.59 | 2.71 | | master Ractor | 0.71 | 0.99 | 1.68 | 3.87 | 14.59 | | RLGC fork | 0.80 | 0.85 | 0.98 | 1.87 | 3.02 | | master fork | 0.74 | 0.80 | 0.97 | 1.83 | 3.04 | ratio (x master-Ractor N=1): | | N=1 | N=2 | N=4 | N=8 | N=16 | |----------------|------|------|------|------|------| | RLGC Ractor | 1.08 | 1.25 | 1.48 | 2.25 | **3.82** | | master Ractor | 1.00 | 1.40 | 2.37 | 5.45 | 20.55 | | RLGC fork | 1.12 | 1.19 | 1.39 | 2.63 | 4.25 | | master fork | 1.04 | 1.12 | 1.37 | 2.58 | 4.29 | At N=16 RLGC's `move` beats even process IPC (2.71s vs 3.02-3.04s: no kernel copy), while master Ractors are 5.4x behind it. The off-heap courier costs ~8% over master's in-heap move at N=1 (0.77 vs 0.71); that is the price of making in-flight messages invisible to both sides' GCs, and a zero-copy fast path remains possible future work. ### 4.4 Single-Ractor performance Measured with no `Ractor.new` at all (the N=1 rows above already run one worker Ractor, which puts the process in multi-Ractor mode). The allocation fast path itself got faster: the per-Ractor newobj cache is gone and objects come straight off the objspace's bump-pointer heaps. | single Ractor, interleaved minima | RLGC | master | | |---------------------------------------|------|--------|------| | `Object.new` x 20M, GC disabled | 1.83 | 2.01 | **-9%** | | `Object.new` x 20M, GC enabled | 1.18 | 1.10 | +7% | | binary-trees (D=16) | 3.79 | 3.71 | +2% | | JSON parse (8000 docs) | 1.55 | 1.49 | +4% | (`Object.new` rows subtract the loop-only baseline of 0.24s.) Pure allocation wins by 9%; sweep-time bitmap maintenance (the shareable and shref bitmaps) gives ~7% back under allocate-and-discard churn, which nets out to +2-4% on GC-heavy end-to-end workloads and less on anything that does real work between allocations. The multi-objspace machinery is gated behind a single-objspace check on hot paths. ### 4.5 Latency Beyond throughput, the qualitative change is who pauses: on master every GC of any Ractor stops all of them, so one allocation-heavy Ractor imposes its GC pauses on every other Ractor in the process. With per-Ractor GC a Ractor pauses only for its own collections; the remaining process-wide pauses are explicit `GC.start`, shareable-population growth and dead-Ractor cleanup, all of which are rare in steady state. ## 5. Limitations and future work * Incremental marking and non-stop-the-world compaction currently run only while the process has a single objspace. Per-objspace incremental marking is possible in principle (the machinery is per objspace already) but needs the single-to-multi transition handled mid-cycle. * `move:` uses an off-heap serialization; a zero-copy fast path (moving whole pages for large graphs) is future work, as is extending the FREEOBJ tracepoint beyond the main Ractor. * A global GC is still fully stop-the-world; making it concurrent or incremental is orthogonal future work. ## 6. Validation `make btest` (2054 tests) and `make test-all` are green on x86_64-linux across release, `RUBY_DEBUG=1`, ASAN, TSAN and MMTk builds. `GC.verify_internal_consistency` includes the containment verifier (`RGENGC_CHECK_MODE` builds run it around every collection); it was used throughout development. The branch adds ~90 Ractor/GC tests covering lifecycle, copy/move semantics, compaction interaction, and regressions for every bug found during a multi-week stress campaign (a corpus of ~1000 generated Ractor programs run under GC.stress, ASAN and TSAN). ## Notes * This work completes the design presented in [Toward Ractor local GC (RubyKaigi 2025)](https://rubykaigi.org/2025/presentations/ko1.html) * Much of this description, and a large part of the code, was written with Claude Code (Fable/Opus). * Acknowledgements * This work started as a collaboration with Rohit Menon. * Thanks to the Shopify Ractor team for their feedback. * And thank you, everyone, for waiting so long for this work. -- https://bugs.ruby-lang.org/

5 6

[ruby-core:126309] [Ruby Feature#22233] Warn when Net::HTTP#set_debug_output enables debug output
by adam.sligar＠getquip.com (Adam Sligar) 06 Aug '26

06 Aug '26

Issue #22233 has been reported by adam.sligar(a)getquip.com (Adam Sligar). ---------------------------------------- Feature #22233: Warn when Net::HTTP#set_debug_output enables debug output https://bugs.ruby-lang.org/issues/22233 * Author: adam.sligar(a)getquip.com (Adam Sligar) * Status: Open ---------------------------------------- # Abstract `Net::HTTP#set_debug_output` logs every request and response in plain text, including `Authorization` and `Cookie` headers and message bodies. Its documentation says never to use it in production, and nothing enforces that. I propose writing a warning to `$stderr` when debug output is enabled, at most once per thread. # Background The method has carried this documentation for as long as it has existed: # *WARNING* This method opens a serious security hole. # Never use this method in production code. The advice is sound and there is nothing behind it. The method behaves identically in a console and in a production worker, so debug output committed by accident, or enabled by hand during an incident and never reverted, keeps writing credentials to whatever stream it was given. The credentials are then retained and indexed. The only warning the method has today concerns call ordering, not safety: warn 'Net::HTTP#set_debug_output called after HTTP started', uplevel: 1 if started? # Proposal Warn when `set_debug_output` is enabled: $ ruby -rnet/http -e 'Net::HTTP.new("example.com").set_debug_output($stdout)' -e:1: warning: Net::HTTP#set_debug_output: every request and response, including Authorization and Cookie headers and message bodies, will be written to the given stream in plain text; never enable this in production Nothing is written when: * `$DEBUG` is set. Ruby already has a flag meaning "this process is being debugged", and enabling wire logging under it is deliberate. `net/http`'s own test harness reads it the same way, in `test/net/http/utils.rb`. * `output` is `nil`, which disables debug output rather than enabling it. * the warning has already been written on this thread. The once-per-thread flag lives in thread-local storage. `set_debug_output` is a configuration setter that `net/http` never calls itself, so a connection configured once and reused pays nothing per request. But wrappers that build a `Net::HTTP` per request and configure it each time would otherwise repeat an identical message on every request and flood the log the warning is meant to draw attention to. Thread-local storage rather than a class-level instance variable because assigning the latter from a non-main Ractor raises `Ractor::IsolationError`, and `Thread#thread_variable_set` has been available since 2.0, so no version guard is needed. # Use cases * A `set_debug_output` call reaches production in a release. Today the first sign is credentials in the log store. * Debug output is enabled while diagnosing a production incident and not reverted. The next deploy carries it forward silently. * A gem enables debug output on a connection it owns. The application never sees the call and today gets no indication that its requests are being logged. # Discussion **Why not detect production and warn only there?** Because Ruby has no concept of an application environment to consult. No file in `lib/` reads `RAILS_ENV`, `RACK_ENV` or `APP_ENV`; there is no `RUBY_ENV`; `RbConfig` describes the build, not the deployment. Reading a framework's variable would put knowledge of that framework into the standard library and force it to pick a side. Rails, which resolves `RAILS_ENV` then `RACK_ENV`, and Sinatra, which resolves `APP_ENV` then `RACK_ENV`, etc. It would also still miss the case most relevant to a standard library: a script with no framework and no variables set. Warning unconditionally and exempting `$DEBUG` avoids all of it and covers more. **Why not gate on `$VERBOSE`?**`$VERBOSE` is false unless `-w` is passed, which would hide this in exactly the unattended runs it exists to catch. The existing "called after HTTP started" warning in this same method is not gated either. **Why not a `Warning` category?** Of the categories Ruby 4.0 defines (`:deprecated`, `:experimental`, `:performance`, `:strict_unused_block`), only `:experimental` is enabled by default, and none of them describes this. A guardrail nobody has enabled does not guard anything. If ruby-core would prefer a category, an always-on `:security` category has precedent in `:experimental` and would be a reasonable separate proposal; I would rather not block this on introducing one. The warning can already be intercepted by overriding `Warning.warn`. **Why not deprecate the method?** It is intended to exist and is useful. `Warning[:deprecated]` is also off by default, so a deprecation would be invisible in production. Feature #19630 used deprecation correctly because that behavior genuinely was scheduled for removal. -- https://bugs.ruby-lang.org/

1 1