August 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:123146] [Ruby Bug#21559] Unicode normalization nfd -> nfc -> nfd is not reversible
by tompng (tomoya ishida) 02 Nov '25

02 Nov '25

Issue #21559 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #21559: Unicode normalization nfd -> nfc -> nfd is not reversible https://bugs.ruby-lang.org/issues/21559 * Author: tompng (tomoya ishida) * Status: Open * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- I expect `nfd(nfc(str)) == nfd(str)` but found a string that doesn't. ~~~ruby # Ruby 3.1 - 3.5 str = "s\u{11930}\u{323}\u{11930}\u{307}" p str.unicode_normalize(:nfd) == str.unicode_normalize(:nfc).unicode_normalize(:nfd) #=> false ~~~ ~~~ruby # ruby 3.5.0dev str = "s\u{1611e}\u{323}\u{1611e}\u{307}\u{1611f}" p str.unicode_normalize(:nfd) == str.unicode_normalize(:nfc).unicode_normalize(:nfd) #=> false ~~~ -- https://bugs.ruby-lang.org/

4 7

[ruby-core:122883] [Ruby Bug#21522] Accessing outer locals via eval in a Ractor returns false
by tenderlovemaking (Aaron Patterson) 24 Oct '25

24 Oct '25

Issue #21522 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #21522: Accessing outer locals via eval in a Ractor returns false https://bugs.ruby-lang.org/issues/21522 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Assignee: ractor * ruby -v: ruby 3.5.0dev (2025-07-28T22:17:45Z master f1acf47ca2) +PRISM [arm64-darwin24] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Normally, if you try to access a local that's been defined outside the scope of the Ractor block, it'll raise an ArgumentError. However, if you try to access the local via `eval` it will return the value `false`. ```ruby foo = Object.new begin Ractor.new { p foo }.value rescue ArgumentError end Ractor.new { p eval("foo") # => false }.value ``` I'm not sure how we fix this, since my guess is that we zero the environment when copying, but I wanted to file the issue so we can track it. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118243] [Ruby master Feature#20564] Switch default parser to Prism
by kddnewton (Kevin Newton) 22 Oct '25

22 Oct '25

Issue #20564 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #20564: Switch default parser to Prism https://bugs.ruby-lang.org/issues/20564 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- This issue is to propose switching the default parser for Ruby to Prism on the master branch. The main goal is to get this default into both master and the next preview so that the community has plenty of time to test before the eventual 3.4 release. Matz has indicated that the final decision about the default parser for Ruby 3.4 will be made later in the year. Decisions about the default parser for Ruby 3.5+ and the relationship between Prism, lrama, parse.y, etc. will also be made at a later time. ## Background For some background on the Prism project itself, here are a couple of links in chronological order that you can read: * https://bugs.ruby-lang.org/issues/19741 * https://railsatscale.com/2023-06-12-rewriting-the-ruby-parser/ * https://kddnewton.com/2024/01/23/prism.html * https://railsatscale.com/2024-04-16-prism-in-2024/ ## Status Prism now passes all Ruby tests and specs. Internally at Shopify, it also passes our Core monolith's CI, which is a large and complex Rails application. We are confident that it is ready for production use. External to CRuby, Prism has already been adopted extensively around the ecosystem, including but not limited to: * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java) * [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java) * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby) * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM) * [repl_type_completor](https://github.com/ruby/repl_type_completor) * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-pa… (via parser translator) * [ruby-lsp](https://github.com/Shopify/ruby-lsp) * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator) Fortunately with the community adoption, we have been able to find many edge cases and bugs that have been fixed. Thank you so much to everyone who has tried, used, reported problems with, and improved Prism in the last couple of years. We would not have hit this milestone without this community involvement. ## Caveats While Prism correctly accepts and parses all valid Ruby syntax, there are still a number of cases where it does not reject invalid syntax as it should. We are currently working on these cases, and we are confident that we can eliminate them before the next preview release. You can find them listed here: https://github.com/ruby/prism/issues?q=is%3Aopen+is%3Aissue+label%3Ainvalid…. This is our top priority right now, and I anticipate these being resolved quickly. The other caveat is unknown unknowns: bugs or syntax issues that we have yet to discover. At Shopify we are currently taking numerous steps to mitigate this risk, including: * Prism currently passes all Ruby tests and specs, and will continue to going forward. The tests and specs are extensive, and we are confident this gets us almost all of the way there. * Grammar-based fuzzing to ensure we handle all possible syntax variations (both valid and invalid). As an example, that recently led to the discovery that the current compiler was segfaulting on safe navigation in for loops (https://bugs.ruby-lang.org/issues/20468) It has also found several memory leaks that have been fixed. * Mutation testing of existing source files (by adding and deleting random bytes) to find additional edge cases. Our goal is 100% compatibility, and we believe this is achievable through the above techniques and others. The goal of this proposal is to allow more companies to test their codebase against Prism, and to make this easier. We believe that we can sort out the remaining problems much more quickly in that manner. ## Implementation The implementation will be relatively straightforward, as there is already the `--parser` flag that can be used to switch between the current parser and Prism. We will change the default to be `"prism"` instead of `"parse.y"`. There is no plan to remove or change any code related to `parse.y`, `lrama`, or `compile.c` at this time, as we want to create the minimal amount of change possible to ensure stability. -- https://bugs.ruby-lang.org/

4 10

[ruby-core:111742] [Ruby master Feature#19326] Please add a better API for passing a Proc to a Ractor
by sdwolfz 18 Oct '25

18 Oct '25

Issue #19326 has been reported by sdwolfz (Codruț Gușoi). ---------------------------------------- Feature #19326: Please add a better API for passing a Proc to a Ractor https://bugs.ruby-lang.org/issues/19326 * Author: sdwolfz (Codruț Gușoi) * Status: Open * Priority: Normal ---------------------------------------- Example 1: ```ruby class Worker def initialize(&block) @block = block end def run Ractor.new(@block, &:call) end end worker = Worker.new { 1 } puts worker.run.take ``` Errors with: ``` <internal:ractor>:271:in `new': allocator undefined for Proc (TypeError) from scripts/run.rb:9:in `run' from scripts/run.rb:14:in `<main>' ``` Example 2: ```ruby class Worker def initialize(&block) @block = Ractor.make_shareable(block) end def run Ractor.new(@block, &:call) end end worker = Worker.new { 1 } puts worker.run.take ``` Errors with: ``` <internal:ractor>:820:in `make_shareable': Proc's self is not shareable: #<Proc:0x00007f00394c38b8 scripts/run.rb:13> (Ractor::IsolationError) from scripts/run.rb:5:in `initialize' from scripts/run.rb:13:in `new' from scripts/run.rb:13:in `<main>' ``` Example 3: ```ruby class Worker def initialize(&block) @block = Ractor.make_shareable(block) end def run Ractor.new(@block, &:call) end end worker = Ractor.current.instance_eval { Worker.new { 1 } } puts worker.run.take ``` Works, but having `Ractor.current.instance_eval` as a wrapper around the block is not ideal, as Ractor is supposed to be only an implementation detail in Worker. I know about https://bugs.ruby-lang.org/issues/18243 and the discussion around `proc.bind(nil)`. That would actually be ideal, as for the purposes if why I want this functionality I don't care what `self` is in a block, and the less it has access to the better. The general idea is to have a Ractor be able to lazily execute an arbitrary proc. And all the bindings it would need would be passed explicitly, either through `args` in the constructor or through `send`/`receive`, so `self` would really not matter. The benefit: this would make it so concurrent code can be more easily be implemented with Ractors as currently you can execute an arbitrary proc by passing it to a Thread (but you don't get the nice data isolation). -- https://bugs.ruby-lang.org/

6 12

[ruby-core:122164] [Ruby Feature#21347] Add `open_timeout` as an overall timeout option for `Socket.tcp`
by shioimm (Misaki Shioi) 07 Oct '25

07 Oct '25

Issue #21347 has been reported by shioimm (Misaki Shioi). ---------------------------------------- Feature #21347: Add `open_timeout` as an overall timeout option for `Socket.tcp` https://bugs.ruby-lang.org/issues/21347 * Author: shioimm (Misaki Shioi) * Status: Open ---------------------------------------- I propose to add an overall timeout option to `Socket.tcp` (and `TCPSocket.new`) ### Background Currently, `TCPSocket.new` and `Socket.tcp` accept two kind of timeout options: - `resolv_timeout`, which controls the timeout for DNS resolution - `connect_timeout`, which controls the timeout for the connection attempt With the introduction of Happy Eyeballs Version 2 (as per [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305)) in[ Feature #20108](https://bugs.ruby-lang.org/issues/20108) and [Feature #20782](https://bugs.ruby-lang.org/issues/20782), both address resolution and connection attempts are now parallelized. As a result, the sum of `resolv_timeout` and `connect_timeout` no longer represents the total timeout duration. This is because, in HEv2, name resolution and connection attempts are performed concurrently, causing the two timeouts to overlap. Example: When `resolv_timeout: 200ms` and `connect_timeout: 100ms` are set: 1. An IPv6 address is resolved after the method starts immediately (IPv4 is still being resolved). 2. A connection attempt is initiated to the IPv6 address 3. After 100ms, `connect_timeout` is exceeded. However, since `resolv_timeout` still has 100ms left, the IPv4 resolution continues. 4. After 200ms from the start, the method raises a `resolv_timeout` error. In this case, the total elapsed time before a timeout is 200ms, not the expected 300ms (100ms + 200ms). Furthermore, in HEv2, connection attempts are also parallelized. It starts a new connection attempts every 250ms for resolved addresses. This makes the definition of `connect_timeout` even more ambiguous—specifically, it becomes unclear from which point the timeout is counted. Additionally, these methods initiate new connection attempts every 250ms (Connection Attempt Delay) for each candidate address, thereby parallelizing connection attempts. However, this behavior makes it unclear from which point in time the connect_timeout is actually measured. Currently, a `connect_timeout` is raised only after the last connection attempt exceeds the timeout. Example: When `connect_timeout: 100ms` is set and 3 address candidates: 1. Connect to address `a` at t=0ms. 2. Connect to address `b` at t=250ms. 3. Connect to address `c` at t=500ms. 4. If all fail, the exception is raised at t=1500ms (1000ms after `c`, 1250ms after `b`, 1500ms after `a`). 1. Start a connection attempt to the address `a` 2. 250ms after step 1, start a new connection attempt to the address `b` 3. 500ms after step 1, start a new connection attempt to the address `c` 4. 1000ms after step 3 (1000ms after starting the connection to `c`, 1250ms after starting the connection to `b,` and 1500ms after starting the connection to `a`) `connect_timeout` is raised This behavior aims to favor successful connections by allowing more time for each attempt, but it results in a timeout model that is difficult to reason about. These methods have supported `resolv_timeout` and `connect_timeout` options even before the introduction of HEv2. However, in many use cases, it would be more convenient if a timeout occurred after a specified duration from the start of the method. Similar functions in other languages (such as PHP, Python, and Go) typically allow specifying only an overall timeout. ### Proposal I propose adding an `open_timeout` option to `Socket.tcp` in the following PR, which triggers a timeout after a specified duration has elapsed from the start of the method. https://github.com/ruby/ruby/pull/13368 The name `open_timeout` aligns with the existing accessor used in `Net::HTTP`. If `open_timeout` is specified together with `resolv_timeout` and `connect_timeout`, I propose that only `open_timeout` be used and the others be ignored. While it is possible to support combinations of `open_timeout`, `resolv_timeout`, and `connect_timeout`, doing so would require defining which timeout takes precedence in which situations. In this case, I believe it is more valuable to keep the behavior simple and easy to understand, rather than supporting more complex use cases. If this proposal is accepted, I also plan to extend `open_timeout` support to `TCPSocket.new`. While the long-term future of `resolv_timeout` and `connect_timeout` may warrant further discussion, I believe the immediate priority is to offer a straightforward way to specify an overall timeout. ### Outcome If `open_timeout` is also supported by `TCPSocket.new`, users would be able to manage total connection timeouts directly in `Net::HTTP#connect` without relying on `Timeout.timeout`. https://github.com/ruby/ruby/blob/aa0f689bf45352c4a592e7f1a044912c40435266/… -- https://bugs.ruby-lang.org/

3 8

[ruby-core:112552] [Ruby master Bug#19461] Time.local performance tanks in forked process (on macOS only?)
by ioquatix (Samuel Williams) 06 Oct '25

06 Oct '25

Issue #19461 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19461: Time.local performance tanks in forked process (on macOS only?) https://bugs.ruby-lang.org/issues/19461 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following program demonstrates a performance regression in forked child processes when invoking `Time.local`: ```ruby require 'benchmark' require 'time' def sir_local_alot result = Benchmark.measure do 10_000.times do tm = ::Time.local(2023) end end $stderr.puts result end sir_local_alot pid = fork do sir_local_alot end Process.wait(pid) ``` On Linux the performance is similar, but on macOS, the performance is over 100x worse on my M1 laptop. -- https://bugs.ruby-lang.org/

5 10

[ruby-core:116946] [Ruby master Bug#20302] Multiple refinements cannot be applied to the same module
by mame (Yusuke Endoh) 06 Oct '25

06 Oct '25

Issue #20302 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20302: Multiple refinements cannot be applied to the same module https://bugs.ruby-lang.org/issues/20302 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: shugo (Shugo Maeda) * ruby -v: ruby 3.4.0dev (2024-02-20T07:17:20Z master a605234bfa) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- If we are using multiple refinements for the Kernel module, it seems that only the last refinement is enabled. In the following example, we can call `f2` of M2, which is enabled later. but cannot call `f1` of M1. ```ruby module M1 refine(Kernel) do def f1 = :f1 end end module M2 refine(Kernel) do def f2 = :f2 end end class Foo using M1 using M2 def test p f2 #=> :f2 p f1 # expected => :f1 # actual => undefined local variable or method 'f1' for an instance of Foo end end Foo.new.test ``` Note that if I change `refine(Kernel)` to `refine(Object)`, it works as expected. So I think there is a problem only when the refinement target is a module. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:122121] [Ruby Bug#21342] Segfault: invalid keeping_mutexes when using Mutex in Thread then Fiber after GC
by maciej.mensfeld (Maciej Mensfeld) 05 Oct '25

05 Oct '25

Issue #21342 has been reported by maciej.mensfeld (Maciej Mensfeld). ---------------------------------------- Bug #21342: Segfault: invalid keeping_mutexes when using Mutex in Thread then Fiber after GC https://bugs.ruby-lang.org/issues/21342 * Author: maciej.mensfeld (Maciej Mensfeld) * Status: Open * ruby -v: 3.4.3 (2025-04-14 revision d0b7e5b6a0) +PRISM [x86_64-linux] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Ruby crashes with a `[BUG] invalid keeping_mutexes error` when attempting to GC locked mutex that was used in a Thread within a Fiber context after garbage collection. The error indicates an attempt to unlock a mutex that is not locked, suggesting a state management issue with mutexes across Thread and Fiber boundaries. ## Ruby Version `ruby 3.4.3 (2025-04-14 revision d0b7e5b6a0) +PRISM [x86_64-linux]` ## Reproduce Process ``` ruby # segv.rb 5.times do m = Mutex.new Thread.new do m.synchronize do end end.join Fiber.new do GC.start m.lock end.resume end ``` 1. Save the above code to a file (e.g., `segv.rb`) 2. Run with `ruby segv.rb` 3. The crash occurs intermittently - sometimes it crashes immediately, sometimes it hangs, once in a while it works ## Actual Result The program crashes with the following error: ``` segv.rb: [BUG] invalid keeping_mutexes: Attempt to unlock a mutex which is not locked ruby 3.4.3 (2025-04-14 revision d0b7e5b6a0) +PRISM [x86_64-linux] ``` whole segfault in the attached txt file. Full crash backtrace shows the error originates from: - `rb_threadptr_unlock_all_locking_mutexes` in thread.c:450 - `rb_thread_terminate_all` in thread.c:467 The crash suggests an issue in mutex state management during thread termination. ## Expected Result The script should complete successfully without crashing. The mutex should be properly managed across Thread and Fiber contexts, and garbage collection should not interfere with mutex state. ---Files-------------------------------- crash.txt (23.4 KB) -- https://bugs.ruby-lang.org/

7 10

[ruby-core:115552] [Ruby master Misc#20032] Propose @kjtsanaktsidis as a commiter
by jeremyevans0 (Jeremy Evans) 26 Sep '25

26 Sep '25

Issue #20032 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Misc #20032: Propose @kjtsanaktsidis as a commiter https://bugs.ruby-lang.org/issues/20032 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal ---------------------------------------- I'd like to propose @kjtsanaktsidis as a committer, with a focus on bug triaging and fixing. During RubyKaigi 2023, KJ expressed interest in helping triage and fix bugs, and since then, he has fixed numerous bugs. What's most impressive about KJ's contributions is the difficulty of the bugs he has fixed, and the wide range of areas in which he has fixed bugs (GC, threading, FreeBSD support, ractors, rubygems, extensions, tests, old bugs, etc.). Here is a list of bugs KJ has fixed: * #20021 * #19994 * #19928 * #19873 * #19387 * #19680 * #4173 Here is a list of open pull requests submitted by KJ to fix bugs: * #20022 https://github.com/ruby/ruby/pull/9041 * #19991 https://github.com/ruby/ruby/pull/8949 and https://github.com/ruby/ruby/pull/8856 * #20001 https://github.com/ruby/ruby/pull/8903 * #19717 https://github.com/ruby/ruby/pull/8331 * #19144 https://github.com/ruby/ruby/pull/7295 * #18795 https://github.com/ruby/ruby/pull/5992 In addition to these Redmine issues, KJ has also fixed other bugs: * https://github.com/ruby/ruby/commit/9a62fd3cbae2ebb60e2f9cad782af1ad18db4319 * https://github.com/ruby/ruby/commit/76dc327eeffefe02577999fe5f8215f762a581b6 * https://github.com/ruby/ruby/commit/082962e857833dfc16881cb4dfb44caa175590ba * https://github.com/ruby/ruby/commit/edee9b6a12ac846d7b3de2d704e170bf28178cb3 * https://github.com/ruby/ruby/commit/dd406c5a911d2f4a16ce97016666760713540185 * https://github.com/ruby/ruby/commit/7bd7aee02e303de27d2cddfc5ef47e612d6782cb * https://github.com/ruby/ruby/commit/05ffc037ad15560cb2c683682c5bc374156571d7 * https://github.com/ruby/ruby/pull/7867 * https://github.com/ruby/ruby/pull/7865 * https://github.com/ruby/zlib/commit/3b9fe962d8 * https://github.com/ruby/zlib/commit/c96e8b9a57 In addition to bug fixing, KJ has also proposed and implemented a useful Socket feature: * #19179 https://github.com/ruby/ruby/pull/6822 -- https://bugs.ruby-lang.org/

8 14

[ruby-core:123136] [Ruby Feature#21557] Ractor.shareable_proc to make sharable Proc objects, safely and flexibly
by Eregon (Benoit Daloze) 23 Sep '25

23 Sep '25

Issue #21557 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #21557: Ractor.shareable_proc to make sharable Proc objects, safely and flexibly https://bugs.ruby-lang.org/issues/21557 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- Following #21039 and #21550, this is a complete proposal which does not require reading these previous proposals (since that caused some confusion). That way, it is hopefully as clear as possible. It also explains how it solves everything we discussed in the previous tickets. To use Ractor effectively, one needs to create Procs which are shareable between Ractors. Of course, such Procs must not refer to any unshareable object (otherwise the Ractor invariant is broken and segfaults follow). One key feature of blocks/Procs is to be able to capture outer variables, e.g.: ```ruby data = ... task = -> { do_work(data) } ``` Ractor shareable procs should be able to use captured variables, because this is one of the most elegant ways to pass data/input in Ruby. But there is a fundamental conflict there, reassigning captured variables cannot be honored by shareable procs, otherwise it breaks the Ractor invariant. So creating a shareable proc internally makes a shallow copy of the environment, to not break the Ractor invariant. We cannot prevent assigning local variables (i.e. raise an exception on `foo = value`), that would be way to weird. But we can raise an error when trying to create a shareable proc in an incompatible situation, that makes it safe by preventing the unsafe cases. ## Reassigning a captured variable inside the block Concretely, it seems we all already agree that this should be a `Ractor::IsolationError`: ```ruby def example a = 1 b = proc { v = a; a += 1; v } r = Ractor.shareable_proc(&b) # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned inside the block [b, r] end example.map(&:call) ``` And that's because without the error the result would be `[1, 1]` which is unexpected (it should be `[1, 2]`), `r.call` should have updated `a` to 2 but it only updated `a` in its environment copy. That basically breaks the lexical scoping of variables captured by blocks. We can check this by static analysis, in fact we already use static analysis for `Ractor.new`: `a = 0; Ractor.new { a = 2 }` which gives `can not isolate a Proc because it accesses outer variables (a). (ArgumentError)`. ## Reassigning a captured variable outside the block The second problematic case is: ```ruby # error: the code clearly assumes it can reassigns `a` but the `shareable_proc` would not respect it, i.e. `shareable_proc` would break Ruby block semantics # Also note the Ractor.shareable_proc call might be far away from the block, so one can't tell when looking at the block that it would be broken by `shareable_proc` (if no error for this case) def example a = 1 b = proc { a } Ractor.shareable_proc(&b) # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned outside the block a = 2 end ``` This is very similar (it is the symmetric case), the `shareable_proc` cannot honor the `a = 2` assignment, so it should not allow creating a `shareable_proc` in that context and should be `Ractor::IsolationError`. If you don't see the issue in that small example, let's use this example: ```ruby page_views = 0 background_jobs.schedule_every(5.seconds) { puts "#{page_views} page views so far" } threaded_webserver.on("/") do page_views += 1 "Hello" end ``` If `background_jobs` uses `Thread`, everything is fine. If it uses `Ractor`, it needs to make that `schedule_every` block shareable, and if we don't add this safety check then it will always incorrectly print `0 page views so far`. This is what I mean by breaking Ruby block semantics. In this proposal, we prevent this broken semantics situation by `Ractor::IsolationError` when trying to make that `schedule_every` block shareable. One more reason here to forbid this case is that a block that is made shareable is never executed immediately on the current Ractor, because there is no need to make it shareable for that case. And so it means the block will be executed later, by some other Ractor. And that block, if it expects to be executed later, then definitely expects to see up-to-date captured variables (as in the author of the block expects that). We would check this situation by static analysis. There are multiple ways to go about it with trade-offs between precision and implementation complexity. I think we could simplify to: disallow `Ractor.shareable_proc` for any block which captures a variable which is potentially reassigned. In other words, only allow `Ractor.shareable_proc` if all the variables it captures are assigned (exactly) once. More on that later in section `Edge Cases`. ## Ractor.new Note that everything about `Ractor.shareable_proc` should also apply to `Ractor.new`, that way it's convenient to pass data via captured variables for `Ractor.new` too, example: ```ruby x = ... y = ... Ractor.new { compute(x, y) } ``` Currently `Ractor.new` does not allow capturing outer variables at all and needs workarounds such as: ```ruby x = ... y = ... Ractor.new(x, y) { |x, y| compute(x, y) } ``` ## define_method `define_method` (and of course `define_singleton_method` too) have been an issue since the beginning of Ractors, because methods defined by `define_method` just couldn't be called from a Ractor (because the block/Proc wouldn't be shareable and so can't be called from other Ractors). A workaround is to make the block/Proc shareable, but this is inconvenient, verbose and shouldn't be necessary: ```ruby def new_ostruct_member!(name) # :nodoc: unless @table.key?(name) || is_method_protected!(name) if defined?(::Ractor) getter_proc = nil.instance_eval{ Proc.new { @table[name] } } setter_proc = nil.instance_eval{ Proc.new {|x| @table[name] = x} } ::Ractor.make_shareable(getter_proc) ::Ractor.make_shareable(setter_proc) else getter_proc = Proc.new { @table[name] } setter_proc = Proc.new {|x| @table[name] = x} end define_singleton_method!(name, &getter_proc) define_singleton_method!("#{name}=", &setter_proc) end end ``` Instead, this proposal brings the idea for `define_method` to automatically call `Ractor.shareable_proc` on the given block/Proc (and fallback to the original Proc if it would raise), as if it was defined like: ```ruby def define_method(name, &body) body = Ractor.shareable_proc(self: nil, body) rescue body Primitive.define_method(name, &body) end ``` (note that `define_method` knows the `body` Proc's `self` won't be the original `self` anyway, so it's fine to change it to `nil`) This way workarounds like above are no longer needed and the code can be as simple as it used to be: ```ruby def new_ostruct_member!(name) # :nodoc: unless @table.key?(name) || is_method_protected!(name) define_singleton_method!(name) { @table[name] } define_singleton_method!("#{name}=") { |x| @table[name] = x } end end ``` Much nicer, and solves a longstanding issue with Ractor. There should be no compatibility issue since the block is only made shareable when it's safe to do so. This is another argument for making `Ractor.shareable_proc` safe. ## Ractor.shareable_proc and Ractor.shareable_lambda I believe we don't need `Ractor.shareable_lambda` (mentioned in other tickets). `Ractor.shareable_proc` should always preserve the lambda-ness (`Proc#lambda?`) of the given Proc. The role of `Ractor.shareable_proc` is to make the Proc shareable, not change arguments handling. If one wants a shareable lambda they can just use `Ractor.shareable_proc(&-> { ... })`. BTW, the added value of `Ractor.shareable_proc(self: nil, &proc)` vs just `Ractor.make_shareable(proc, copy: true)` is that it enables changing the receiver of the Proc without needing `nil.instance_eval { ... }` around, and it is much clearer. `Ractor.make_shareable(proc)` should be an error [as mentioned here](https://bugs.ruby-lang.org/issues/21039#note-14), because it would mutate the proc inplace and that's too surprising and unsafe (e.g. it would break `Proc#binding` on that Proc instance). `Ractor.make_shareable(proc, copy: true)` can be the same as `Ractor.shareable_proc(self: self, &proc)` (only works if `self` is shareable then), or an error. ## Edge Cases For these examples I'll use `enqueue`, which defines a block to execute later, either in a Thread or Ractor. For the Ractor case, `enqueue` would make the block shareable and send it to a Ractor to execute it. This is a bit more realistic than using plain `Ractor.shareable_proc` instead of `enqueue`, since it makes it clearer the block won't be executed right away on the main Ractor but later on some other Ractor. ### Nested Block Cases If the assignment is in a nested block, it's an error (this case is already detected for `Ractor.new` BTW): ```ruby a = 1 enqueue { proc { a = 1 } } # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned inside the block ``` Similarly, if the assignment is in an some block outside, it's the same as if it was assigned directly outside: ```ruby a = 1 p = proc { a = 2 } enqueue { a } # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned outside the block ``` ### Loop Cases This would be a `Ractor::IsolationError`, because `a` is reassigned. It would read a stale value and silently ignore reassignments if there was no `Ractor::IsolationError`. ```ruby a = 0 while condition enqueue { p a } # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned outside the block a += 1 end ``` This is the same case, using a rescue-retry loop: ```ruby begin enqueue { p a } # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned outside a += 1 raise rescue retry if condition end ``` A `for` loop is like a while `loop` because the LHS variable (`a`) and all variables in the loop body are actually declared outside (weird, but that's how it is). ```ruby for a in enum b = rand enqueue { p a } # Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned outside the block end binding.local_variables # => [:a, :b] ``` Any assignment inside one of these loops can potentially happen multiple times, so any variable assigned inside one of these loops cannot be captured by a shareable block (i.e., `Ractor::IsolationError` when trying to make a shareable block in such a case). We will need the static analysis to detect such loops. That probably doesn't need a full Control Flow Graph, we just need to determine if an assignment is "inside a while/for/retry" loop (up to a scope barrier like `def`/`class`/`module`). Regular "loops" using blocks are fine though, because they create a new environment/frame for each iteration. These 2 blocks will always see `[0, 1]` and `[0, 2]`, whether shareable or not: ```ruby a = 0 [1, 2].each do |e| enqueue { p [a, e] } # OK, each of these variables is assigned only once end ``` ### eval and binding Static analysis cannot detect `eval` or `binding`. In such an extreme and very rare case the fact that `shareable_proc` makes a copy of the environment is visible: ```ruby a = 1 b = proc { a } s = Ractor.shareable_proc(&b) eval("a = 2") # or binding.local_variable_set(:a, 2), or b.binding.local_variable_set(:a, 2) b.call # => 2 s.call # => 1 ``` This seems unavoidable, unless we prevent shareable procs to use captured variables at all (quite restrictive). BTW, `Proc#binding` is already not supported for a `shareable_proc`: ``` $ ruby -e 'nil.instance_exec { a = 1; b = proc { a }; b2 = Ractor.make_shareable(b); p b2.binding }' -e:1:in `binding': Can't create Binding from isolated Proc (ArgumentError) ``` So `binding`/`eval` is in general already not fully respected with Ractor anyway (and cannot be). ### Multiple Assignments Before This simple example assigns `a` twice. It would be safe because `a` is always assigned before (in execution, not necessarily in source order) creating the block instance/Proc instance, but it is not so easy to detect. Depending on how precise the static analysis is it might allow this case. We can always allow more and start with something simple. ```ruby a = 1 a = 2 Ractor.shareable_proc { a } # Ractor::IsolationError if using the single-assignment-only static analysis, seems OK because not so common ``` ### Error Message Differentiating `... which are reassigned inside/outside the block` might be needlessly complicated, in such a case I think it's fine to simplify the error message and omit the part after `... which are reassigned`. The important part is that the outer variable is reassigned, not whether it's inside or outside. ## Alternatives ### Relaxing the checks for literal blocks `Kernel#lambda` for example has behavior which depends on whether it's given a literal block or not: ```ruby lambda(&proc {}) # => the lambda method requires a literal block (ArgumentError) ``` We could have such a difference, but I don't think it's very useful, if a variable is reassigned, it seems a bad idea to capture and shallow-copy with a shareable proc (unclear, unsafe, etc). The semantics are also simpler if they are the same whether the block is literal or not. ### Removing the checks for reassigning a captured variable outside the block That's basically option 1 of #21550. This would allow known unsafe behavior and break the examples shown above (i.e. it would break code in a nasty way: some assignments are silently ignored, good luck to debug that). It would be hard to then forbid such cases later as it could then be considered incompatible. In my opinion we would commit a big language design mistake if we just give up and allow known unsafe cases like that, people wouldn't be able to trust that local variable assignments are respected (pretty fundamental, isn't it?) and that Ruby blocks behave as they always have been (with lexical scoping for local variables). It would also do nothing to help with `define_method`. `Ractor.shareable_proc(Proc)` is currently unsafe (the main point of #21039), let's address it, not ignore known problems, especially after a lot of discussion and thoughts on how to solve it properly. -- https://bugs.ruby-lang.org/

2 3