[ruby-core:125003] [Ruby Feature#21953] Allow accessing unshareable objects within a Ractor-local Ruby Box
Issue #21953 has been reported by tikkss (Tsutomu Katsube). ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by tikkss (Tsutomu Katsube). I describe the use case of test-unit. Now, we are implementing `Ractor` based parallel test runner. The tests are run on non-main ractors. We have met a lot of `Ractor::IsolationError`. Each time, We have tried using `freeze` or `make_shareable`, but since writing to class variables or class instance variables within a non-main ractor is not permitted, that code simply won't work. Therefore, by allowing writes to these class variables and class instance variables within a `Ruby::Box` created inside a non-main ractor, the above code will work. This is also effective from the perspective of test isolation. You might argue, "Why not run it in a multi-process?" However, compared to multi-process, ractor has advantages in terms of low overhead and memory usage. Details follow below. A main ractor has the following roles: * Receives a ready signal from non-main ractors then sends a test name to non-main ractors * Receives a test result from non-main ractors then collects it * Receives an event from non-main ractors then emits it Non-main ractors have the following roles: * Collect an entire test suite on each non-main ractors * Send a ready signal to a main ractor * Receive a test name from a main ractor then search a test in an entire test suite and **run it** * Send a test result to a main ractor * Send an event to a main ractor A main ractor and non-main ractors have a 1:N relationship. Communication between a main ractor and non-main ractors is handled the following ways: * A shared `Ractor::Port` object creating in a main ractor * A Default `Ractor::Port` object on each non-main ractors ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117130 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by Eregon (Benoit Daloze). tikkss (Tsutomu Katsube) wrote in #note-1:
You might argue, "Why not run it in a multi-process?" However, compared to multi-process, ractor has advantages in terms of low overhead and memory usage.
Did you measure the difference in memory usage? I'd expect it to be small because each Ruby::Box has pretty much a copy of all classes, i.e. there is very little memory shared between boxes. ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117131 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by tikkss (Tsutomu Katsube). File 1000.html added File 10000.html added File 100000.html added Eregon (Benoit Daloze) wrote in #note-2:
Did you measure the difference in memory usage? I'd expect it to be small because each Ruby::Box has pretty much a copy of all classes, i.e. there is very little memory shared between boxes.
Thanks for your reply. No, I didn't. But I took this opportunity to give it a try. I increased the number of classes with three methods to 1,000, 10,000 and 100,000 and measured the RSS for both `spawn` and `Ractor` + `Ruby::Box` (I used `spawn` for portability). To summarize, `Ractor` + `Ruby::Box` had lower RSS in every case. However, as the number of classes increased, the difference gradually narrowed. The script used for the measurements is as follows: ```rb # spawn.rb monitor_pid = spawn(Gem.ruby, "monitor.rb", Process.pid.to_s) sleep 0.1 # Wait a moment until the monitor process starts up n_workers = 4 N_CLASSES = ARGV[0] pids = n_workers.times.collect do spawn(Gem.ruby, "worker.rb", N_CLASSES) end pids.each do |pid| Process.waitpid(pid) end Process.kill("TERM", monitor_pid) ``` ```rb # monitor.rb pid = Process.pid ppid = ARGV[0] # Sum RSS (KB) for PPID and children (excludes self for accuracy). IO.popen("while ps -o pid=,ppid=,rss= | awk '$1 != #{pid} && ($1 == #{ppid} || $2 == #{ppid}) {sum += $3} END {print sum}'; do :; done") do |io| loop do puts io.gets end end ``` ```rb # worker.rb N_CLASSES = ARGV[0].to_i N_CLASSES.times do |index| Object.const_set("X" + index.to_s, Class.new do def foo end def bar end def baz end end) end sleep 5 # Wait a moment until the other workers finish their processing ``` ```rb # ractor-ruby-box.rb monitor_pid = spawn(Gem.ruby, "monitor.rb", Process.pid.to_s) sleep 0.1 # Wait a moment until the monitor process starts up n_workers = 4 N_CLASSES = ARGV[0] workers = n_workers.times.collect do Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY #{N_CLASSES}.times do |index| Object.const_set("X" + index.to_s, Class.new do def foo end def bar end def baz end end) end sleep 5 # Wait a moment until the other workers finish their processing RUBY end end workers.each(&:join) Process.kill("TERM", monitor_pid) ``` To run the measurements, executes the following commands for each case: ```console $ ruby -v spawn.rb ruby 4.1.0dev (2026-04-30T09:44:32Z master f037b47af9) +PRISM [x86_64-darwin24] (snip) ``` ```console $ RUBY_BOX=1 ruby ractor-ruby-box.rb ruby 4.1.0dev (2026-04-30T09:44:32Z master f037b47af9) +PRISM [x86_64-darwin24] (snip) ``` The results summarizing the maximum RSS are as follows (in CSV format): ```csv TYPE,N_CLASSES,RSS(KB) spawn,1000,72632 spawn,10000,116240 spawn,100000,623692 ractor-ruby-box,1000,21392 ractor-ruby-box,10000,63136 ractor-ruby-box,100000,526572 ``` Please refer to the attached file for a graphical summary of the results. ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117149 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by tagomoris (Satoshi Tagomori). Basically, I agree to this proposal. The described use case is very clear and understandable to me. Technically, we can find several things to be considered. 1) It's not clear how a box is tied with a ractor This proposal is based on an idea that a "Ractor-local" box (for non-main ractors). But currently, Ruby::Box instances (kind of Module) are shareable objects without any information that tell us which Ractor the box was created in. 2) Non-boxed global variables
Currently, non-main ractors prohibit access to the following objects to prevent data races:
Global variables Class variables Unshareable class instance variables Unshareable constants
Class variables, Class instance variables and constants (under a Class) seem to not have any problems to be handled as Ractor-local variables if the Class is created in a user box and the box is a non-main Ractor-local box. We can see the box of the class's classext to know a Class was originally created in a box or not. Global variables are cached/separated by Ruby Box in general. But some of global variables are out of Ruby Box control - for example, `$!` and `$_`. (See https://github.com/ruby/ruby/pull/16303 for example). So the Ractor-local box's global variable management should take care of those global variables. In my instant idea, accessing "Box ready" or "Box dynamic" global variables should be prohibited unless it is marked as Ractor local. But I may miss something in corner cases. ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117156 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by Eregon (Benoit Daloze). @tikkss I think there is an issue with your accounting of memory because it doesn't match the RSS of worker process. This is running your example as-is. ``` $ ruby -v ruby 4.0.2 (2026-03-17 revision d3da9fec82) +PRISM [x86_64-linux] $ ruby spawn.rb 1000 15756 15756 15756 15756 70556 76628 81552 81552 81552 81552 ... ``` But that process I think never used that much memory: ``` $ /usr/bin/time -v ruby worker.rb 1000 ... Maximum resident set size (kbytes): 16248 ``` ``` $ ruby worker_with_rss.rb 1000 16492 16552 16552 16552 ``` with ```ruby # worker_with_rss.rb N_CLASSES = ARGV[0].to_i Thread.new { loop { puts `ps -o rss #{Process.pid}`.lines.last.to_i sleep 0.5 } } N_CLASSES.times do |index| Object.const_set("X" + index.to_s, Class.new do def foo end def bar end def baz end end) end sleep 2 ``` ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117158 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by byroot (Jean Boussier). I understand the reasoning, however given the implementation of boxes, I fear implementing this proposal would reintroduce the need for many ractor locks, negating their usefulness. Also it's not just `Ruby::Box` that would need to become unsheareable, but all objects and classes allocated from a non-main ractor box, making communication with other ractors close to impossible. ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117163 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by tikkss (Tsutomu Katsube). @Eregon Thank you for even creating a verification script. If you are comparing the RSS of only a single process, I think your example is fine. However, I wanted to compare the total RSS including all worker processes. This is because I wanted to compare multi-process and multi-ractor approaches. When running `spawn.rb`, because the number of workers is set to 4, the following 6 processes are started. My script compares the sum of the RSS values of 1, 3, 4, 5 and 6. 1. spawn.rb 2. monitor.rb 3. worker.rb 4. worker.rb 5. worker.rb 6. worker.rb When running `ractor-ruby-box.rb`, because non-main ractors are executed within the process, the following 2 processes are started. My script compares only the RSS of 1. 1. ractor-ruby-box.rb 2. monitor.rb ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117240 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
Issue #21953 has been updated by tikkss (Tsutomu Katsube). byroot (Jean Boussier) wrote in #note-6:
I understand the reasoning, however given the implementation of boxes, I fear implementing this proposal would reintroduce the need for many ractor locks, negating their usefulness.
Also it's not just `Ruby::Box` that would need to become unsheareable, but all objects and classes allocated from a non-main ractor box, making communication with other ractors close to impossible.
Thanks for your opinions. I think your concerns as follows: 1. Increase ractor locks 2. Loss of shareability ### 1. Increase ractor locks As of now, I don't know whether ractor locks will increase. I try a prototype implementation to see if it actually becomes an issue. ### 2. Loss of shareability Actually, "not being shareable" is exactly what I'm aiming for. My idea is to keep the existing shareable box functionality as is, and allow non-shareable boxes (e.g.: `Ruby::Box.new(shareable: false)`) to coexicit. This would give users a choice, which I believe would be even more convenient. For example, if users have a non-shareable box, they could make their code "Ractor ready" without modifying the existing logic. They would only need to focus on the Ractor communication part, allowing them to utilize cpu resources more effectively. I think this approach would be much more practical, especially when modifying existing code to be fully shareable is difficult. ---------------------------------------- Feature #21953: Allow accessing unshareable objects within a Ractor-local Ruby Box https://bugs.ruby-lang.org/issues/21953#change-117241 * Author: tikkss (Tsutomu Katsube) * Status: Open ---------------------------------------- ### Status Currently, non-main ractors prohibit access to the following objects to prevent data races: * Global variables * Class variables * Unshareable class instance variables * Unshareable constants ### Proposal I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor: ```ruby # lib/x.rb class X # can write unshareable objects XXX = "1" @@cvar = "1" @ivar = "1" class << self def cvar; @@cvar; end def ivar; @ivar; end end end # can read unshareable objects $LOAD_PATH # => returns $LOAD_PATH includes lib directory X.cvar # => "1" X.ivar # => "1" X::XXX # => "1" # main.rb Ractor.new do local_box = Ruby::Box.new local_box.eval <<~RUBY base_dir = File.expand_path(File.dirname(__FILE__)) lib_dir = File.join(base_dir, "lib") # can write unshareable objects $LOAD_PATH.unshift(lib_dir) RUBY local_box.require("x") x = local_box::X.new end.join ``` Ruby Box can isolate global/class variables, class/module definitions from other boxes. A Ruby Box created inside a non-main ractor cannot be accessed from other ractor. If that is the case, wouldn’t it be fine to access unshareable objects inside that box? So I would like to propose that allow reading/writing unshareable objects inside a Ruby Box created in a non-main ractor. ### Background We are working on implementing a Ractor based parallel test runner for the test-unit gem. Ractor is a great for parallel processing. However, many existing libraries still rely on class variables or class instance variables for configuration. Ideally, we should reduce the use of class variables or class instance variables. Currently, we tried fixing several non-shareable objects, but we could not resolve all of them yet. We will continue working on this issue, but we are also exploring other approaches. This is the idea begind this proposal. I think the work needed to make objects shareable when running exisiting libraries with Ractor can be reduced. ### FAQ Q: Can we create a Ruby Box inside a non-main ractor? A: Yes: ```ruby Ractor.new {Ruby::Box.new}.join ``` Q: Is a Ruby Box created in a non-main ractor truly inaccessible from other ractor? A: No. I'm not sure if it's intentional, but a Ruby Box is a shareable object. Also, it can be accessed from the main Ractor by using `Ractor#value`: ```ruby Ractor.shareable?(Ruby::Box.new) # => true ``` ```ruby Ractor.new {Ruby::Box.new}.value # => #<Ruby::Box:3,user,optional> ``` To implement this proposal, Ruby Box may need to be an unshareable object, and passing it with `Ractor#value` may need to be disallowed. ---Files-------------------------------- 1000.html (1.01 KB) 10000.html (1.02 KB) 100000.html (1.02 KB) -- https://bugs.ruby-lang.org/
participants (4)
-
byroot (Jean Boussier) -
Eregon (Benoit Daloze) -
tagomoris (Satoshi Tagomori) -
tikkss (Tsutomu Katsube)