August 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114543] [Ruby master Bug#17678] Ractors do not restart after fork
by jeremyevans0 (Jeremy Evans) 25 Aug '23

25 Aug '23

Issue #17678 has been updated by jeremyevans0 (Jeremy Evans). I updated https://github.com/ruby/ruby/pull/8283 to mark non-main ractors as terminated after fork. ---------------------------------------- Bug #17678: Ractors do not restart after fork https://bugs.ruby-lang.org/issues/17678#change-104345 * Author: ivoanjo (Ivo Anjo) * Status: Assigned * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello there! I'm working at Datadog on the `ddtrace` gem -- <https://github.com/DataDog/dd-trace-rb> and we're experimenting with using Ractors in our library but run into a few issues. ### Background When running a Ractor as a background process, the Ractor stops & does not restart when the application forks. ### How to reproduce (Ruby version & script) `ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]` ```ruby r2 = Ractor.new do loop { puts "[#{Process.pid}] Ractor"; sleep(1) } end sleep(1) puts "[#{Process.pid}] Forking..." fork do sleep(5) puts "[#{Process.pid}] End fork." end loop do sleep(1) end ``` ### Expectation and result The application prints “Ractor” each second in the main process, but not in the fork. Expected the Ractor (defined as `r2`) to run in the fork. ``` [29] Ractor [29] Ractor [29] Forking... [29] Ractor [29] Ractor [29] Ractor [29] Ractor [29] Ractor [32] End fork. [29] Ractor [29] Ractor [29] Ractor ``` ### Additional notes Threads do not restart across forks either, so it might not be unreasonable to expect consistent behavior. However, it’s possible to detect a dead Thread and recreate it after a fork (e.g. with `#alive?`, `#status`), but there’s no such mechanism for Ractors. ### Suggested solutions 1. Auto-restart Ractors after fork 2. Add additional methods to Ractors that allow users to check & manage the status of the Ractor, similar to Thread. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114535] [Ruby master Bug#19851] Ripper returns and extra block argument for :params with :args_forward
by thyresias (Thierry Lambert) 25 Aug '23

25 Aug '23

Issue #19851 has been reported by thyresias (Thierry Lambert). ---------------------------------------- Bug #19851: Ripper returns and extra block argument for :params with :args_forward https://bugs.ruby-lang.org/issues/19851 * Author: thyresias (Thierry Lambert) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby require 'ripper' class BasicParser < Ripper EVENTS.each do |event| module_eval(<<~RUBY, __FILE__, __LINE__ + 1) def on_#{event}(*args) puts "#{event}(\#{args.inspect})" args.unshift :#{event} args end RUBY end end BasicParser.new('def m(...); end').parse # stdout: # kw(["def"]) # sp([" "]) # ident(["m"]) # lparen(["("]) # op(["..."]) # args_forward([]) # params([nil, nil, nil, nil, nil, [:args_forward], :&]) ``` I would have expected the last `:params` argument to be `nil`, not `:&`. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114540] [Ruby master Bug#17678] Ractors do not restart after fork
by ivoanjo (Ivo Anjo) 25 Aug '23

25 Aug '23

Issue #17678 has been updated by ivoanjo (Ivo Anjo). Ack, that seems a reasonable way of looking at this, having a way to detect that the ractor is dead would be enough to write some auto-restart code after fork (Erlang supervisor trees here we go :D). ---------------------------------------- Bug #17678: Ractors do not restart after fork https://bugs.ruby-lang.org/issues/17678#change-104341 * Author: ivoanjo (Ivo Anjo) * Status: Assigned * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello there! I'm working at Datadog on the `ddtrace` gem -- <https://github.com/DataDog/dd-trace-rb> and we're experimenting with using Ractors in our library but run into a few issues. ### Background When running a Ractor as a background process, the Ractor stops & does not restart when the application forks. ### How to reproduce (Ruby version & script) `ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]` ```ruby r2 = Ractor.new do loop { puts "[#{Process.pid}] Ractor"; sleep(1) } end sleep(1) puts "[#{Process.pid}] Forking..." fork do sleep(5) puts "[#{Process.pid}] End fork." end loop do sleep(1) end ``` ### Expectation and result The application prints “Ractor” each second in the main process, but not in the fork. Expected the Ractor (defined as `r2`) to run in the fork. ``` [29] Ractor [29] Ractor [29] Forking... [29] Ractor [29] Ractor [29] Ractor [29] Ractor [29] Ractor [32] End fork. [29] Ractor [29] Ractor [29] Ractor ``` ### Additional notes Threads do not restart across forks either, so it might not be unreasonable to expect consistent behavior. However, it’s possible to detect a dead Thread and recreate it after a fork (e.g. with `#alive?`, `#status`), but there’s no such mechanism for Ractors. ### Suggested solutions 1. Auto-restart Ractors after fork 2. Add additional methods to Ractors that allow users to check & manage the status of the Ractor, similar to Thread. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114538] [Ruby master Bug#17678] Ractors do not restart after fork
by jeremyevans0 (Jeremy Evans) 25 Aug '23

25 Aug '23

Issue #17678 has been updated by jeremyevans0 (Jeremy Evans). Tracker changed from Feature to Bug Backport set to 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN Thanks for that information. It looks like ractors contain information about their state (shown in `inspect`), but that state is not updated on fork, unlike threads. So I think there is a bug, and it's that non-main ractors do not have their state to `terminated` upon fork (forking from non-main ractors is going to be prohibited, see #17516). ---------------------------------------- Bug #17678: Ractors do not restart after fork https://bugs.ruby-lang.org/issues/17678#change-104340 * Author: ivoanjo (Ivo Anjo) * Status: Assigned * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello there! I'm working at Datadog on the `ddtrace` gem -- <https://github.com/DataDog/dd-trace-rb> and we're experimenting with using Ractors in our library but run into a few issues. ### Background When running a Ractor as a background process, the Ractor stops & does not restart when the application forks. ### How to reproduce (Ruby version & script) `ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]` ```ruby r2 = Ractor.new do loop { puts "[#{Process.pid}] Ractor"; sleep(1) } end sleep(1) puts "[#{Process.pid}] Forking..." fork do sleep(5) puts "[#{Process.pid}] End fork." end loop do sleep(1) end ``` ### Expectation and result The application prints “Ractor” each second in the main process, but not in the fork. Expected the Ractor (defined as `r2`) to run in the fork. ``` [29] Ractor [29] Ractor [29] Forking... [29] Ractor [29] Ractor [29] Ractor [29] Ractor [29] Ractor [32] End fork. [29] Ractor [29] Ractor [29] Ractor ``` ### Additional notes Threads do not restart across forks either, so it might not be unreasonable to expect consistent behavior. However, it’s possible to detect a dead Thread and recreate it after a fork (e.g. with `#alive?`, `#status`), but there’s no such mechanism for Ractors. ### Suggested solutions 1. Auto-restart Ractors after fork 2. Add additional methods to Ractors that allow users to check & manage the status of the Ractor, similar to Thread. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114536] [Ruby master Bug#17516] forking in a ractor causes Ruby to crash
by jeremyevans0 (Jeremy Evans) 25 Aug '23

25 Aug '23

Issue #17516 has been updated by jeremyevans0 (Jeremy Evans). I submitted a pull request to disallow fork from inside a ractor: https://github.com/ruby/ruby/pull/8283 ---------------------------------------- Bug #17516: forking in a ractor causes Ruby to crash https://bugs.ruby-lang.org/issues/17516#change-104338 * Author: pkmuldoon (Phil Muldoon) * Status: Assigned * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin20] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- I just want to point out, there's absolutely no reason to do this, but r = Ractor.new do Process.fork() end Will cause: <internal:ractor>:267: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues. [BUG] rb_thread_terminate_all: called by child thread (0x0000700004ddca40, 0x00007f981b567ee0) ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin20] -- Crash Report log information -------------------------------------------- See Crash Report log file under the one of following: * ~/Library/Logs/DiagnosticReports * /Library/Logs/DiagnosticReports for more details. Don't forget to include the above Crash Report log file in bug reports. -- Control frame information ----------------------------------------------- c:0001 p:---- s:0003 e:000002 (none) [FINISH] -- C level backtrace information ------------------------------------------- => #<Ractor:#3 (pry):5 terminated> [4] pry(main)> /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_vm_bugreport+0x6cf) [0x103084d1f] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_bug_without_die+0x206) [0x102e9e2b6] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_bug+0x71) [0x103091e6b] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_thread_terminate_all+0x329) [0x10301e5b9] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_ractor_terminate_all+0xa3) [0x102f8acc3] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(rb_ec_cleanup+0x229) [0x102ea9299] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(ruby_stop+0x9) [0x102ea9509] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(thread_start_func_2+0x8ce) [0x103027fce] /Users/phillipmuldoon/.rubies/ruby-3.0.0/bin/ruby(thread_start_func_1+0x10d) [0x10302753d] /usr/lib/system/libsystem_pthread.dylib(_pthread_start+0xe0) [0x7fff20382950] ---Files-------------------------------- ruby_2021-01-06-104315_phillip-muldoon-FA588.crash (31.8 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114532] [Ruby master Bug#17926] spec/ruby/core/file/atime_spec.rb: a random failing test on Travis ppc64le
by jaruga (Jun Aruga) 25 Aug '23

25 Aug '23

Issue #17926 has been updated by jaruga (Jun Aruga). Status changed from Feedback to Closed jeremyevans0 (Jeremy Evans) wrote in #note-5: > @jaruga We haven't been seeing this error on the ppc64le RubyCI server recently. Is this OK to close? @jeremyevans0 Yeah, it's ok to close. ---------------------------------------- Bug #17926: spec/ruby/core/file/atime_spec.rb: a random failing test on Travis ppc64le https://bugs.ruby-lang.org/issues/17926#change-104334 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- I just observed the following test failing randomly on Travis ppc64le at the commit:fd65ef2a5aa8629676d1edb6410e4d4cf60b8045. When running this commit on my forked repo, the test passed. https://travis-ci.com/github/ruby/ruby/jobs/509310540#L2255 ``` $ $SETARCH make -s test-spec MSPECOPT=-ff ... 1) File.atime returns the last access time for the named file with microseconds FAILED Expected 0 == 123456 to be truthy but was false /home/travis/build/ruby/ruby/spec/ruby/core/file/atime_spec.rb:25:in `block (3 levels) in <top (required)>' /home/travis/build/ruby/ruby/spec/ruby/core/file/atime_spec.rb:3:in `<top (required)>' ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114530] [Ruby master Bug#17676] Accessing ENV from Ractor raises IsolationError
by ivoanjo (Ivo Anjo) 25 Aug '23

25 Aug '23

Issue #17676 has been updated by ivoanjo (Ivo Anjo). Thank you Jeremy! :) ---------------------------------------- Bug #17676: Accessing ENV from Ractor raises IsolationError https://bugs.ruby-lang.org/issues/17676#change-104329 * Author: delner (David Elner) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- ## Background Accessing ENV variables from a Ractor will raise Ractor::IsolationError, as the Hash is non-shareable. ## How to reproduce ``` ruby ractor = Ractor.new do ENV['PATH'] end ractor.take ``` ## Expectation and result Ractor raises Ractor::IsolationError for ENV Expect to be able to access ENV variables from Ractors without raising an error. ``` `block in <main>': can not access non-shareable objects in constant Object::ENV by non-main Ractor. (Ractor::IsolationError) ``` ## Additional Notes Attempting to create a shareable copy of ENV currently fails: ```ruby Ractor.make_shareable(ENV, copy: true) # TypeError: cannot freeze ENV # from <internal:ractor>:812:in `freeze' ``` ## Suggested solutions 1. If ENV is a mutable hash, make a Ractor-compatible copy of ENV isolated to that Ractor. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114528] [Ruby master Feature#17678] Ractors do not restart after fork
by ivoanjo (Ivo Anjo) 25 Aug '23

25 Aug '23

Issue #17678 has been updated by ivoanjo (Ivo Anjo). > The addition of Ractor#alive? and/or Ractor#status makes sense to me. Even in non-forked processes such methods could be useful. Note that you can get what you want already, by calling Ractor#inspect, so these methods would only need to expose information that Ractor is already storing. Thanks for looking into this! I don't think the info is there in #inspect... At least I don't get it on stable or latest ruby-head? 😅 Here's an updated example: ```ruby puts RUBY_DESCRIPTION r2 = Ractor.new { puts "[#{Process.pid}] Ractor started!"; sleep(1000) } puts "[#{Process.pid}] In parent process, ractor status is #{r2.inspect}" sleep(1) puts "[#{Process.pid}] Forking..." fork do puts "[#{Process.pid}] In child process, ractor status is #{r2.inspect}" end Process.wait ``` and here's what I get: ``` $ ruby ractor-test.rb ruby 3.3.0dev (2023-08-24T12:12:51Z master 5ec1fc52c1) [x86_64-linux] ractor-test.rb:3: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues. [10] In parent process, ractor status is #<Ractor:#2 ractor-test.rb:3 blocking> [10] Ractor started! [10] Forking... [12] In child process, ractor status is #<Ractor:#2 ractor-test.rb:3 blocking> ``` ---------------------------------------- Feature #17678: Ractors do not restart after fork https://bugs.ruby-lang.org/issues/17678#change-104327 * Author: ivoanjo (Ivo Anjo) * Status: Assigned * Priority: Normal * Assignee: ko1 (Koichi Sasada) ---------------------------------------- Hello there! I'm working at Datadog on the `ddtrace` gem -- <https://github.com/DataDog/dd-trace-rb> and we're experimenting with using Ractors in our library but run into a few issues. ### Background When running a Ractor as a background process, the Ractor stops & does not restart when the application forks. ### How to reproduce (Ruby version & script) `ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]` ```ruby r2 = Ractor.new do loop { puts "[#{Process.pid}] Ractor"; sleep(1) } end sleep(1) puts "[#{Process.pid}] Forking..." fork do sleep(5) puts "[#{Process.pid}] End fork." end loop do sleep(1) end ``` ### Expectation and result The application prints “Ractor” each second in the main process, but not in the fork. Expected the Ractor (defined as `r2`) to run in the fork. ``` [29] Ractor [29] Ractor [29] Forking... [29] Ractor [29] Ractor [29] Ractor [29] Ractor [29] Ractor [32] End fork. [29] Ractor [29] Ractor [29] Ractor ``` ### Additional notes Threads do not restart across forks either, so it might not be unreasonable to expect consistent behavior. However, it’s possible to detect a dead Thread and recreate it after a fork (e.g. with `#alive?`, `#status`), but there’s no such mechanism for Ractors. ### Suggested solutions 1. Auto-restart Ractors after fork 2. Add additional methods to Ractors that allow users to check & manage the status of the Ractor, similar to Thread. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114525] [Ruby master Bug#17263] Fiber context switch degrades with number of fibers, limit on number of fibers
by ioquatix (Samuel Williams) 25 Aug '23

25 Aug '23

Issue #17263 has been updated by ioquatix (Samuel Williams). Status changed from Open to Closed My current conclusion is this: Based on the `perf` `cpu-cycles:k`, we see proportional increase in overhead related to the number of fibers, despite ultimately having the same total number of context switches. This is unfortunate, but not exactly unexpected as we are stressing virtual memory. Here are the results of my testing: ``` | fibers | elapsed time (s) | rate (t/s) | | ---------------- | ---------------- | ---------------- | | 1 | 0.91 | 10998609.17 | | 2 | 0.82 | 12239077.16 | | 4 | 0.77 | 12930013.16 | | 8 | 0.79 | 12678091.91 | | 16 | 0.79 | 12578625.99 | | 32 | 0.79 | 12598729.93 | | 64 | 0.79 | 12597254.54 | | 128 | 0.79 | 12643086.20 | | 256 | 0.83 | 12116891.53 | | 512 | 0.94 | 10654248.57 | | 1024 | 1.01 | 9865286.58 | | 2048 | 1.04 | 9644781.53 | | 4096 | 1.06 | 9455585.41 | | 8192 | 1.10 | 9070485.29 | | 16384 | 1.98 | 5054997.19 | | 32768 | 3.14 | 3189286.37 | | 65536 | 3.39 | 2949265.02 | | 131072 | 3.39 | 2951698.03 | | 262144 | 3.44 | 2910388.50 | | 524288 | 3.43 | 2915666.38 | | 1048576 | 3.43 | 2917077.46 | ``` ---------------------------------------- Bug #17263: Fiber context switch degrades with number of fibers, limit on number of fibers https://bugs.ruby-lang.org/issues/17263#change-104323 * Author: ciconia (Sharon Rosner) * Status: Closed * Priority: Normal * ruby -v: 2.7.1 * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- I'm working on developing [Polyphony](https://github.com/digital-fabric/polyphony), a Ruby gem for writing highly-concurrent Ruby programs with fibers. In the course of my work I have come up against two problems using Ruby fibers: 1. Fiber context switching performance seem to degrade as the number of fibers is increased. This is both with `Fiber#transfer` and `Fiber#resume/Fiber.yield`. 2. The number of concurrent fibers that can exist at any time seems to be limited. Once a certain number is reached (on my system this seems to be 31744 fibers), calling `Fiber#transfer` will raise a `FiberError` with the message `can't set a guard page: Cannot allocate memory`. This is not due to RAM being saturated. With 10000 fibers, my test program hovers at around 150MB RSS (on Ruby 2.7.1). Here's a program for testing the performance of `Fiber#transfer`: ```ruby # frozen_string_literal: true require 'fiber' class Fiber attr_accessor :next end def run(num_fibers) count = 0 GC.start GC.disable first = nil last = nil supervisor = Fiber.current num_fibers.times do fiber = Fiber.new do loop do count += 1 if count == 1_000_000 supervisor.transfer else Fiber.current.next.transfer end end end first ||= fiber last.next = fiber if last last = fiber end last.next = first t0 = Time.now first.transfer elapsed = Time.now - t0 rss = `ps -o rss= -p #{Process.pid}`.to_i puts "fibers: #{num_fibers} rss: #{rss} count: #{count} rate: #{count / elapsed}" rescue Exception => e puts "Stopped at #{count} fibers" p e end run(100) run(1000) run(10000) run(100000) ``` With Ruby 2.6.5 I'm getting: ``` fibers: 100 rss: 23212 count: 1000000 rate: 3357675.1688139187 fibers: 1000 rss: 31292 count: 1000000 rate: 2455537.056439736 fibers: 10000 rss: 127388 count: 1000000 rate: 954251.1674325482 Stopped at 22718 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` With Ruby 2.7.1 I'm getting: ``` fibers: 100 rss: 23324 count: 1000000 rate: 3443916.967616508 fibers: 1000 rss: 34676 count: 1000000 rate: 2333315.3862491543 fibers: 10000 rss: 151364 count: 1000000 rate: 916772.1008060966 Stopped at 31744 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` With ruby-head I get an almost identical result to that of 2.7.1. As you can see, the performance degradation is similar in all the three versions of Ruby, going from ~3.4M context switches per second for 100 fibers to less then 1M context switches per second for 10000 fibers. Running with 100000 fibers fails to complete. Here's a program for testing the performance of `Fiber#resume/Fiber.yield`: ```ruby # frozen_string_literal: true require 'fiber' class Fiber attr_accessor :next end # This program shows how the performance of Fiber.transfer degrades as the fiber # count increases def run(num_fibers) count = 0 GC.start GC.disable fibers = [] num_fibers.times do fibers << Fiber.new { loop { Fiber.yield } } end t0 = Time.now while count < 1000000 fibers.each do |f| count += 1 f.resume end end elapsed = Time.now - t0 puts "fibers: #{num_fibers} count: #{count} rate: #{count / elapsed}" rescue Exception => e puts "Stopped at #{count} fibers" p e end run(100) run(1000) run(10000) run(100000) ``` With Ruby 2.7.1 I'm getting the following output: ``` fibers: 100 count: 1000000 rate: 3048230.049946255 fibers: 1000 count: 1000000 rate: 2362235.6455160403 fibers: 10000 count: 1000000 rate: 950251.7621725246 Stopped at 21745 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` As I understand it, theoretically at least switching between fibers should have a constant cost in terms of CPU cycles, irrespective of the number of fibers currently existing in memory. I am completely ignorant the implementation details of Ruby fibers, so at least for now I don't have any idea where this problem is coming from. ---Files-------------------------------- clipboard-202308251514-grqb1.png (81.3 KB) clipboard-202308251514-r7g4l.png (81 KB) clipboard-202308251538-kmofk.png (13.8 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114524] [Ruby master Bug#17263] Fiber context switch degrades with number of fibers, limit on number of fibers
by ioquatix (Samuel Williams) 25 Aug '23

25 Aug '23

Issue #17263 has been updated by ioquatix (Samuel Williams). File clipboard-202308251538-kmofk.png added 1 million fibers is causing ~600GiB of virtual address space to be consumed. That seems like quite a lot: ![](clipboard-202308251538-kmofk.png) I'm not surprised if this is causing the the OS to thrash/have issues. ---------------------------------------- Bug #17263: Fiber context switch degrades with number of fibers, limit on number of fibers https://bugs.ruby-lang.org/issues/17263#change-104320 * Author: ciconia (Sharon Rosner) * Status: Open * Priority: Normal * ruby -v: 2.7.1 * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- I'm working on developing [Polyphony](https://github.com/digital-fabric/polyphony), a Ruby gem for writing highly-concurrent Ruby programs with fibers. In the course of my work I have come up against two problems using Ruby fibers: 1. Fiber context switching performance seem to degrade as the number of fibers is increased. This is both with `Fiber#transfer` and `Fiber#resume/Fiber.yield`. 2. The number of concurrent fibers that can exist at any time seems to be limited. Once a certain number is reached (on my system this seems to be 31744 fibers), calling `Fiber#transfer` will raise a `FiberError` with the message `can't set a guard page: Cannot allocate memory`. This is not due to RAM being saturated. With 10000 fibers, my test program hovers at around 150MB RSS (on Ruby 2.7.1). Here's a program for testing the performance of `Fiber#transfer`: ```ruby # frozen_string_literal: true require 'fiber' class Fiber attr_accessor :next end def run(num_fibers) count = 0 GC.start GC.disable first = nil last = nil supervisor = Fiber.current num_fibers.times do fiber = Fiber.new do loop do count += 1 if count == 1_000_000 supervisor.transfer else Fiber.current.next.transfer end end end first ||= fiber last.next = fiber if last last = fiber end last.next = first t0 = Time.now first.transfer elapsed = Time.now - t0 rss = `ps -o rss= -p #{Process.pid}`.to_i puts "fibers: #{num_fibers} rss: #{rss} count: #{count} rate: #{count / elapsed}" rescue Exception => e puts "Stopped at #{count} fibers" p e end run(100) run(1000) run(10000) run(100000) ``` With Ruby 2.6.5 I'm getting: ``` fibers: 100 rss: 23212 count: 1000000 rate: 3357675.1688139187 fibers: 1000 rss: 31292 count: 1000000 rate: 2455537.056439736 fibers: 10000 rss: 127388 count: 1000000 rate: 954251.1674325482 Stopped at 22718 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` With Ruby 2.7.1 I'm getting: ``` fibers: 100 rss: 23324 count: 1000000 rate: 3443916.967616508 fibers: 1000 rss: 34676 count: 1000000 rate: 2333315.3862491543 fibers: 10000 rss: 151364 count: 1000000 rate: 916772.1008060966 Stopped at 31744 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` With ruby-head I get an almost identical result to that of 2.7.1. As you can see, the performance degradation is similar in all the three versions of Ruby, going from ~3.4M context switches per second for 100 fibers to less then 1M context switches per second for 10000 fibers. Running with 100000 fibers fails to complete. Here's a program for testing the performance of `Fiber#resume/Fiber.yield`: ```ruby # frozen_string_literal: true require 'fiber' class Fiber attr_accessor :next end # This program shows how the performance of Fiber.transfer degrades as the fiber # count increases def run(num_fibers) count = 0 GC.start GC.disable fibers = [] num_fibers.times do fibers << Fiber.new { loop { Fiber.yield } } end t0 = Time.now while count < 1000000 fibers.each do |f| count += 1 f.resume end end elapsed = Time.now - t0 puts "fibers: #{num_fibers} count: #{count} rate: #{count / elapsed}" rescue Exception => e puts "Stopped at #{count} fibers" p e end run(100) run(1000) run(10000) run(100000) ``` With Ruby 2.7.1 I'm getting the following output: ``` fibers: 100 count: 1000000 rate: 3048230.049946255 fibers: 1000 count: 1000000 rate: 2362235.6455160403 fibers: 10000 count: 1000000 rate: 950251.7621725246 Stopped at 21745 fibers #<FiberError: can't set a guard page: Cannot allocate memory> ``` As I understand it, theoretically at least switching between fibers should have a constant cost in terms of CPU cycles, irrespective of the number of fibers currently existing in memory. I am completely ignorant the implementation details of Ruby fibers, so at least for now I don't have any idea where this problem is coming from. ---Files-------------------------------- clipboard-202308251514-grqb1.png (81.3 KB) clipboard-202308251514-r7g4l.png (81 KB) clipboard-202308251538-kmofk.png (13.8 KB) -- https://bugs.ruby-lang.org/

1 0