[ruby-core:119772] [Ruby master Feature#20876] Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop.
Issue #20876 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #20876: Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop. https://bugs.ruby-lang.org/issues/20876 * Author: ioquatix (Samuel Williams) * Status: Open ---------------------------------------- This is an evolution of the previous proposal: https://bugs.ruby-lang.org/issues/20855 ## Background The current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking operations for efficient task execution. ## Proposal Pull Request: https://github.com/ruby/ruby/pull/12016 We will introduce a new fiber scheduler hook called `blocking_operation_work`: ```ruby class MySchduler # ... def blocking_operation_wait(work) # Example implementation: Thread.new(&work).join end end ``` We introduce a new flag for `rb_nogvl`: `RB_NOGVL_BLOCKING_OPERATION` which indicates that `rb_nogvl(func, ...)` is a blocking operation that is safe to execute on a different thread or thread pool. When a C extension invokes `rb_nogvl(..., RB_NOGVL_BLOCKING_OPERATION)`, and a fiber scheduler is available, all the arguments will be saved into a instance of a callable object (at this time a `Proc`) called `work`. When `work` is `#call`ed, it will execute `rb_nogvl` again with all the same arguments. The fiber scheduler can decide how to execute that work, e.g. on a separate thread, to mitigate the performance impact of the blocking operation on the event loop.  ## Example Using the branch of `async` gem: https://github.com/socketry/async/pull/352/files and enabling zlib deflate to use this feature, the following performance improvement was achieved: ```ruby require "zlib" require "async" require "benchmark" DATA = Random.new.bytes(1024*1024*100) duration = Benchmark.measure do Async do 10.times do Async do Zlib.deflate(DATA) end end end end # Ruby 3.3.4: ~16 seconds # Ruby 3.4.0 + PR: ~2 seconds. ``` ---Files-------------------------------- clipboard-202411070126-fbqpn.png (135 KB) -- https://bugs.ruby-lang.org/
Issue #20876 has been updated by ioquatix (Samuel Williams). @matz said:
go ahead
https://github.com/ruby/dev-meeting-log/blob/master/2024/DevMeeting-2024-11-... @ko1 are we good to merge? ---------------------------------------- Feature #20876: Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop. https://bugs.ruby-lang.org/issues/20876#change-110578 * Author: ioquatix (Samuel Williams) * Status: Open ---------------------------------------- This is an evolution of the previous proposal: https://bugs.ruby-lang.org/issues/20855 ## Background The current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking operations for efficient task execution. ## Proposal Pull Request: https://github.com/ruby/ruby/pull/12016 We will introduce a new fiber scheduler hook called `blocking_operation_work`: ```ruby class MySchduler # ... def blocking_operation_wait(work) # Example (trivial) implementation: Thread.new(&work).join end end ``` We introduce a new flag for `rb_nogvl`: `RB_NOGVL_OFFLOAD_SAFE` which indicates that `rb_nogvl(func, ...)` is a blocking operation that is safe to execute on a different thread or thread pool (or some other context). When a C extension invokes `rb_nogvl(..., RB_NOGVL_OFFLOAD_SAFE)`, and a fiber scheduler is available, all the arguments will be saved into a instance of a callable object (at this time a `Proc`) called `work` and passed to the `blocking_operation_wait` fiber scheduler hook. When `work` is `#call`ed, it will execute `rb_nogvl` again with all the same arguments. The fiber scheduler can decide how to execute that work, e.g. on a separate thread or thread pool, to mitigate the performance impact of the blocking operation on the event loop.  ### Cancellation `rb_nogvl` takes several arguments, a `func` for the actual work, and `unblock_func` to cancel `func` if possible. These arguments are preserved in the `work` proc, and cancellation works the same. However, some extra effort may be required in the fiber scheduler hook, e.g. ```ruby class MySchduler # ... def blocking_operation_wait(work) thread = Thread.new(&work) thread.join thread = nil ensure thread&.kill end end ``` ## Example Using the branch of `async` gem: https://github.com/socketry/async/pull/352/files and enabling zlib deflate to use this feature, the following performance improvement was achieved: ```ruby require "zlib" require "async" require "benchmark" DATA = Random.new.bytes(1024*1024*100) duration = Benchmark.measure do Async do 10.times do Async do Zlib.deflate(DATA) end end end end # Ruby 3.3.4: ~16 seconds # Ruby 3.4.0 + PR: ~2 seconds. ``` To run this benchmark yourself, you must compile CRuby with these two PRs: - https://github.com/ruby/ruby/pull/12016 - https://github.com/ruby/zlib/pull/88 In addition, enable `RB_NOGVL_OFFLOAD_SAFE` in `zlib.c`'s call to `rb_nogvl`. Then, use this branch of async: https://github.com/socketry/async/pull/352 ---Files-------------------------------- clipboard-202411071531-gw8tg.png (200 KB) -- https://bugs.ruby-lang.org/
Issue #20876 has been updated by ko1 (Koichi Sasada). go ahead. ---------------------------------------- Feature #20876: Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop. https://bugs.ruby-lang.org/issues/20876#change-110699 * Author: ioquatix (Samuel Williams) * Status: Open ---------------------------------------- This is an evolution of the previous proposal: https://bugs.ruby-lang.org/issues/20855 ## Background The current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking operations for efficient task execution. ## Proposal Pull Request: https://github.com/ruby/ruby/pull/12016 We will introduce a new fiber scheduler hook called `blocking_operation_work`: ```ruby class MySchduler # ... def blocking_operation_wait(work) # Example (trivial) implementation: Thread.new(&work).join end end ``` We introduce a new flag for `rb_nogvl`: `RB_NOGVL_OFFLOAD_SAFE` which indicates that `rb_nogvl(func, ...)` is a blocking operation that is safe to execute on a different thread or thread pool (or some other context). When a C extension invokes `rb_nogvl(..., RB_NOGVL_OFFLOAD_SAFE)`, and a fiber scheduler is available, all the arguments will be saved into a instance of a callable object (at this time a `Proc`) called `work` and passed to the `blocking_operation_wait` fiber scheduler hook. When `work` is `#call`ed, it will execute `rb_nogvl` again with all the same arguments. The fiber scheduler can decide how to execute that work, e.g. on a separate thread or thread pool, to mitigate the performance impact of the blocking operation on the event loop.  ### Cancellation `rb_nogvl` takes several arguments, a `func` for the actual work, and `unblock_func` to cancel `func` if possible. These arguments are preserved in the `work` proc, and cancellation works the same. However, some extra effort may be required in the fiber scheduler hook, e.g. ```ruby class MySchduler # ... def blocking_operation_wait(work) thread = Thread.new(&work) thread.join thread = nil ensure thread&.kill end end ``` ## Example Using the branch of `async` gem: https://github.com/socketry/async/pull/352/files and enabling zlib deflate to use this feature, the following performance improvement was achieved: ```ruby require "zlib" require "async" require "benchmark" DATA = Random.new.bytes(1024*1024*100) duration = Benchmark.measure do Async do 10.times do Async do Zlib.deflate(DATA) end end end end # Ruby 3.3.4: ~16 seconds # Ruby 3.4.0 + PR: ~2 seconds. ``` To run this benchmark yourself, you must compile CRuby with these two PRs: - https://github.com/ruby/ruby/pull/12016 - https://github.com/ruby/zlib/pull/88 In addition, enable `RB_NOGVL_OFFLOAD_SAFE` in `zlib.c`'s call to `rb_nogvl`. Then, use this branch of async: https://github.com/socketry/async/pull/352 ---Files-------------------------------- clipboard-202411071531-gw8tg.png (200 KB) -- https://bugs.ruby-lang.org/
participants (2)
-
ioquatix (Samuel Williams) -
ko1 (Koichi Sasada)