[ruby-core:119887] [Ruby master Feature#20876] Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop.

12 Nov 2024

      Issue #20876 has been updated by ioquatix (Samuel Williams).

@matz said:
...
go ahead
https://github.com/ruby/dev-meeting-log/blob/master/2024/DevMeeting-2024-11-...

@ko1 are we good to merge?

----------------------------------------
Feature #20876: Introduce `Fiber::Scheduler#blocking_operation_wait` to avoid stalling the event loop.
https://bugs.ruby-lang.org/issues/20876#change-110578

* Author: ioquatix (Samuel Williams)
* Status: Open
----------------------------------------
This is an evolution of the previous proposal: https://bugs.ruby-lang.org/issues/20855

## Background

The current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking operations for efficient task execution.

## Proposal

Pull Request: https://github.com/ruby/ruby/pull/12016

We will introduce a new fiber scheduler hook called `blocking_operation_work`:

```ruby
class MySchduler
  # ...
  def blocking_operation_wait(work)
    # Example (trivial) implementation:
    Thread.new(&work).join
  end
end
```

We introduce a new flag for `rb_nogvl`: `RB_NOGVL_OFFLOAD_SAFE` which indicates that `rb_nogvl(func, ...)` is a blocking operation that is safe to execute on a different thread or thread pool (or some other context).

When a C extension invokes `rb_nogvl(..., RB_NOGVL_OFFLOAD_SAFE)`, and a fiber scheduler is available, all the arguments will be saved into a instance of a callable object (at this time a `Proc`) called `work` and passed to the `blocking_operation_wait` fiber scheduler hook. When `work` is `#call`ed, it will execute `rb_nogvl` again with all the same arguments.

The fiber scheduler can decide how to execute that work, e.g. on a separate thread or thread pool, to mitigate the performance impact of the blocking operation on the event loop.

![](clipboard-202411071531-gw8tg.png)

### Cancellation

`rb_nogvl` takes several arguments, a `func` for the actual work, and `unblock_func` to cancel `func` if possible. These arguments are preserved in the `work` proc, and cancellation works the same. However, some extra effort may be required in the fiber scheduler hook, e.g.

```ruby
class MySchduler
  # ...
  def blocking_operation_wait(work)
    thread = Thread.new(&work)

    thread.join
    thread = nil
  ensure
    thread&.kill
  end
end
```

## Example

Using the branch of `async` gem: https://github.com/socketry/async/pull/352/files and enabling zlib deflate to use this feature, the following performance improvement was achieved:

```ruby
require "zlib"
require "async"
require "benchmark"

DATA = Random.new.bytes(1024*1024*100)

duration = Benchmark.measure do
  Async do
    10.times do
      Async do
        Zlib.deflate(DATA)
      end
    end
  end
end

# Ruby 3.3.4: ~16 seconds
# Ruby 3.4.0 + PR: ~2 seconds.
```

To run this benchmark yourself, you must compile CRuby with these two PRs:
- https://github.com/ruby/ruby/pull/12016
- https://github.com/ruby/zlib/pull/88

In addition, enable `RB_NOGVL_OFFLOAD_SAFE` in `zlib.c`'s call to `rb_nogvl`.

Then, use this branch of async: https://github.com/socketry/async/pull/352

---Files--------------------------------
clipboard-202411071531-gw8tg.png (200 KB)

-- 
https://bugs.ruby-lang.org/