Issue #22435 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #22435: Autoload under a Fiber scheduler causes a deadlock or [BUG] https://bugs.ruby-lang.org/issues/22435 * Author: mame (Yusuke Endoh) * Status: Assigned * Assignee: ioquatix (Samuel Williams) * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- The following code causes a deadlock. test/fiber/scheduler.rb is the one in the ruby/ruby repository (https://github.com/ruby/ruby/blob/master/test/fiber/scheduler.rb). ```ruby # fs-autoload-deadlock.rb require_relative "test/fiber/scheduler" class MyScheduler < Scheduler def blocking_operation_wait(work) @workers ||= Fiber.blocking { Etc.nprocessors } # lazily size a worker pool super end end autoload :Etc, "etc" autoload :JSON, "json" $LOADED_FEATURES << "dummy" # force to rebuild the index of $LOADED_FEATURES Thread.new do Fiber.set_scheduler(MyScheduler.new) Fiber.schedule { p JSON } end.join ``` ``` $ ruby fs-autoload-deadlock.rb #<Thread:0x00007ce4cf1eed90 fs-autoload-deadlock.rb:14 run> terminated with exception (report_on_exception is true): fs-autoload-deadlock.rb:5:in 'block in MyScheduler#blocking_operation_wait': deadlock; recursive locking (ThreadError) from fs-autoload-deadlock.rb:5:in 'Fiber.blocking' from fs-autoload-deadlock.rb:5:in 'MyScheduler#blocking_operation_wait' from fs-autoload-deadlock.rb:16:in 'block (2 levels) in <main>' ... ``` Also, the following code causes [BUG]. ```ruby # fs-autoload-segv.rb require_relative "test/fiber/scheduler" class MyScheduler < Scheduler def block(blocker, timeout = nil) blocker.inspect #=> [BUG] Segmentation fault at 0x0000000000000000 super end end autoload :JSON, "json" $LOADED_FEATURES << "dummy" # force to rebuild the index of $LOADED_FEATURES Thread.new do Fiber.set_scheduler(MyScheduler.new) Fiber.schedule { JSON } Fiber.schedule { JSON } end.join ``` ``` $ ruby fs-autoload-segv.rb fs-autoload-segv.rb:5: [BUG] Segmentation fault at 0x0000000000000000 ruby 4.1.0dev (2026-08-21T07:50:27Z master c9e9101403) +PRISM [x86_64-linux] ... -- Ruby level backtrace information ---------------------------------------- fs-autoload-segv.rb:16:in 'block (2 levels) in <main>' fs-autoload-segv.rb:5:in 'block' ... ``` Both reproduce on master and 4.0.6. `$LOADED_FEATURES << "dummy"` is to make Ruby rebuild the index of `$LOADED_FEATURES` (stat(2) during the rebuild calls `blocking_operation_wait`). ## Cause The cause is the autoload mutex introduced by commit:32de6097b2b5d8394b3a1399e13d309444697954 (https://github.com/ruby/ruby/pull/5898). The deadlock above occurs because this mutex is locked recursively. Also, when a fiber blocks on locking the mutex, the mutex is passed to `Fiber::Scheduler#block` as the blocker, but it is a hidden object, which leads to rb_bug. ## Analysis Let me recap the problem that commit:32de6097b2 tried to solve, [Bug #18782]. Autoload roughly proceeded in the following steps 1, 2, and 3. 1. If the mutex to synchronize requiring the feature has already been created and another thread is locking it, wait, because that thread is requiring the feature 2. If the feature has not been required yet, create a mutex for the feature, lock it, and require the feature 3. Refer to the constant that should have been defined Consider the following code. ```ruby autoload :Foo, "foobar.rb" autoload :Bar, "foobar.rb" tA = Thread.new {Foo} tB = Thread.new {Bar} ``` If tA and tB executed the steps 1, 2, and 3 in the following order, a NameError was raised in the thread tA. * tA: executes step 1 (there is no mutex, so it proceeds without waiting) * context switch * tB: executes step 1 (there is no mutex, so it proceeds without waiting) * tB: executes step 2 (creates and locks the mutex, and requires the feature. When the require finishes, the feature is marked as required, but the autoload constants are defined after that) * context switch * tA: executes step 2 (the feature has been required, so it does nothing) * tA: executes step 3 (the constant is not defined yet, so NameError) The check in step 2, whether the feature has not been required yet, may call `to_str` of an element of `$LOADED_FEATURES` or release the GVL, so a context switch can occur there. ## Solution commit:32de6097b2 tried to solve it by adding a new mutex to lock the steps 1, 2, and 3 of autoload, but as shown above, this is a new source of deadlocks and adds complexity, so I don't think it is good. I think it can be solved simply by swapping the steps 1 and 2 of autoload. 1'. If the feature has not been required yet, create a mutex for the feature, lock it, and require the feature 2'. If the mutex to synchronize requiring the feature has already been created and another thread is locking it, wait, because that thread is requiring the feature 3. Refer to the constant that should have been defined The thread that requires the feature keeps the mutex locked until it finishes defining the constants. So, if the feature looks "required" at 1' but the constants are not defined yet, the mutex is still locked at 2', and the thread waits for it. Also, even if both threads see the feature as not required yet at 1', only one mutex is created. Checking whether the mutex exists and creating it involve no Ruby code nor GVL release, so no thread switch occurs between them thanks to the GVL. The thread that comes later finds the mutex created by the other, and waits for it. This solves the NameError problem of [Bug #18782] without the global autoload mutex. So, how about removing the global autoload mutex introduced by commit:32de6097b2, and swapping the steps as above? PR: https://github.com/ruby/ruby/pull/19335 In addition to passing all Ruby tests, I also checked the tests of zeitwerk and bootsnap just in case. All tests of zeitwerk pass. One test of bootsnap (`test_load_static_libaries`) fails, but it is because prism is no longer a default gem on master, and it is unrelated to this change. -- https://bugs.ruby-lang.org/