Issue #22216 has been updated by matz (Yukihiro Matsumoto). You are right that the blocking based rule does not work, since fibers for enumerators are non-blocking. I withdraw that idea. I agree that plain Fiber-local should be the default. But I still think `e.next` is a real problem. The same block behaves differently with `e.each` and `e.next`, only because `next` uses a fiber internally. Both directions break: ```ruby e = log.lazy.select { |line| line =~ /"(\w+) (\S+) HTTP/ } e.next $1 # nil with Fiber-local svars "abc" =~ /(b)/ e = [1, 2].lazy.map { |x| "#{$1}#{x}" } e.next # "1" instead of "b1" ``` The `lazy.select` example in #note-13 works in Ruby 4.0. The observable behavior should not depend on whether the implementation uses a fiber. The difference is how the fiber is used. `e.next` is a synchronous call. The caller stops until the value comes back, so there is no concurrency. Sibling fibers under a scheduler are independent units, and that is where the problem in #note-12 is. We cannot tell them apart by `resume`, since schedulers may also use `resume`. So it should be decided when the fiber is created. My proposal: ```ruby Fiber.new(transparent: true) { ... } # name is tentative ``` * While resumed, the fiber shares the svars of frames outside its own stack (e.g. the method that owns the block) with the fiber that resumed it. Frames on its own stack have their own svars. * It cannot be transferred (`FiberError`). * Without the option, fibers are isolated (Fiber-local). Schedulers need no change. * `Enumerator` uses this option internally. Users who write their own generators with fibers can use it too. We already have `blocking:` and `storage:` to describe the relation between a fiber and its surroundings, so this fits there. For now it covers only svars, but the same issue exists for `Thread#[]` inside `e.next`, so the name should describe the role rather than svars. @jhawthorn, do you think this can be implemented on top of your PR? Matz. ---------------------------------------- Bug #22216: Special variables (ex. Regexp backref and IO lastline) are thread-unsafe in some cases, incompatible with Ractor https://bugs.ruby-lang.org/issues/22216#change-118935 * Author: headius (Charles Nutter) * Status: Open * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ## Problem Several Regexp-matching methods currently write (and sometimes read) the implicit "backref" `$~` variable in the local frame (and related variables like `$'`). Several IO methods read or write the "last line" `$_` variable in the same way. In both cases, the result is a mutable object, which makes these variables already problematic for ractors. Making matters worse, the frame might be shared if a proc is captured and used across threads or ractors, and there's no static way to inspect a piece of code to know if it expects to read or write these variables. Where procs can be rejected by a proc for accessing captured state, there's no such check possible for these variables. All of these facts make the backref and lastline variables fundamentally incompatible with Ractor. ## Possible remedies A wholesale removal of these variables would solve the problem, but there's a lot of code that depends on them... much of that code without even realizing it, since they might not access the variables directly. In some cases, the dependencies are internal and part of the behavior of core methods. Hard errors when using methods that read or write these variables would avoid introducing threading problems into a Ractor, but would also break a large number of commonly-used methods. There have been experiments to make these variables both frame and thread-local, but they have never been made standard. Updates to backref and lastline are visible across threads and already can lead to concurrency issues even on CRuby. Deprecating the implicit behavior and making it opt-in (or opt-out?), perhaps with keyword arguments or file pragmas, might be a halfway measure. It would probably not be an easy transition. I don't know the right path forward, but I believe this issue needs to be discussed. ## JRuby perspective We continue to mimic CRuby behavior, which has led to our users occasionally running into issues when a proc accesses these variables across threads. Our recommendation: "don't do that". We also have had our frustrations optimizing around these variables, since they implicitly require access across calls. Because we cannot statically detect when they will be used, we essentially treat all method names that might *potentially* access them as deopt triggers. It's not ideal. I'd like to hear ideas for how to make these variables less "magic", less implicit and easier to deal with across calls (and across threads/ractors). -- https://bugs.ruby-lang.org/