[ruby-core:115879] [Ruby master Bug#20081] Transfered Fiber doesn't return to Fiber that started it
Issue #20081 has been reported by rmosolgo (Robert Mosolgo). ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by rmosolgo (Robert Mosolgo). I see that Ruby 3.3 has `Fiber#kill` coming (https://github.com/ruby/ruby/pull/7823), which I _think_ will solve my problem: I can manually `.kill` Fibers instead of running them until they return. But still, I'm interested to learn whether this is a bug or not! ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-105836 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). My initial feeling is that this is not a bug. Transferring to a fiber is a uni-directional operation and no information is expected to be kept about the caller. If that is what you desire, use resume. There is nothing that prevents you from using resume and then transfer. If you do desire to transfer back to a specific fiber, you can code to that, e.g. ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" ensure # Exit: parent.transfer end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I don't know if there is a better semantic worth trying to tease out of this, but the semantics of `transfer` are fairly straight forward and I'm not sure we should change it as the chance of breaking something is probably fairly high. ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-105893 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by rmosolgo (Robert Mosolgo). Thanks for taking a look! I was going from the example in the docs (https://docs.ruby-lang.org/en/master/Fiber.html#method-i-transfer), where the return value of `.transfer` is used by the caller. And `.transfer` _does_ return to the caller, as long as the caller is the main Fiber, for example, this prints out a sequence of messages: ```ruby parent = Fiber.current worker = Fiber.new do |initial_arg| puts initial_arg # => 2. Begin Worker puts parent.transfer("3. Resume Parent") # => 4. Resume Worker "5. Exit Worker" end puts "1. Start" puts worker.transfer("2. Begin Worker") # => "3. Resume Parent" puts worker.transfer("4. Resume Worker") # => "5. Exit Worker" puts "6. Exit Script" ``` But it behaves differently when inside a surrounding, non-main Fiber: ```ruby manager = Fiber.new do |manager_init| puts manager_init # => 1. Start parent = Fiber.current worker = Fiber.new do |initial_arg| puts initial_arg # => 2. Begin Worker puts parent.transfer("3. Resume Parent") "5. Exit Worker" end puts worker.transfer("2. Begin Worker") # => "3. Resume Parent" puts worker.transfer("4. Resume Worker") # => "5. Exit Worker" "6. Exit Script" # This isn't printed end puts manager.transfer("1. Start") # 1. Start # 2. Begin Worker # 3. Resume Parent # 4. Resume Worker # 5. Exit Worker # # ... (6 doesn't print here) ``` Maybe the first example is a special case, since it's the main Fiber. I'd be happy to add a comment about that to the Fiber docs if that'd be helpful! ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-105896 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). In think clarifying the documentation is a good idea.
And .transfer does return to the caller, as long as the caller is the main Fiber, for example, this prints out a sequence of messages:
I think that's just intuitively correct based on the semantics. Actually, one could argue that a fiber that was transferred to, that exits, without an explicit transfer, should be considered exiting the thread completely and in your case terminating the program. In this case, Ruby is being a bit generous with "transferring back to the main fiber" IMHO. Transfer should be seen as a one way operation. ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-105897 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). Possible fix: https://github.com/ruby/ruby/pull/17601 ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-117837 * Author: rmosolgo (Robert Mosolgo) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by mame (Yusuke Endoh). @ioquatix I don't think the PR solves this issue. In the following example, worker 1 is abandoned in a suspended state: ```ruby manager = Fiber.new do parent = Fiber.current worker2 = nil worker1 = Fiber.new do puts "worker 1: start" worker2.transfer puts "worker 1: end" end worker2 = Fiber.new do puts "worker 2: start" parent.transfer puts "worker 2: end" end puts "manager: start" worker1.transfer # worker1 -> worker2 -> parent puts "manager: middle" worker2.transfer # worker2 -> (exit) -> parent puts "manager: end" # Now manager's prev is worker2, but it is already over, so it returns to the root fiber. # So "worker 1: end" is not reached. end puts "script: start" manager.transfer puts "script: end" ``` My understanding is that returning to the root fiber when a fiber started by transfer reaches its end is by design. This spec is already confusing, and I'm afraid this PR makes it even more confusing. (IMO, the thread should be terminated when a transferred fiber reaches its end.) As you said, I think the solution here is: "If that is what you desire, use resume." ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-117838 * Author: rmosolgo (Robert Mosolgo) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). I think the crux of the issue here is that there is no way for a user to a final transfer (or raise that transfers) that goes back to a specific fiber, but it probably should be possible. ``` fiber = Fiber.new do # Do stuff # Final transfer is back to scheduler: scheduler.transfer end # from scheduler fiber.transfer fiber.alive? # false ``` Basically, in order to exit a fiber (for it to become dead) you have to rely on it exiting, but you don't have control over where it goes. I'm still not sure what the right solution here is but the above PR was just an experiment. ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-117840 * Author: rmosolgo (Robert Mosolgo) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). Actually, this behaviour isn't as bad as I thought it was: ``` f1 = Fiber.new do f2 = Fiber.new do puts "f2" end.transfer puts "f1" end.resume f2 f1 ``` I thought f2 would transfer back out to the main fiber, but it goes back to the nested resumed fiber. ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-117841 * Author: rmosolgo (Robert Mosolgo) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
Issue #20081 has been updated by ioquatix (Samuel Williams). Okay, I spoke too soon, I think this clearly demonstrates the confusion: ```ruby def event_loop loop_fiber = Fiber.new do task = Fiber.new{puts "task"} task.transfer # io-event drives tasks via transfer. puts "loop: after task" # never runs. end loop_fiber.resume # resume, not transfer. puts "outer: after loop" # should run after the task exits. end puts "Works:" event_loop puts "Doesn't work:" Fiber.new do # transferred-to by root. event_loop end.transfer # the one transfer hop that breaks it. ``` ---------------------------------------- Bug #20081: Transfered Fiber doesn't return to Fiber that started it https://bugs.ruby-lang.org/issues/20081#change-117842 * Author: rmosolgo (Robert Mosolgo) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to figure out how to make sure that Fibers started with `.transfer` end up _terminated_, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.) So, I'm looking for way to make sure that any Fiber I start with `.transfer` will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue: ```ruby manager = Fiber.new do parent = Fiber.current worker = Fiber.new do puts "2. Begin Worker" parent.transfer puts "4. End Worker" end puts "1. Transfer 1" worker.transfer puts "3. Transfer 2" worker.transfer puts "5. Finished manager" end manager.transfer puts "6. Finished script" ``` I expect the steps to print in order: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 5. Finished manager 6. Finished script ``` But instead, `5. ...` is skipped: ``` 1. Transfer 1 2. Begin Worker 3. Transfer 2 4. End Worker 6. Finished script ``` I think that's because my `worker` fiber terminates and passes control back to the top-level Fiber. Should it have passed control back to the `manager`? Or is there another way to make sure `worker` is terminated, and `manager` gets control? -- https://bugs.ruby-lang.org/
participants (3)
-
ioquatix (Samuel Williams) -
mame (Yusuke Endoh)
-
rmosolgo (Robert Mosolgo)