[ruby-core:124051] [Ruby Feature#21767] Consider procs which `self` is Ractor-shareable as Ractor shareable
Issue #21767 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Feature #21767: Consider procs which `self` is Ractor-shareable as Ractor shareable https://bugs.ruby-lang.org/issues/21767 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- I would like to allow procs which `self` is Ractor-shareable to be automatically eligible as shareable, without an explicit `Ractor.make_shareable` call. ```ruby class C PROC = proc { p ARRAY }.freeze end Ractor.new { C::PROC.call }.value # Allow this, since `C` is shareable ``` Proposal is: Consider procs/lambdas which meet the following condition as Ractor-shareable. - The proc is frozen. - The proc's `self` is shareable. This proposal is has taken inspiration from #21033 . ## Usecase The main usecase in mind is procs/lambdas in class-level constants. Some libraries store procs in constants as a convenient place for library-wide logic. Those procs usually do not access unshareable state, thus conceptually safe to be shared across Ractors. However, the current limitation completely blocks this. Examples may be found in ruby/ruby, and I have submitted a pull request to migrate one to `Ractor.shareable_proc`. ```ruby class Pathname SAME_PATHS = if File::FNM_SYSCASE.nonzero? # Avoid #zero? here because #casecmp can return nil. proc {|a, b| a.casecmp(b) == 0} else proc {|a, b| a == b} end end ``` https://github.com/search?q=repo%3Aruby%2Fruby%20%2F(%3F-i)%5BA-Z%5D%20%3D%20(proc%7Clambda)%2F&type=code More examples can be found in public code. https://github.com/search?q=language%3Aruby+%2F%28%3F-i%29%5BA-Z%5D+%3D+%28proc%7Clambda%29%2F&type=code It can be observed that a good portion of these do not access unshareable state. Appending `.freeze` would be much more acceptable than redefining using `Ractor.shareable_proc`, which is a Ruby 4.0-only feature. ## Discussion: Change of behavior when illegal access occurs in proc Consider this code. The proc accesses non-frozen `C::ARRAY`, which is against the rules of Ractors regardless of this patch. ```ruby class C ARRAY = [] PROC = proc { p ARRAY } end # Still illegal since C::ARRAY is not shareable Ractor.new { C::PROC.call }.value ``` This code used to raise on `C::PROC.call` (`can not access non-shareable objects in constant C::PROC by non-main Ractor.`). When this patch is applied, it will raise on `p ARRAY` (`can not access non-shareable objects in constant C::ARRAY by non-main ractor.`). This could be debatable change. If this is not acceptable, I'd like to revisit #21033 . -- https://bugs.ruby-lang.org/
Issue #21767 has been updated by Eregon (Benoit Daloze). Naturally it would need to follow the rules of `Ractor.shareable_proc` (#21557) regarding accessing captured variables: * Any captured variable must not be reassigned * The value of every captured variable must be shareable (not sure how common that is the case, so this might limit the usefulness of this proposal quite a bit in practice) Besides that, it sounds convenient that Procs which satisfy those rules (i.e. no exception when `Ractor.shareable_proc` is called on them) + already have a shareable `self` are automatically `shareable`. It's just of course that Procs already having a shareable `self` are not very common. There is one incompatibility though, that by making that Proc shareable it can behave differently on the main Ractor than before this proposal: ```ruby class Foo a = 42 SOME_PROC = -> { eval "a" } a += 1 SOME_PROC_SHAREABLE = Ractor.shareable_proc(&SOME_PROC) end p Foo::SOME_PROC.call puts Foo::SOME_PROC_SHAREABLE.call ``` `SOME_PROC` is not detected as accessing captured variables because of `eval`. If the Proc is not shareable it still works as it always did (return 43). If `SOME_PROC` is made automatically shareable by this proposal then it now raises an error, as simulated by `SOME_PROC_SHAREABLE`: ``` $ ruby auto_shareable_proc.rb 43 (eval at auto_shareable_proc.rb:3): (eval at auto_shareable_proc.rb:3):1: can not access variable 'a' from isolated Proc (SyntaxError) ``` And the same when using `binding` inside the block. Also `SOME_PROC.binding` doesn't work for a shareable Proc. ---------------------------------------- Feature #21767: Consider procs which `self` is Ractor-shareable as Ractor shareable https://bugs.ruby-lang.org/issues/21767#change-115485 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- I would like to allow procs which `self` is Ractor-shareable to be automatically eligible as shareable, without an explicit `Ractor.make_shareable` call. ```ruby class C PROC = proc { p ARRAY }.freeze end Ractor.new { C::PROC.call }.value # Allow this, since `C` is shareable ``` Proposal is: Consider procs/lambdas which meet the following condition as Ractor-shareable. - The proc is frozen. - The proc's `self` is shareable. This proposal is has taken inspiration from #21033 . ## Usecase The main usecase in mind is procs/lambdas in class-level constants. Some libraries store procs in constants as a convenient place for library-wide logic. Those procs usually do not access unshareable state, thus conceptually safe to be shared across Ractors. However, the current limitation completely blocks this. Examples may be found in ruby/ruby, and I have submitted a pull request to migrate one to `Ractor.shareable_proc`. ```ruby class Pathname SAME_PATHS = if File::FNM_SYSCASE.nonzero? # Avoid #zero? here because #casecmp can return nil. proc {|a, b| a.casecmp(b) == 0} else proc {|a, b| a == b} end end ``` https://github.com/search?q=repo%3Aruby%2Fruby%20%2F(%3F-i)%5BA-Z%5D%20%3D%20(proc%7Clambda)%2F&type=code More examples can be found in public code. https://github.com/search?q=language%3Aruby+%2F%28%3F-i%29%5BA-Z%5D+%3D+%28proc%7Clambda%29%2F&type=code It can be observed that a good portion of these do not access unshareable state. Appending `.freeze` would be much more acceptable than redefining using `Ractor.shareable_proc`, which is a Ruby 4.0-only feature. ## Discussion: Change of behavior when illegal access occurs in proc Consider this code. The proc accesses non-frozen `C::ARRAY`, which is against the rules of Ractors regardless of this patch. ```ruby class C ARRAY = [] PROC = proc { p ARRAY } end # Still illegal since C::ARRAY is not shareable Ractor.new { C::PROC.call }.value ``` This code used to raise on `C::PROC.call` (`can not access non-shareable objects in constant C::PROC by non-main Ractor.`). When this patch is applied, it will raise on `p ARRAY` (`can not access non-shareable objects in constant C::ARRAY by non-main ractor.`). This could be debatable change. If this is not acceptable, I'd like to revisit #21033 . -- https://bugs.ruby-lang.org/
Issue #21767 has been updated by osyoyu (Daisuke Aritomo). @eregon I'm not sure I fully understand your point, but conceptually your code should raise an `Ractor::IsolationError` by accessing a unshareable, mutable variable `a`. My proposal's intent was to preserve `self` while making the proc shareable. This is unlike `Ractor.shareable_proc(pr)`, which swaps the self with `nil`. (Could it be the case that that behavior is hard to implement, since Ractors check only the shareablity of the _value_, not the _variable_?) I'd also like to hear from @ko1. ---------------------------------------- Feature #21767: Consider procs which `self` is Ractor-shareable as Ractor shareable https://bugs.ruby-lang.org/issues/21767#change-115822 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- I would like to allow procs which `self` is Ractor-shareable to be automatically eligible as shareable, without an explicit `Ractor.make_shareable` call. ```ruby class C PROC = proc { p ARRAY }.freeze end Ractor.new { C::PROC.call }.value # Allow this, since `C` is shareable ``` Proposal is: Consider procs/lambdas which meet the following condition as Ractor-shareable. - The proc is frozen. - The proc's `self` is shareable. This proposal is has taken inspiration from #21033 . ## Usecase The main usecase in mind is procs/lambdas in class-level constants. Some libraries store procs in constants as a convenient place for library-wide logic. Those procs usually do not access unshareable state, thus conceptually safe to be shared across Ractors. However, the current limitation completely blocks this. Examples may be found in ruby/ruby, and I have submitted a pull request to migrate one to `Ractor.shareable_proc`. ```ruby class Pathname SAME_PATHS = if File::FNM_SYSCASE.nonzero? # Avoid #zero? here because #casecmp can return nil. proc {|a, b| a.casecmp(b) == 0} else proc {|a, b| a == b} end end ``` https://github.com/search?q=repo%3Aruby%2Fruby%20%2F(%3F-i)%5BA-Z%5D%20%3D%20(proc%7Clambda)%2F&type=code More examples can be found in public code. https://github.com/search?q=language%3Aruby+%2F%28%3F-i%29%5BA-Z%5D+%3D+%28proc%7Clambda%29%2F&type=code It can be observed that a good portion of these do not access unshareable state. Appending `.freeze` would be much more acceptable than redefining using `Ractor.shareable_proc`, which is a Ruby 4.0-only feature. ## Discussion: Change of behavior when illegal access occurs in proc Consider this code. The proc accesses non-frozen `C::ARRAY`, which is against the rules of Ractors regardless of this patch. ```ruby class C ARRAY = [] PROC = proc { p ARRAY } end # Still illegal since C::ARRAY is not shareable Ractor.new { C::PROC.call }.value ``` This code used to raise on `C::PROC.call` (`can not access non-shareable objects in constant C::PROC by non-main Ractor.`). When this patch is applied, it will raise on `p ARRAY` (`can not access non-shareable objects in constant C::ARRAY by non-main ractor.`). This could be debatable change. If this is not acceptable, I'd like to revisit #21033 . -- https://bugs.ruby-lang.org/
Issue #21767 has been updated by ko1 (Koichi Sasada). I think a number of "frozen Proc" is enough small so the incompatibility is enough small to ignore. So I'm +1. ---------------------------------------- Feature #21767: Consider procs which `self` is Ractor-shareable as Ractor shareable https://bugs.ruby-lang.org/issues/21767#change-115838 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- I would like to allow procs which `self` is Ractor-shareable to be automatically eligible as shareable, without an explicit `Ractor.make_shareable` call. ```ruby class C PROC = proc { p ARRAY }.freeze end Ractor.new { C::PROC.call }.value # Allow this, since `C` is shareable ``` Proposal is: Consider procs/lambdas which meet the following condition as Ractor-shareable. - The proc is frozen. - The proc's `self` is shareable. This proposal is has taken inspiration from #21033 . ## Usecase The main usecase in mind is procs/lambdas in class-level constants. Some libraries store procs in constants as a convenient place for library-wide logic. Those procs usually do not access unshareable state, thus conceptually safe to be shared across Ractors. However, the current limitation completely blocks this. Examples may be found in ruby/ruby, and I have submitted a pull request to migrate one to `Ractor.shareable_proc`. ```ruby class Pathname SAME_PATHS = if File::FNM_SYSCASE.nonzero? # Avoid #zero? here because #casecmp can return nil. proc {|a, b| a.casecmp(b) == 0} else proc {|a, b| a == b} end end ``` https://github.com/search?q=repo%3Aruby%2Fruby%20%2F(%3F-i)%5BA-Z%5D%20%3D%20(proc%7Clambda)%2F&type=code More examples can be found in public code. https://github.com/search?q=language%3Aruby+%2F%28%3F-i%29%5BA-Z%5D+%3D+%28proc%7Clambda%29%2F&type=code It can be observed that a good portion of these do not access unshareable state. Appending `.freeze` would be much more acceptable than redefining using `Ractor.shareable_proc`, which is a Ruby 4.0-only feature. ## Discussion: Change of behavior when illegal access occurs in proc Consider this code. The proc accesses non-frozen `C::ARRAY`, which is against the rules of Ractors regardless of this patch. ```ruby class C ARRAY = [] PROC = proc { p ARRAY } end # Still illegal since C::ARRAY is not shareable Ractor.new { C::PROC.call }.value ``` This code used to raise on `C::PROC.call` (`can not access non-shareable objects in constant C::PROC by non-main Ractor.`). When this patch is applied, it will raise on `p ARRAY` (`can not access non-shareable objects in constant C::ARRAY by non-main ractor.`). This could be debatable change. If this is not acceptable, I'd like to revisit #21033 . -- https://bugs.ruby-lang.org/
Issue #21767 has been updated by Eregon (Benoit Daloze). osyoyu (Daisuke Aritomo) wrote in #note-3:
@eregon I'm not sure I fully understand your point, but conceptually your code should raise an `Ractor::IsolationError` by accessing a unshareable, mutable variable `a`.
You can try the example code above, it runs on master. No, it should not raise anything and return 43 for `p Foo::SOME_PROC.call`, because that code is not using Ractor at all. The `SOME_PROC_SHAREABLE` is only to illustrate what would happen with your proposal. With your proposal, `SOME_PROC` would be the same as `SOME_PROC_SHAREABLE`. And so instead of printing 43 it would error, even on the main Ractor, which is incompatible. It would also break `Kernel#binding` inside the block, and `SOME_PROC.binding` wouldn't work anymore. i.e. a number of "seemingly randomly chosen" blocks would stop working as they always did, even if Ractor is not used at all! I think it's just too incompatible, and the advantage is too small because most procs don't have a shareable self anyway.
My proposal's intent was to preserve `self` while making the proc shareable. This is unlike `Ractor.shareable_proc(pr)`, which swaps the self with `nil`.
That doesn't change anything for this example, you can change it to `Ractor.shareable_proc(self: self, &SOME_PROC)` it will be the same. ---------------------------------------- Feature #21767: Consider procs which `self` is Ractor-shareable as Ractor shareable https://bugs.ruby-lang.org/issues/21767#change-115840 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- I would like to allow procs which `self` is Ractor-shareable to be automatically eligible as shareable, without an explicit `Ractor.make_shareable` call. ```ruby class C PROC = proc { p ARRAY }.freeze end Ractor.new { C::PROC.call }.value # Allow this, since `C` is shareable ``` Proposal is: Consider procs/lambdas which meet the following condition as Ractor-shareable. - The proc is frozen. - The proc's `self` is shareable. This proposal is has taken inspiration from #21033 . ## Usecase The main usecase in mind is procs/lambdas in class-level constants. Some libraries store procs in constants as a convenient place for library-wide logic. Those procs usually do not access unshareable state, thus conceptually safe to be shared across Ractors. However, the current limitation completely blocks this. Examples may be found in ruby/ruby, and I have submitted a pull request to migrate one to `Ractor.shareable_proc`. ```ruby class Pathname SAME_PATHS = if File::FNM_SYSCASE.nonzero? # Avoid #zero? here because #casecmp can return nil. proc {|a, b| a.casecmp(b) == 0} else proc {|a, b| a == b} end end ``` https://github.com/search?q=repo%3Aruby%2Fruby%20%2F(%3F-i)%5BA-Z%5D%20%3D%20(proc%7Clambda)%2F&type=code More examples can be found in public code. https://github.com/search?q=language%3Aruby+%2F%28%3F-i%29%5BA-Z%5D+%3D+%28proc%7Clambda%29%2F&type=code It can be observed that a good portion of these do not access unshareable state. Appending `.freeze` would be much more acceptable than redefining using `Ractor.shareable_proc`, which is a Ruby 4.0-only feature. ## Discussion: Change of behavior when illegal access occurs in proc Consider this code. The proc accesses non-frozen `C::ARRAY`, which is against the rules of Ractors regardless of this patch. ```ruby class C ARRAY = [] PROC = proc { p ARRAY } end # Still illegal since C::ARRAY is not shareable Ractor.new { C::PROC.call }.value ``` This code used to raise on `C::PROC.call` (`can not access non-shareable objects in constant C::PROC by non-main Ractor.`). When this patch is applied, it will raise on `p ARRAY` (`can not access non-shareable objects in constant C::ARRAY by non-main ractor.`). This could be debatable change. If this is not acceptable, I'd like to revisit #21033 . -- https://bugs.ruby-lang.org/
participants (3)
-
Eregon (Benoit Daloze) -
ko1 (Koichi Sasada) -
osyoyu (Daisuke Aritomo)