Issue #22097 has been updated by headius (Charles Nutter). Thank you for considering JRuby! I will review your PR and also start a ruby-4.1 branch you can target. ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-117509 * Author: shugo (Shugo Maeda) * Status: Open ---------------------------------------- ## Abstract I propose `Proc#with_refinements` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Limitation Similar to `Proc#binding`, `Proc#with_refinements` raises ArgumentError if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/shugo/jruby/pull/2 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/

Issue #22097 has been updated by shugo (Shugo Maeda). For maintainability, I've replaced the hand-written iseq deep-copy with an in-memory IBF dump+load round-trip in: https://github.com/ruby/ruby/pull/17248/changes/f27cf1d98c18f4137ace0243ca69... ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-117560 * Author: shugo (Shugo Maeda) * Status: Open ---------------------------------------- ## Abstract I propose `Proc#with_refinements(mod, ...)` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` When no argument is given, `ArgumentError` is raised. When a non-`Module` argument is given, `TypeError` is raised. ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Limitations * Similar to `Proc#binding`, `Proc#with_refinements` raises `ArgumentError` if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` * Chained application of `Proc#with_refinements` is not allowed. `ArgumentError` is raised if the receiver is a `Proc` returned by `Proc#with_refinements`. ```ruby refined = prc.with_refinements(StringExt) refined.with_refinements(IntegerExt) #=> ArgumentError ``` * `define_method` (and `define_singleton_method`) rejects a `Proc` with refinements. `ArgumentError` is raised if the return value of `Proc#with_refinements` is given to `define_method`. ```ruby refined = prc.with_refinements(StringExt) define_method(:foo, &refined) #=> ArgumentError ``` * `Proc#with_refinements` can only be called from the main Ractor. `Ractor::IsolationError` is raised when called from a non-main Ractor. ```ruby Ractor.new { prc.with_refinements(StringExt) }.value #=> Ractor::IsolationError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/jruby/jruby/pull/9486 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/

Issue #22097 has been updated by shugo (Shugo Maeda). Thank you for the feedback! Eregon (Benoit Daloze) wrote in #note-8:

Since the performance relies on having `with_refinements` called always with the same Refinement module for a given block, how about raising an exception if it doesn't hold?

I would prefer not to. The memo is just a cache, and this restriction would make the cache observable: whether `prc.with_refinements(B)` succeeds would depend on whether some other code called it with `A` before. For example, two libraries applying different refinements to Procs created from the same block would conflict, and the failure would depend on call order. That seems harder to debug than the performance issue it prevents. Instead, how about emitting a performance warning (`Warning[:performance]`, like the object shapes warnings) when `with_refinements` discards the cached copy because it was called with different modules for the same block? That makes the performance issue visible without changing the semantics.

Semantically, nested blocks also get access to the refinements, as shown in `test_with_refinements_nested_block`, or for clarity:

Yes, nested blocks also see the refinements. This is intended behavior. The refinements also apply to methods defined with `def` inside the body. I have documented this in the RDoc in https://github.com/ruby/ruby/pull/17248/changes/5c84f091bb3f01a646554b368c62...

Copying a block IR's, *and the IR of all nested blocks* (IR = bytecode for CRuby) is quite expensive. It's cached but it's still going to be a significant cost on either application startup/on the first request/etc. It would be good to get some numbers on that, e.g. creating and calling N blocks vs the same but also using `with_refinements`.

That makes sense. Here are some numbers, with 10,000 distinct blocks of a realistic size (about 15 lines, 2 nested blocks, so each copy duplicates 3 iseqs): ``` call 10000 original blocks 27.2 ms (2.72 us/block) with_refinements x10000 (first time: copy) 261.8 ms (26.18 us/block) call 10000 refined blocks 27.6 ms (2.76 us/block) with_refinements x10000 (memoized) 3.2 ms (0.32 us/block) with_refinements x10000, alternating A/B 251.8 ms (25.18 us/block) iseq tree size: original 4040 bytes, copy 3992 bytes ``` So the copy costs about 26 us and 4 KB per block per module set, and it happens only once thanks to the memoization. Even 10,000 refined blocks add only ~0.3 seconds to startup. Call speed is the same as the original. The full script is at: https://gist.github.com/shugo/07e62c44bc4765ecff6d2b8e704b5f38

The increased memory footprint would be worth documenting.

I have documented the memory footprint in the RDoc in https://github.com/ruby/ruby/pull/17248/changes/5c84f091bb3f01a646554b368c62...

To make the semantics cleaner, maybe we should prevent the original block to be called (i.e. raise an exception if it's called) once `with_refinements` has been called on it?

In the intended use cases, only the refined Proc is called. But I would prefer not to enforce it, for two reasons: * Calling both is well-defined: each Proc behaves consistently, and their inline caches are isolated. There is no "mixed" state. * Storing a flag in the block would mutate state shared by all Proc instances of that block. Calling `original.call` would suddenly raise because some other code called `with_refinements` on a sibling Proc. This is the same action-at-a-distance problem that `Proc#using` had, which this proposal was redesigned to avoid. Note that a block can already behave differently depending on how it is invoked (`instance_exec` changes `self`, instance variables, and method resolution). `with_refinements` is similar: an explicit, opt-in re-binding of the resolution context, and the new Proc object makes the boundary visible.

One way to fully address that would be to make this lexical, like: (snip) But I guess the use case here wants more flexibility?

As you guessed, the main use case is the opposite direction: a library (e.g. a DSL) applies *its own* refinements to blocks written by *its users*, so that users do not need to write `using` or name the modules. A lexical form cannot express this, because the modules are chosen by the library that receives the block, not by the author of the block. For the case where the author of the block names the modules, a convenience method close to your `proc_using_refinements` can be built on top of the primitive: ```ruby module Kernel private def using_refinements(*modules, &block) block.with_refinements(*modules).call end end using_refinements(StringExt) { "hi".shout } #=> "HI!" ``` I think adding such a helper is fine (the name is open to discussion). But I want `Proc#with_refinements` as the primitive, because the case where a library receives a block cannot be expressed lexically. ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-117587 * Author: shugo (Shugo Maeda) * Status: Open ---------------------------------------- ## Abstract I propose `Proc#with_refinements(mod, ...)` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` When no argument is given, `ArgumentError` is raised. When a non-`Module` argument is given, `TypeError` is raised. ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Limitations * Similar to `Proc#binding`, `Proc#with_refinements` raises `ArgumentError` if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` * Chained application of `Proc#with_refinements` is not allowed. `ArgumentError` is raised if the receiver is a `Proc` returned by `Proc#with_refinements`. ```ruby refined = prc.with_refinements(StringExt) refined.with_refinements(IntegerExt) #=> ArgumentError ``` * `define_method` (and `define_singleton_method`) rejects a `Proc` with refinements. `ArgumentError` is raised if the return value of `Proc#with_refinements` is given to `define_method`. ```ruby refined = prc.with_refinements(StringExt) define_method(:foo, &refined) #=> ArgumentError ``` * `Proc#with_refinements` can only be called from the main Ractor. `Ractor::IsolationError` is raised when called from a non-main Ractor. ```ruby Ractor.new { prc.with_refinements(StringExt) }.value #=> Ractor::IsolationError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/jruby/jruby/pull/9486 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/

Issue #22097 has been updated by Eregon (Benoit Daloze). Thank you for your replies, they address most of my concerns and show this was well though out. Yes, I think a performance warning is fine and important to add, nothing would be a silent performance trap. shugo (Shugo Maeda) wrote in #note-9:

The refinements also apply to methods defined with `def` inside the body

`def` is usually a lexical boundary but not for refinements (i.e., `using`), so I guess it makes some sense. I'm not sure it's really needed though and might be surprising, is it needed? I think for examples like ```ruby proc { Foo.class_eval do def bar "hi".shout end end Foo.new.bar }.with_refinements(StringRefinement) ``` it's quite strange and seem safer to not support that. This seems confusing, it looks like mixing `refine` and `using` together, even though it's only `using` and so defining a global method with some refinements applied. `module`/`class` would still be boundaries, right? Then the above would work but not this? ```ruby proc { class Foo def bar "hi".shout end end Foo.new.bar }.with_refinements(StringRefinement) ``` OTOH, I suppose `Proc#with_refinements` is basically doing the same as if the proc had `using M` on the first line, and the refinements wouldn't apply past the end of the proc. But I think going through `def` or `module/class` is the action-at-a-distance problem again, e.g. if the user defines some method inside a block (which at the place the block is declared is not clear at all it has refinements) then it unexpectedly get refinements too. I think we should limit it no nested blocks, those already inherit the local variables so it's natural. `define_method` would also not use those refinements. ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-117721 * Author: shugo (Shugo Maeda) * Status: Open ---------------------------------------- ## Abstract I propose `Proc#with_refinements(mod, ...)` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` When no argument is given, `ArgumentError` is raised. When a non-`Module` argument is given, `TypeError` is raised. ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Use Cases `Proc#with_refinements` is useful to implement internal DSLs applying refinements implicitly. * [activerecord-refinements](https://github.com/shugo/activerecord-refinements/blob/8869717fcba96fae3965d...) * DSL for SQL * Example: `User.where { :name == 'matz' }` * [radd_djur](https://github.com/shugo/radd_djur/blob/proc-with_refinements/lib/radd_djur/...) * DSL for monadic parser combinators ## Limitations * Similar to `Proc#binding`, `Proc#with_refinements` raises `ArgumentError` if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` * Chained application of `Proc#with_refinements` is not allowed. `ArgumentError` is raised if the receiver is a `Proc` returned by `Proc#with_refinements`. ```ruby refined = prc.with_refinements(StringExt) refined.with_refinements(IntegerExt) #=> ArgumentError ``` * `define_method` (and `define_singleton_method`) rejects a `Proc` with refinements. `ArgumentError` is raised if the return value of `Proc#with_refinements` is given to `define_method`. ```ruby refined = prc.with_refinements(StringExt) define_method(:foo, &refined) #=> ArgumentError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/jruby/jruby/pull/9486 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/

Issue #22097 has been updated by shugo (Shugo Maeda). At the dev meeting Matz said he did not like the name and proposed `Proc#using`. I previously proposed `Proc#using` as the name for the destructive version, so it felt a bit odd at first, but the name `using` itself does not imply destructive behavior, so I agree with it. ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-117970 * Author: shugo (Shugo Maeda) * Status: Open ---------------------------------------- ## Abstract I propose `Proc#with_refinements(mod, ...)` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` When no argument is given, `ArgumentError` is raised. When a non-`Module` argument is given, `TypeError` is raised. ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Use Cases `Proc#with_refinements` is useful to implement internal DSLs applying refinements implicitly. * [activerecord-refinements](https://github.com/shugo/activerecord-refinements/blob/8869717fcba96fae3965d...) * DSL for SQL * Example: `User.where { :name == 'matz' }` * [radd_djur](https://github.com/shugo/radd_djur/blob/proc-with_refinements/lib/radd_djur/...) * DSL for monadic parser combinators ## Limitations * Similar to `Proc#binding`, `Proc#with_refinements` raises `ArgumentError` if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` * Chained application of `Proc#with_refinements` is not allowed. `ArgumentError` is raised if the receiver is a `Proc` returned by `Proc#with_refinements`. ```ruby refined = prc.with_refinements(StringExt) refined.with_refinements(IntegerExt) #=> ArgumentError ``` * `define_method` (and `define_singleton_method`) rejects a `Proc` with refinements. `ArgumentError` is raised if the return value of `Proc#with_refinements` is given to `define_method`. ```ruby refined = prc.with_refinements(StringExt) define_method(:foo, &refined) #=> ArgumentError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/jruby/jruby/pull/9486 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/

Issue #22097 has been updated by shugo (Shugo Maeda). Status changed from Open to Assigned Assignee set to shugo (Shugo Maeda) matz (Yukihiro Matsumoto) wrote in #note-19:

I suggest `Proc#refined(mod, ...)`. It is short, and the past participle cannot be read as a command: `original.refined(StringExt)` can only mean "a refined version of the original", so the copying semantics are built into the word form itself. `Enumerable#lazy` is a precedent for an adjective-like name returning a derived object.

@shugo, if `refined` sounds acceptable to you, let's go with it.

Thank you, I agree. I have renamed the method in both implementations: * CRuby: https://github.com/ruby/ruby/pull/17248 * JRuby: https://github.com/jruby/jruby/pull/9486 While updating the implementations, I also made one semantic change that I would like to note here: calling `using` inside the body of a refined Proc now raises RuntimeError. This applies to the proc body itself, to nested blocks, and to class/module bodies opened inside it, including when the proc is passed to `module_eval`. The reason is that the refinement set of a refined Proc is fixed when `refined` is called. Internally, procs created from the same block with the same modules share a single copied instruction sequence and its method-resolution caches, and that sharing is only sound while all of them run under exactly the same refinement set. A `using` in mid-body would change the set for one proc at runtime and leak the newly activated refinements to its siblings through the shared caches. Pass all modules in a single `refined()` call instead. `Module#refine` itself (which only defines refinements without activating them) remains allowed inside the body. Two smaller updates since #note-16: * `Proc#refined` can be called on any Ractor, and the memoized copy is now Ractor-shareable (the refinement table is frozen at creation), so the cache also works across Ractors. * Calling `refined` on the same block with different module sets evicts the single-entry cache; this now emits a `:performance`-category warning (disabled by default, like other performance warnings). If there are no further concerns, I will proceed with `Proc#refined`. ---------------------------------------- Feature #22097: Add Proc#with_refinements https://bugs.ruby-lang.org/issues/22097#change-118041 * Author: shugo (Shugo Maeda) * Status: Assigned * Assignee: shugo (Shugo Maeda) ---------------------------------------- ## Abstract I propose `Proc#with_refinements(mod, ...)` to support block-level refinements. ```ruby module StringExt refine String do def shout = upcase + "!" end end original = ->(s) { s.shout } refined = original.with_refinements(StringExt) p refined.call("hello") # "HELLO!" p original.call("hello") # NoMethodError ``` When no argument is given, `ArgumentError` is raised. When a non-`Module` argument is given, `TypeError` is raised. ## Background and Motivation I previously proposed `Proc#using` in [Feature #16461], but it introduced semantic complexities because it mutated existing blocks. Instead of mutating the existing block, `Proc#with_refinements` returns a new `Proc` object with its own isolated call sites. This approach makes its semantics much simpler than `Proc#using`, and it avoids thread-safety issues and plays nicely with inline caches. ## Use Cases `Proc#with_refinements` is useful to implement internal DSLs applying refinements implicitly. * [activerecord-refinements](https://github.com/shugo/activerecord-refinements/blob/8869717fcba96fae3965d...) * DSL for SQL * Example: `User.where { :name == 'matz' }` * [radd_djur](https://github.com/shugo/radd_djur/blob/proc-with_refinements/lib/radd_djur/...) * DSL for monadic parser combinators ## Limitations * Similar to `Proc#binding`, `Proc#with_refinements` raises `ArgumentError` if the receiver is not created from a Ruby block. ```ruby :to_s.to_proc.with_refinements(StringExt) #=> ArgumentError ``` * Chained application of `Proc#with_refinements` is not allowed. `ArgumentError` is raised if the receiver is a `Proc` returned by `Proc#with_refinements`. ```ruby refined = prc.with_refinements(StringExt) refined.with_refinements(IntegerExt) #=> ArgumentError ``` * `define_method` (and `define_singleton_method`) rejects a `Proc` with refinements. `ArgumentError` is raised if the return value of `Proc#with_refinements` is given to `define_method`. ```ruby refined = prc.with_refinements(StringExt) define_method(:foo, &refined) #=> ArgumentError ``` ## Implementation I've opened a pull request: https://github.com/ruby/ruby/pull/17248 A PoC for JRuby is also available at: https://github.com/jruby/jruby/pull/9486 ### Data structure changes * Added a bit field `has_refinements` to `rb_proc_t`. * Added a hidden instance variable to `Proc` to store a `cref` with the applied refinements. * Added a single-entry cache `refinement_memo` to `rb_iseq_constant_body`. ### Deep copy of iseq and caching `Proc#with_refinements` performs a deep copy of the receiver's `iseq` to isolate its call sites from the original `Proc`. While a deep copy can be an expensive operation, the single-entry cache in `rb_iseq_constant_body` mitigates this overhead effectively for most practical use cases where the same refinements are applied repeatedly. ### Overhead for code not using Proc#with_refinements * Memory footprint: Neither internal structure grows in size. `has_refinements` is a 1-bit field added to rb_proc_t's existing bit field, and `refinement_memo` shares a union with `mandatory_only_iseq` in rb_iseq_constant_body. * Execution speed: The common `Proc#call` path is kept frameless and only adds a single `has_refinements` bit check. * GC: The mark/free/memsize functions add a single branch per `iseq` to select the union member. Benchmark results: https://gist.github.com/shugo/ddfe92f28ea31e6527a2f270e6daee7c Here's an excerpt from the results, where `compare-ruby` is master and `built-ruby` is the branch for this feature (focusing on Proc/Block operations): | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_proc | 47.215M| 46.149M| | | 1.02x| -| |vm_yield | 1.649| 1.754| | | -| 1.06x| -- https://bugs.ruby-lang.org/