February 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112342] [Ruby master Feature#15374] Proposal: Enable refinements to `#method_missing`
by shreeve (Steve Shreeve) 10 Feb '23

10 Feb '23

Issue #15374 has been updated by shreeve (Steve Shreeve). > I don't see any real-world usage of allowing #method_missing refinable. Maybe it can be used only for tricks and obfusticated code. We use this heavily and it would be great if `method_missing` could be refinable. Here's an example: ```ruby class Hash alias_method :default_lookup, :[] def [](key, miss=nil) key?(key) and return default_lookup(key) || miss ary = key.to_s.split(/(?:[.\/\[]|\][.\/]?)/) val = ary.inject(self) do |obj, sub| if obj == self then default_lookup(sub.to_sym) elsif obj == nil then break elsif sub =~ /\A-?\d*\z/ then obj[sub.to_i] else obj[sub.to_sym] end end or miss end def method_missing(name, *args) name =~ /=$/ ? send(:[]=, $`.to_sym, *args) : send(:[], name, *args) end end book = { name: "Ruby Object Model", url: [ "https://google.com", "https://pepsi.com", "https://byu.edu", ], team: { boss: { name: "Mark", age: 23, }, janitor: { name: "Bob", age: 56, kids: %w[ Billy Sue Tim Nebo Dash ], }, }, } p book.name # => "Ruby Object Model" p book.color # => nil p book.color("red") # => "red" p book.url[2] # => "https://byu.edu" p book["team/janitor/age"] # => 56 p book["team.janitor.age"] # => 56 p book["team/janitor/song", "None"] # => "None" p book["team/janitor/kids[3]"] # => "Nebo" ``` This approach has been *extremely* helpful and useful, but I can't get it to work with refinements and I always feel a little dirty with bastardizing the core `Hash` class. Can this code be made to work with refinements? This is a real world case, where we rely on this heavily for an api in production for several years. Refinement support would be great! ---------------------------------------- Feature #15374: Proposal: Enable refinements to `#method_missing` https://bugs.ruby-lang.org/issues/15374#change-101786 * Author: osyo (manga osyo) * Status: Rejected * Priority: Normal ---------------------------------------- Proposal enable refinements to `#method_missing`. It can be used in the following cases. ```ruby # Access key value with method using Module.new { refine Hash do # name is Symbol or String def method_missing(name) self[name.to_sym] || self[name.to_s] end end } hash = { name: "homu", "age" => 14 } pp hash.name # => "homu" pp hash.age # => "age" ``` `method_missing` is hard hacking. I would like to use Refinements with `method_missing`. pull request: https://github.com/ruby/ruby/pull/2036 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112052] [Ruby master Bug#19381] SEGV - ivars, both Ubuntu & Windows
by MSP-Greg (Greg L) 10 Feb '23

10 Feb '23

Issue #19381 has been reported by MSP-Greg (Greg L). ---------------------------------------- Bug #19381: SEGV - ivars, both Ubuntu & Windows https://bugs.ruby-lang.org/issues/19381 * Author: MSP-Greg (Greg L) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-01-26T07:31:08Z master 6422fef90c) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I've used code similar to the below for a long time. When max is set to 50, it runs. When max is set to 51, it SEGV faults. Rubies tested: ruby 3.3.0dev (2023-01-26T07:31:08Z master 6422fef90c) [x86_64-linux] ruby 3.3.0dev (2023-01-26T07:31:08Z master 6422fef90c) [x64-mingw-ucrt] ruby 3.3.0dev (2023-01-26T07:31:08Z master 6422fef90c) [x64-mswin64_140] I suspect it involves the changes in https://github.com/ruby/ruby/pull/7183 'Limit maximum number of IVs on a shape' ```ruby module Test class << self def run max = 51 (1..max).each do |v| instance_variable_set("@iv#{v}".to_sym, nil) end end end end Test.run puts Test.instance_variables ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:112318] [Ruby master Bug#11230] Should rb_struct_s_members() be public API?
by mame (Yusuke Endoh) 10 Feb '23

10 Feb '23

Issue #11230 has been updated by mame (Yusuke Endoh). Discussed at the dev meeting. When we considered removing rb_struct_s_members two years ago, we did not find its use in a major gem. Therefore, we agreed to remove it once. However, two years have passed since then, and now we have found that oj gem is using it. ``` $ gem-codesearch rb_struct_s_members | sort ... 2023-02-01 /srv/gems/oj-3.14.1/ext/oj/custom.c: ma = rb_struct_s_members(clas); 2023-02-01 /srv/gems/oj-3.14.1/ext/oj/dump_object.c: VALUE ma = rb_struct_s_members(clas); 2023-02-01 /srv/gems/oj-3.14.1/ext/oj/rails.c: ma = rb_struct_s_members(rb_obj_class(obj)); ``` It's possible to fix oj gem and have a deprecation warning period. However, it would be easier to just keep this C API, unless there is a strong reason why we really have to remove it. ---------------------------------------- Bug #11230: Should rb_struct_s_members() be public API? https://bugs.ruby-lang.org/issues/11230#change-101759 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal * ruby -v: ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux] * Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN ---------------------------------------- rb_struct_s_members() is declared in include/ruby/intern.h. However it seems this is a fairly internal API as it returns an hidden Array. For instance, there was a try to specify the behavior in https://github.com/rubinius/rubinius/commit/2642a30c57973987d2a7b7e85b3ce1e… but that produces only erratic behavior on MRI because that usage is not intended. A valid usage might be: static VALUE get_struct_member(VALUE self, VALUE obj, VALUE i) { return RARRAY_AREF(rb_struct_s_members(obj), NUM2INT(i)); } p get_struct_member(Struct.new(:a, :bb), 1) But do we need such access and is it worth exposing a potentially dangerous function like this? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111604] [Ruby master Misc#19304] Kernel vs Object documentation
by zverok (Victor Shepelev) 10 Feb '23

10 Feb '23

Issue #19304 has been reported by zverok (Victor Shepelev). ---------------------------------------- Misc #19304: Kernel vs Object documentation https://bugs.ruby-lang.org/issues/19304 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- The restating of the problems raised in #19300, from scratch. I believe it is rather a topic of **community significance** and would be happy if it is possible to **discuss on dev-meeting**, even if the outcome would probably "let's change RDoc in this or that way". So, the problem statement: 1. `Kernel` module defines two "types" of methods: private ones, that pretend to be global (like `puts`), but actually available from inside every object; and public ones (like `#class` or `#frozen?`) that are present in every object including `Kernel`. 2. Since the beginning of times, docs provided an illusion that the second type of the methods belongs to `Object` class, which is, in fact, devoid of its own definitions, just includes `Kernel`. This was handled by special RDoc hack (which, basically, forcefully [reattached definition](https://github.com/ruby/rdoc/blob/017bb9fa496ee0e0959facba694a0… of public methods if they are defined in `Kernel`, to `Object`) 3. The RDoc hack was working in C code only, so after introduction of `kernel.rb` the illusion started to crumble: methods like `#tap`, `#then`, `#class` [are now documented](https://docs.ruby-lang.org/en/3.2/Kernel.html) as belonging to `Kernel` (breaking the tradition of public methods being documented in `Object`) 4. This is all inconsistent and confusing, and I believe, adds to friction both for newcomers and curious investigators of the language! Additionally, I believe: 1. That the distinction of "two kinds of methods" is useful (saying from a practical experience with explaining Ruby while mentoring, writing articles, and discussing with colleagues) 2. But, considering everything, I agree with what @Eregon and @nobu say in #19300: pretending that methods belong not to the module they really belong to is not the optimal way to document things! So, the options I see (save for "do nothing and let things sort themselves somehow"): 1. Change Ruby's implementation so public methods would really belong to `Object`. Actually, I don't believe anybody would agree to experiment on this scale, so just listing it here for completeness. 2. Just remove the RDoc hack; all methods would belong to `Kernel`, and maybe some introductory free-form text will instruct that they are different. **Pro:** Easy to do. **Contra:** The `Kernel` docs would be super-confusing. 3. Continue to maintain the hack in RDoc and extend it to `kernel.rb` (so methods like `#clone` and `#frozen?` would be still in `Object`). **Pro:** Relatively easy to do, will maintain structure many people used to. **Contra:** This is still a lie, methods belong to `Kernel`, not `Object`. 4. Adjust RDoc to a) be able to separately list **public** and **private** methods in two different lists, and add intro for `Kernel` explaining how to treat those. **Pro:** Probably the best correspondence to reality? **Contra:** Not sure how easy it is to change RDoc this way; and other renderers (like ruby-doc.com and rubyapi.org) would need to adjust themselves. So far, (4) looks the most reasonable (if (1) is 100% impossible, that is!) -- https://bugs.ruby-lang.org/

6 7

[ruby-core:112316] [Ruby master Feature#17753] Add Module#namespace
by shioyama (Chris Salzberg) 10 Feb '23

10 Feb '23

Issue #17753 has been updated by shioyama (Chris Salzberg). > Regarding your proposal, in English would be `parent_module` right? My interpretation here, but to me "parent module" would signify the "parent thing that is a module" of the current thing (whatever that thing may be), whereas "module parent" would signify the "module's parent", implying the _parent_ could be anything (but kind of implying it is also a module). English-wise as a method name either is possible, they just have slightly different emphasis. I suggested `module_parent` because it exists and is being used for the same thing, so there's a precedent, which might make it easier to agree on. ---------------------------------------- Feature #17753: Add Module#namespace https://bugs.ruby-lang.org/issues/17753#change-101757 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal ---------------------------------------- Given code like this: ```ruby module A module B class C; end class D; end end end ``` We can get from `C` to `B` like `C.outer_scope`, or to `A` like `C.outer_scope.outer_scope`. I want to use this in cases where I don't know the outer scope, but I want to find constants that are "siblings" of a constant. For example, I can do `A::B::C.outer_scope.constants` to find the list of "sibling" constants to `C`. I want to use this feature when walking objects and introspecting. For example: ```ruby ObjectSpace.each_object(Class) do |k| p siblings: k.outer_scope.constants end ``` I've attached a patch that implements this feature, and there is a pull request on GitHub [here](https://github.com/ruby/ruby/pull/4326). ---Files-------------------------------- 0001-Add-Module-outer_scope.patch (5.93 KB) 0001-Add-Module-namespace.patch (5.89 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111842] [Ruby master Feature#19347] Add Dir.fchdir
by jeremyevans0 (Jeremy Evans) 10 Feb '23

10 Feb '23

Issue #19347 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Feature #19347: Add Dir.fchdir https://bugs.ruby-lang.org/issues/19347 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal ---------------------------------------- Recently, I was working on a program that passes file descriptors over UNIX sockets (using `send_io`/`recv_io`). For file/socket/device descriptors, this works fine using `File#reopen`. However, I found that while Ruby supports `Dir#fileno` to return a directory file descriptor, it cannot use that file descriptor when changing directories. I worked around this in my program by writing the directory path over the UNIX socket, but this results in a TOCTOU vulnerability in certain cases. Passing the directory file descriptor would be simpler and avoid the vulnerability. I've submitted a pull request to implement this method: https://github.com/ruby/ruby/pull/7135 -- https://bugs.ruby-lang.org/

4 8

[ruby-core:112097] [Ruby master Feature#19388] Deprecate SecurityError
by jeremyevans0 (Jeremy Evans) 09 Feb '23

09 Feb '23

Issue #19388 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Feature #19388: Deprecate SecurityError https://bugs.ruby-lang.org/issues/19388 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal ---------------------------------------- I think we should deprecate `SecurityError` in Ruby 3.3 and remove it in Ruby 3.4. I think we probably should have deprecated `SecurityError` when `$SAFE` was deprecated, but better late than never. I've submitted a pull request to deprecate it: https://github.com/ruby/ruby/pull/7193 -- https://bugs.ruby-lang.org/

3 2

[ruby-core:112299] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by zverok (Victor Shepelev) 09 Feb '23

09 Feb '23

Issue #18368 has been updated by zverok (Victor Shepelev). Thanks @matz I'll be working on the implementation. ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-101734 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112297] [Ruby master Feature#19061] Proposal: make a concept of "consuming enumerator" explicit
by zverok (Victor Shepelev) 09 Feb '23

09 Feb '23

Issue #19061 has been updated by zverok (Victor Shepelev). @matz Thanks for your answer. I'll gather more evidence/real-life examples and will adjust the proposal. My main concern though was not as much some particular usage but general awareness of the difference between the two types of enumerators. The latest evidence of the fact that it is a problem is bug #19294 in the new feature of Ruby 3.2, where even the core team member implementing new functionality hasn't considered that some enumerators would be "consumed" by the first iteration. I believe it to be a pretty important distinction frequently leading to idiosyncrasies and not just a random feature request. But I need to think about how to communicate my intentions and proposals better. ---------------------------------------- Feature #19061: Proposal: make a concept of "consuming enumerator" explicit https://bugs.ruby-lang.org/issues/19061#change-101732 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- **The problem** Let's imagine this synthetic data: ```ruby lines = [ "--EMAIL--", "From: zverok.offline(a)gmail.com", "To; bugs(a)ruby-lang.org", "Subject: Consuming Enumerators", "", "Here, I am presenting the following proposal.", "Let's talk about consuming enumerators..." ] ``` The logic of parsing it is more or less clear: * skip the first line * take lines until meet empty, to read the header * take the rest of the lines to read the body It can be easily translated into Ruby code, almost literally: ```ruby def parse(enumerator) puts "Testing: #{enumerator.inspect}" enumerator.next p enumerator.take_while { !_1.empty? } p enumerator.to_a end ``` Now, let's try this code with two different enumerators on those lines: ```ruby require 'stringio' enumerator1 = lines.each enumerator2 = StringIO.new(lines.join("\n")).each_line(chomp: true) puts "Array#each" parse(enumerator1) puts puts "StringIO#each_line" parse(enumerator2) ``` Output (as you probably already guessed): ``` Array#each Testing: #<Enumerator: [...]:each> ["--EMAIL--", "From: zverok.offline(a)gmail.com", "To; bugs(a)ruby-lang.org", "Subject: Consuming Enumerators"] ["--EMAIL--", "From: zverok.offline(a)gmail.com", "To; bugs(a)ruby-lang.org", "Subject: Consuming Enumerators", "", "Here, I am presenting the following proposal.", "Let's talk about consuming enumerators..."] StringIO#each_line Testing: #<Enumerator: #<StringIO:0x00005581018c50a0>:each_line(chomp: true)> ["From: zverok.offline(a)gmail.com", "To; bugs(a)ruby-lang.org", "Subject: Consuming Enumerators"] ["Here, I am presenting the following proposal.", "Let's talk about consuming enumerators..."] ``` Only the second enumerator behaves the way we wanted it to. Things to notice here: 1. Both enumerators are of the same class, "just enumerator," but they behave differently: one of them is **consuming** data on each iteration method, the other does not; but there is no programmatic way to tell whether some enumerator instance is consuming 2. There is no easy way to **make a non-consuming enumerator behave in a consuming way**, to open a possibility of a sequence of processing "skip this, take that, take the rest" **Concrete proposal** 1. Introduce an `Enumerator#consuming?` method that will allow telling one of the other (and make core enumerators like `#each_line` properly report they are consuming). 2. Introduce `consuming: true` parameter for `Enumerator.new` so it would be easy for user's code to specify the flag 3. Introduce `Enumerator#consuming` method to produce a consuming enumerator from a non-consuming one: ```ruby # reference implementation is trivial: class Enumerator def consuming source = self Enumerator.new { |y| loop { y << source.next } } end end enumerator3 = lines.each.consuming parse(enumerator3) ``` Output: ``` ["From: zverok.offline(a)gmail.com", "To; bugs(a)ruby-lang.org", "Subject: Consuming Enumerators"] ["Here, I am presenting the following proposal.", "Let's talk about consuming enumerators..."] ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112287] [Ruby master Bug#19425] Merge the internal only "private" GC headers together
by eightbitraptor (Matthew Valentine-House) 09 Feb '23

09 Feb '23

Issue #19425 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Bug #19425: Merge the internal only "private" GC headers together https://bugs.ruby-lang.org/issues/19425 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Github PR: [7273](https://github.com/ruby/ruby/pull/7273) Currently we have 4 headers named `gc.h` * `gc.h` * `internal/gc.h` * `include/ruby/internal/gc.h` * `include/ruby/internal/intern/gc.h` The first two are private and internal to the CRuby codebase, the latter 2 are visible to C extensions. [This PR](https://github.com/ruby/ruby/pull/7273) attempts to merge the two internal headers `gc.h` and internal/gc.h` into a single header `internal/gc.h` in order to simplify the codebase and put all the gc related interface in a single place. Here is the history of the 2 internal only `gc.h` headers as I've been able to understand from the commit logs: * `gc.h` was created as part of the YARV merge in 2006, it contained only some `defines` for debug related macros. * In 2010 some function declarations were added to it to expose some functionality (`rb_objspace_each_objects`) to the `objspace` extension * In May 2011 `internal.h` was created as a place to put shared, but internal only declarations. * In June 2011 GC related functions started being committed to `internal.h` * In 2019 `internal.h` was refactored and `internal/gc.h` was created As of 2023 internally used code continues to be added to both `gc.h` and `internal/gc.h`, which one is chosen in any particular circumstance seems to be whichever header is already being included for legacy reasons in the `.c` file being edited. I'd like to merge these two files together in order to reduce the cognitive overhead of working on the GC. Am I missing something - are there good reasons why these files should remain separate? -- https://bugs.ruby-lang.org/

2 1