January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116037] [Ruby master Bug#20153] Backport 7f9c174102 to fix --yjit-stats with RubyVM::YJIT.enable
by k0kubun (Takashi Kokubun) 20 Mar '24

20 Mar '24

Issue #20153 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #20153: Backport 7f9c174102 to fix --yjit-stats with RubyVM::YJIT.enable https://bugs.ruby-lang.org/issues/20153 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Ruby 3.3.0 ignored --yjit-stats when `RubyVM::YJIT.enable` (no argument) is used, which was an unintended behavior. https://github.com/ruby/ruby/pull/9415 should be backported to ruby_3_3. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116454] [Ruby master Bug#20214] Backport https://github.com/ruby/ruby/pull/9711 to fix exits on Ruby 3.3's new instruction
by k0kubun (Takashi Kokubun) 20 Mar '24

20 Mar '24

Issue #20214 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #20214: Backport https://github.com/ruby/ruby/pull/9711 to fix exits on Ruby 3.3's new instruction https://bugs.ruby-lang.org/issues/20214 * Author: k0kubun (Takashi Kokubun) * Status: Closed * Priority: Normal * Assignee: naruse (Yui NARUSE) * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Ruby 3.3.0 YJIT missed the support for the instruction that was added shortly before the 3.3.0 release. It's used in Rails, and we didn't mean to exit on such method calls. It'd be nice if we can fix the issue in Ruby 3.3.1 by backporting https://github.com/ruby/ruby/pull/9711. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114309] [Ruby master Feature#19787] Add Enumerable#uniq_map, Enumerable::Lazy#uniq_map, Array#uniq_map and Array#uniq_map!
by joshuay03 (Joshua Young) 16 Mar '24

16 Mar '24

Issue #19787 has been reported by joshuay03 (Joshua Young). ---------------------------------------- Feature #19787: Add Enumerable#uniq_map, Enumerable::Lazy#uniq_map, Array#uniq_map and Array#uniq_map! https://bugs.ruby-lang.org/issues/19787 * Author: joshuay03 (Joshua Young) * Status: Open * Priority: Normal ---------------------------------------- I would like to propose a collection of new methods, `Enumerable#uniq_map`, `Enumerable::Lazy#uniq_map`, `Array#uniq_map` and `Array#uniq_map!`. TL;DR: It's a drop in replacement for `.map { ... }.uniq`, with better performance. I've quite often had to map over an array and get its unique elements. It occurred to me when doing so recently that Ruby doesn't have a short form method for doing that, similar to how `flat_map { ... }` replaces `.map { ... }.flatten` and `filter_map { ... }` replaces `.map { ... }.compact` (with minor differences). I think these new methods could be beneficial both in terms of better performance and writing more succinct code. I have already got a draft PR up with some initial benchmarks in the description: https://github.com/ruby/ruby/pull/8140. -- https://bugs.ruby-lang.org/

4 6

[ruby-core:116491] [Ruby master Bug#20225] Inconsistent behavior of regex matching for a regex has a null loop
by make_now_just (Hiroya Fujinami) 15 Mar '24

15 Mar '24

Issue #20225 has been reported by make_now_just (Hiroya Fujinami). ---------------------------------------- Bug #20225: Inconsistent behavior of regex matching for a regex has a null loop https://bugs.ruby-lang.org/issues/20225 * Author: make_now_just (Hiroya Fujinami) * Status: Open * Priority: Normal * Assignee: make_now_just (Hiroya Fujinami) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Usually, in Ruby (Onigmo), when a null loop (a loop consuming no characters) occurs on regex matching, this loop is terminated. But, if a loop has a capture and some complex condition is satisfied, this causes backtracking. This behavior invokes unexpected results, for example, ```ruby p /(?:.B.(?<a>(?:[C-Z]|.)*)+){2}/ =~ "ABCABC" # => nil p /(?:.B.(?:(?:[C-Z]|.)*)+){2}/ =~ "ABCABC" # => 0 ``` Because the above regex has a capture and the below does not, different matching results are returned. It is not very intuitive that the presence of a capture changes the matching result. The detailed condition for changing the null-loop behavior is 1) a previous capture in this loop holds the empty string, and 2) this capture's position is different from the current matching position. This condition is checked in `STACK_NULL_CHECK_MEMST` (https://github.com/ruby/ruby/blob/bbb7ab906ec64b963bd4b5d37e47b14796d64371/…) Perhaps, you cannot understand what this condition means. Don't worry, I also cannot understand. This condition has been introduced for at least 20 years, and no one may remember the reason for this necessity. (If you know, please tell me!) Even if there is a reason, I believe that there is no reasonable authority for allowing counter-intuitive behavior, such as the above example. This behavior can also cause memoization to be buggy. Memoization relies on the fact that backtracking only depends on positions and states (byte-code offsets of a regex). However, this condition additionally refers to captures, and the memoization is broken. My proposal is to **correct this inconsistent behavior**. Specifically, a null loop should be determined solely on the basis of whether the matching position has changed, without referring to captures. This fix changes the behavior of regex matching, but I believe that the probability that this will actually cause backward compatibility problems is remarkably low. This is because I have never seen any mention of this puzzling behavior before. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:116016] [Ruby master Bug#20150] Memory leak in grapheme clusters
by peterzhu2118 (Peter Zhu) 14 Mar '24

14 Mar '24

Issue #20150 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20150: Memory leak in grapheme clusters https://bugs.ruby-lang.org/issues/20150 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9414 String#grapheme_cluters and String#each_grapheme_cluster leaks memory because if the string is not UTF-8, then the created regex will not be freed. For example: ```ruby str = "hello world".encode(Encoding::UTF_32LE) 10.times do 1_000.times do str.grapheme_clusters end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 26000 42256 59008 75792 92528 109232 125936 142672 159392 176160 ``` After: ``` 9264 9504 9808 10000 10128 10224 10352 10544 10704 10896 ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:115912] [Ruby master Bug#20090] Anonymous arguments are now syntax errors in unambiguous cases
by willcosgrove (Will Cosgrove) 14 Mar '24

14 Mar '24

Issue #20090 has been reported by willcosgrove (Will Cosgrove). ---------------------------------------- Bug #20090: Anonymous arguments are now syntax errors in unambiguous cases https://bugs.ruby-lang.org/issues/20090 * Author: willcosgrove (Will Cosgrove) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- It looks like the changes that were made in #19370 may have gone further than intended. It's also possible I'm misunderstanding what decision was made. But it was my understanding that the goal was to make ambiguous cases a syntax error. The test cases added are all testing the ambiguous cases: ```rb assert_syntax_error("def b(&) ->(&) {c(&)} end", /anonymous block parameter is also used/) # ... assert_syntax_error("def b(*) ->(*) {c(*)} end", /anonymous rest parameter is also used/) assert_syntax_error("def b(a, *) ->(*) {c(1, *)} end", /anonymous rest parameter is also used/) assert_syntax_error("def b(*) ->(a, *) {c(*)} end", /anonymous rest parameter is also used/) # ... assert_syntax_error("def b(**) ->(**) {c(**)} end", /anonymous keyword rest parameter is also used/) assert_syntax_error("def b(k:, **) ->(**) {c(k: 1, **)} end", /anonymous keyword rest parameter is also used/) assert_syntax_error("def b(**) ->(k:, **) {c(**)} end", /anonymous keyword rest parameter is also used/) ``` However it is now also producing syntax errors in all of these cases: ```rb def b(&) -> { c(&) } end def b(*) -> { c(*) } end def b(a, *) -> { c(1, *) } end def b(*) ->(a) { c(a, *) } end def b(**) -> { c(**) } end def b(k:, **) -> { c(k: 1, **) } end def b(**) ->(k:) { c(k:, **) } end ``` Again, it's possible I misunderstood the scope of the previous change. But it would be sad to lose the unambiguous case, as I've used that pattern quite a bit in my own projects. This is my first time opening an issue here, so I apologize in advance if I've done anything non-standard. -- https://bugs.ruby-lang.org/

6 7

[ruby-core:116356] [Ruby master Bug#20198] Threaded DNS resolver does not propagate errno to the calling thread
by kjtsanaktsidis (KJ Tsanaktsidis) 14 Mar '24

14 Mar '24

Issue #20198 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20198: Threaded DNS resolver does not propagate errno to the calling thread https://bugs.ruby-lang.org/issues/20198 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- If we get a return value of `EAI_SYSTEM` from `getaddrinfo`, we transform that into an appropriate `Errno::` exception on the Ruby side. However, because we now run the actual call to `getaddrinfo` in a thread, we lose that `errno` value (because `errno` is thread-local). So, what we actually raise in case of `EAI_SYSTEM` is just the last error which happened on the calling thread - e.g. this `ECHILD` which presumably got set in the bowels of pthreads somewhere: ``` 1) Socket::IPSocket#getaddress raises an error on unknown hostnames ERROR Expected SocketError but got: Errno::ECHILD (No child processes - getaddrinfo) /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:22:in `getaddress' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:22:in `block (3 levels) in <top (required)>' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:21:in `block (2 levels) in <top (required)>' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:4:in `<top (required)>' ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116073] [Ruby master Bug#20161] Memory leak in regexp grapheme clusters
by peterzhu2118 (Peter Zhu) 14 Mar '24

14 Mar '24

Issue #20161 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20161: Memory leak in regexp grapheme clusters https://bugs.ruby-lang.org/issues/20161 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9447 The cc->mbuf gets overwritten, so we need to free it to not leak memory. For example: ```ruby str = "hello world".encode(Encoding::UTF_32LE) 10.times do 1_000.times do str.grapheme_clusters end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 15536 15760 15920 16144 16304 16480 16640 16784 17008 17280 ``` After: ``` 15584 15584 15760 15824 15888 15888 15888 15888 16048 16112 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116489] [Ruby master Bug#20224] Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3
by yui-knk (Kaneko Yuichiro) 14 Mar '24

14 Mar '24

Issue #20224 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Bug #20224: Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3 https://bugs.ruby-lang.org/issues/20224 * Author: yui-knk (Kaneko Yuichiro) * Status: Closed * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- It's nice if we can backport https://github.com/ruby/ruby/pull/9634 (3d19409637de1462b6790d2a92344bf0a10d8c52) to Ruby 3.3 branch. Even so previous codes work as expected but it depends on how parse.c works and misleading. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113966] [Ruby master Bug#19742] Introduce `Module#anonymous?`
by ioquatix (Samuel Williams) 12 Mar '24

12 Mar '24

Issue #19742 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19742: Introduce `Module#anonymous?` https://bugs.ruby-lang.org/issues/19742 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- As a follow-on <from https://bugs.ruby-lang.org/issues/19521>, I'd like propose we introduce `Module#anonymous?`. In some situations, like logging/formatting, serialisation/deserialization, debugging or meta-programming, we might like to know if a class is a proper constant or not. However, this brings about some other issues which might need to be discussed. After assigning a constant, then removing it, the internal state of Ruby still believes that the class name is permanent, even thought it's no longer true. e.g. ``` m = Module.new m.anonymous? # true M = m m.anonyomous # false Object.send(:remove_const, :M) M # uninitialized constant M (NameError) M.anonymous? # false ``` Because RCLASS data structure is not updated after the constant is removed, internally the state still has a "permanent class name". I want to use this proposal to discuss this issue and whether there is anything we should do about such behaviour (or even if it's desirable). Proposed PR: https://github.com/ruby/ruby/pull/7966 cc @fxn -- https://bugs.ruby-lang.org/

9 46