November 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:120006] [Ruby master Bug#20909] Backport String indexing bug fixes
by nobu (Nobuyoshi Nakada) 14 Jan '25

14 Jan '25

Issue #20909 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20909: Backport String indexing bug fixes https://bugs.ruby-lang.org/issues/20909 * Author: nobu (Nobuyoshi Nakada) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- commit:02b70256b5171d4b85ea7eeab836d3d7cfb3dbfc commit:6b4f8945d600168bf530d21395da8293fbd5e8ba -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119751] [Ruby master Bug#20871] Including methods in Enumerable doesn't make them available in Array
by sanderd17 (Sander Deryckere) 14 Jan '25

14 Jan '25

Issue #20871 has been reported by sanderd17 (Sander Deryckere). ---------------------------------------- Bug #20871: Including methods in Enumerable doesn't make them available in Array https://bugs.ruby-lang.org/issues/20871 * Author: sanderd17 (Sander Deryckere) * Status: Open * ruby -v: 3.3.6 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Today, our CI pipeline started failing after the automatic update from v3.3.5 to v3.3.6. After researching, it turned out that our core extensions to the `Array` class weren't loaded anymore. The core-extensions code looks like this: ```ruby module CoreExt module Enumerable def average sum(&:to_f) / count if any? end # def ... end end Enumerable.include CoreExt::Enumerable ``` After some debugging, it turned out that the `average` method was included in the `instance_methods` of `Enumerable`, but not in the `instance_methods` of `Array`. Explicitly adding `Array.include CoreExt::Enumerable` fixes CI for our case. The very strange thing is that it only happens on a release branch we are still maintaining. It doesn't happen on our main development branch (which also updated to v3.3.6 today). I have been unable to find the difference between both branches so far (they diverged a couple of months ago, but the base system is still regularly updated on both, and pretty similar). After some digging around, I assume this commit is related to our issue: https://github.com/ruby/ruby/commit/edeb0319f7a95dfe3f9b895bcf32371dd8514726 -- https://bugs.ruby-lang.org/

6 12

[ruby-core:119903] [Ruby master Bug#20892] `ObjectSpace.dump` can produce broken JSON for classes with temporary names
by byroot (Jean Boussier) 14 Jan '25

14 Jan '25

Issue #20892 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20892: `ObjectSpace.dump` can produce broken JSON for classes with temporary names https://bugs.ruby-lang.org/issues/20892 * Author: byroot (Jean Boussier) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ```ruby puts ObjectSpace.dump(Class.new.tap { |c| c.set_temporary_name('" little boby table') }) ``` Produces the following JSON: ```json { "address":"0x120bea980", "type":"CLASS", "shape_id":2, "slot_size":160, "class":"0x120bea840", "variation_count":0, "superclass":"0x1041ffd38", "name":"" little boby table", "references":["0x1041ffd38", "0x123e71a98"], "memsize":192, "flags":{"wb_protected":true} } ``` Notice how the `name` property is broken. This only impacts 3.3.x, as that's when `set_temporary_name` was introduced, before that, it was impossible to assign a name that would cause such bug. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119884] [Ruby master Bug#20886] Crash due to double free on regex timeout after stack allocations
by jhawthorn (John Hawthorn) 14 Jan '25

14 Jan '25

Issue #20886 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #20886: Crash due to double free on regex timeout after stack allocations https://bugs.ruby-lang.org/issues/20886 * Author: jhawthorn (John Hawthorn) * Status: Open * ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [x86_64-linux] * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- As of the change from #20650 ([1057485](https://github.com/ruby/ruby/commit/10574857ce167869524b97ee862b61…) it's possible to crash on a double free due to `stk_alloc` AKA `msa->stack_p` being freed twice, once at the end of match_at and a second time in `FREE_MATCH_ARG` in the parent caller. It's fairly, but not quite 100% reliable to reproduce, adjusting the timeout or number of spaces can help. I reduced this test case from a larger real-world regex, I believe the first part is important just to disable the match cache. ``` $ ruby -e 'Regexp.new("d()*+|a*a*bc", timeout: 0.2) === "b" + "a"*800' double free or corruption (!prev) ``` https://github.com/ruby/ruby/pull/12030 -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119764] [Ruby master Bug#20873] `printf("%f")` returns wrong result for `Rational` with `FIXNUM_MIN`
by nobu (Nobuyoshi Nakada) 14 Jan '25

14 Jan '25

Issue #20873 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20873: `printf("%f")` returns wrong result for `Rational` with `FIXNUM_MIN` https://bugs.ruby-lang.org/issues/20873 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since the beginning, ruby 2.3. ```ruby require 'rbconfig/sizeof' fmin, fmax = RbConfig::LIMITS.values_at("FIXNUM_MIN", "FIXNUM_MAX") sprintf("%f", Rational(fmin, fmax)) #=> --1.000000 ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119855] [Ruby master Bug#20883] sprintf with binary strings has incorrect coderange
by jhawthorn (John Hawthorn) 14 Jan '25

14 Jan '25

Issue #20883 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #20883: sprintf with binary strings has incorrect coderange https://bugs.ruby-lang.org/issues/20883 * Author: jhawthorn (John Hawthorn) * Status: Open * ruby -v: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux] * Backport: 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- In [9dc6065](https://github.com/ruby/ruby/commit/9dc60653db186b1ae9400ed75b413a… we stopped updating the coderange when the encoding became ENCINDEX_ASCII_8BIT. Though ASCII-8BIT (binary) can't ever have a broken coderange, it does need to distinguish between 7bit and valid. ``` ruby puts ("%s" % "\xe2".b).ascii_only? # => false from Ruby 2.1 to 3.1 # => true (incorrect) on Ruby 3.2 and 3.3 # => false again on HEAD ``` This was fixed in master as part of #20189, which clears the coderange in more cases of `rb_str_resize` which happens to fix this, but we shouldn't rely on it. https://github.com/ruby/ruby/pull/12029 -- https://bugs.ruby-lang.org/

3 4

[ruby-core:119738] [Ruby master Bug#20868] Method#hash changes after compaction
by peterzhu2118 (Peter Zhu) 14 Jan '25

14 Jan '25

Issue #20868 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20868: Method#hash changes after compaction https://bugs.ruby-lang.org/issues/20868 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Fix: https://github.com/ruby/ruby/pull/12004 The hash value of a Method may change after a compaction but must remain constant because otherwise it may not work as the key in a hash table. For example: ```ruby def a; end # Need this method here because otherwise the iseq may be on the C stack # which would get pinned and not move during compaction def get_hash method(:a).hash end puts get_hash # => 2993401401091578131 GC.verify_compaction_references(expand_heap: true, toward: :empty) puts get_hash # => -2162775864511574135 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119470] [Ruby master Bug#20787] IO#readline does not check its arguments like IO#gets and will read more data than limit
by javanthropus (Jeremy Bopp) 14 Jan '25

14 Jan '25

Issue #20787 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Bug #20787: IO#readline does not check its arguments like IO#gets and will read more data than limit https://bugs.ruby-lang.org/issues/20787 * Author: javanthropus (Jeremy Bopp) * Status: Open * ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- In revision d3574c117a637a4456aa3ee78e24d8df510b9355, the implementation of IO#readline was modified and consequently broke argument handling in a subtle way. It no longer checks that the encoding of the separator string is compatible with the internal encoding of the stream. Prior to version 3.3.0, the following script raises an ArgumentError when calling #readline: ```ruby require "tempfile" Tempfile.open(encoding: "utf-8:utf-32le") { |f| f.write("0123456789"); f.rewind; f.readline("\0", 1); } ``` After 3.3.0, the script will read all the data in the file, in this case 40 bytes, even though the limit argument is 1. Replacing #readline with #gets raises the ArgumentError in all versions. I'm fairly sure that the failure to check the separator string encoding leads to the incorrect handling of the limit argument. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119621] [Ruby master Bug#20816] Potential regression in Ruby 3.3.x (compared with 3.1 and 3.2) regarding fast syscalls and multi-threading.
by adrienjarthon (Adrien Jarthon) 14 Jan '25

14 Jan '25

Issue #20816 has been reported by adrienjarthon (Adrien Jarthon). ---------------------------------------- Bug #20816: Potential regression in Ruby 3.3.x (compared with 3.1 and 3.2) regarding fast syscalls and multi-threading. https://bugs.ruby-lang.org/issues/20816 * Author: adrienjarthon (Adrien Jarthon) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ok I'm sorry in advance if this is not really a regression (maybe it's the cost to pay for other improvements) but it still showed as a 30% performance drop in my benchmark so I think it may be worth a look. For the full story about how I came down this rabbit hole I just wrote a blog post to avoid cluttering the issue: https://updown.io/blog/weird-results-comparing-ruby-3-1-3-2-3-3-with-jemall… (please read it first) Here I'm gonna put my last findings and the smallest reproducible example I was able to write. So basically I noticed that quick syscalls (at least with a good SSD) like `File.exists?` or `File.mtime`, when used in a multi-threaded environement (Sidekiq in my initial testing but I reduced it to a few Thread.new) are significantly slower in 3.3.x (tested with the same results on 3.3.0, 3.3.2 and 3.3.5 for the moment) in comparison with 3.1.2 or 3.2.5: about twice as slow. These syscalls are heavily used by the Rails Reloader which is how it had such a huge impact (30%) in my testing. Here is the small reproduction script I wrote: ``` ruby #!/usr/bin/env ruby GC.disable # just to make sure it doesn't skew results THREADS = (ARGV.first || 10).to_i # choose thread count from CLI N = 10_000_000 # to make the test longer or shorter puts "Ruby #{RUBY_VERSION} / YJIT: #{RubyVM::YJIT.enabled?} / #{THREADS} Threads" Array.new(THREADS).map do Thread.new do (N/THREADS).times { File.mtime(__FILE__) } end end.each(&:join) puts "#{N} calls completed" ``` The goal is just to run a lot of fast syscalls, in various number of threads, and here are the numbers on my machine: |Threads\Ruby|3.1.2 |3.3.5 | |--|--|--| |1 | 13.7s | ✅ 14.2s | |2 | 9.7s | ⚠️ 33.4s | |4 | 23.2s | ⚠️ 37.0s | |8 | 21.8s | ⚠️ 39.7s | |16 | 23.1s | ⚠️ 41.5s | (more numbers in the [blog post](https://updown.io/blog/weird-results-comparing-ruby-3-1-3-2-3-3-with-jemalloc-and-yjit)) I suspected the [new N:M thread scheduler introduced in 3.3](https://github.com/ruby/ruby/pull/8629), even though it's "disabled" by default on the main Ractor, thinking maybe these changes degraded a bit this (quite specific scenario). So I gave a quick try with the feature ENABLED, to see if it would at least change anything and it did ! (though not was I expected): |Threads\Ruby|3.1.2 |3.3.5 | 3.3.5 with RUBY_MN_THREADS=1 | |--|--|--|--| |1 | 13.7s | 14.2s | 14.5s ✅ | |2 | 9.7s | ⚠️ 33.4s | 12.3s 🎉 | |4 | 23.2s | ⚠️ 37.0s | 40.5s 🧐 | |8 | 21.8s | ⚠️ 39.7s | 37.9s 🧐 | |16 | 23.1s | ⚠️ 41.5s | 36.3s 🧐 | (I can make graphs for these numbers if it helps) Now I'm a bit stuck in my investigation, I'm not sure what change in 3.3 caused this (the new thread scheduler is a good lead IMO) or if it even is a regression. Maybe this downside is innevitable for other benefits? I am happy to contribute more time to this issue and perform more tests but I would need some guidance on what could be interesting to test, from people who understand a bit better than me how this works internally. Thanks ! (and sorry for the long read) -- https://bugs.ruby-lang.org/

4 5

[ruby-core:112005] [Ruby master Bug#19374] Issue with Ractor.make_shareable with curried procs
by luke-gru (Luke Gruber) 13 Jan '25

13 Jan '25

Issue #19374 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19374: Issue with Ractor.make_shareable with curried procs https://bugs.ruby-lang.org/issues/19374 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This works, but shouldn't: ```ruby class Worker def start(&blk) blk = blk.curry # bug in ruby allows sharing of non-shareable proc Ractor.make_shareable(blk) @ractor = Ractor.new(blk) do |b| main = b.call p "from ractor: #{main}" end end def work @ractor.take end end worker = Worker.new a = self # unshareable main object p "from main: #{a}" worker.start { a } worker.work ``` The curried proc has a reference to the original proc and it's not checked for shareability. -- https://bugs.ruby-lang.org/

3 4