July 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118446] [Ruby master Bug#20608] Hash#find always allocates each iterated pair
by ivoanjo (Ivo Anjo) 10 Jul '24

10 Jul '24

Issue #20608 has been reported by ivoanjo (Ivo Anjo). ---------------------------------------- Bug #20608: Hash#find always allocates each iterated pair https://bugs.ruby-lang.org/issues/20608 * Author: ivoanjo (Ivo Anjo) * Status: Open * ruby -v: ruby 3.4.0preview1 (2024-05-16 master 9d69619623) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hey there! Recently I ran into this sharp edge in `Hash#find`: ```ruby puts RUBY_DESCRIPTION def allocated_now = GC.stat(:total_allocated_objects) dummy_hash = 10_000.times.each_with_index.to_h before_allocs = allocated_now dummy_hash.any? { |k, v| } puts "Allocated #{allocated_now - before_allocs} for #{File.read(__FILE__).lines[__LINE__-2]}" before_allocs = allocated_now dummy_hash.each { |k, v| } puts "Allocated #{allocated_now - before_allocs} for #{File.read(__FILE__).lines[__LINE__-2]}" before_allocs = allocated_now dummy_hash.find { |k, v| } puts "Allocated #{allocated_now - before_allocs} for #{File.read(__FILE__).lines[__LINE__-2]}" before_allocs = allocated_now dummy_hash.any? { |k| } puts "Allocated #{allocated_now - before_allocs} for #{File.read(__FILE__).lines[__LINE__-2]}" ``` Result: ``` ruby 3.4.0preview1 (2024-05-16 master 9d69619623) [x86_64-linux] Allocated 0 for dummy_hash.any? { |k, v| } Allocated 0 for dummy_hash.each { |k, v| } Allocated 10002 for dummy_hash.find { |k, v| } Allocated 10000 for dummy_hash.any? { |k| } ``` That is, while `Hash#any?`, `Hash#each`, etc avoid doing any allocations during iteration, `Hash#find` does not hit the `rb_block_pair_yield_optimizable` => `each_pair_i_fast` fast path, and so is massively costly compared to the others. This is very surprising, as I'd expect a `find` to have a comparable cost to `any?` (and I ended up [redoing some code to avoid find](https://github.com/DataDog/dd-trace-rb/pull/3757)) Also while experimenting a bit, it was surprising to me that the allocation optimization only kicks in when `|k, v|` are declared, and thus `.any? { |k| }` is also more expensive. -- https://bugs.ruby-lang.org/

2 5

[ruby-core:117653] [Ruby master Bug#20446] OUtdated https://cache.ruby-lang.org/pub/ruby/index.txt
by vo.x (Vit Ondruch) 09 Jul '24

09 Jul '24

Issue #20446 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20446: OUtdated https://cache.ruby-lang.org/pub/ruby/index.txt https://bugs.ruby-lang.org/issues/20446 * Author: vo.x (Vit Ondruch) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Is the https://cache.ruby-lang.org/pub/ruby/index.txt the place, where all Ruby releases can be found? If yes, then can this file be update with Ruby 3.3.1, 3.2.4 etc? If no, can this file be removed to avoid confusion? -- https://bugs.ruby-lang.org/

3 6

[ruby-core:118428] [Ruby master Bug#20605] PRs with backported fix for https://github.com/ruby/ruby/pull/11036 to Ruby 3.3 and 3.2
by ivoanjo (Ivo Anjo) 09 Jul '24

09 Jul '24

Issue #20605 has been reported by ivoanjo (Ivo Anjo). ---------------------------------------- Bug #20605: PRs with backported fix for https://github.com/ruby/ruby/pull/11036 to Ruby 3.3 and 3.2 https://bugs.ruby-lang.org/issues/20605 * Author: ivoanjo (Ivo Anjo) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hey! I work for Datadog on the Ruby profiler part of the [datadog (previously ddtrace)](https://github.com/datadog/dd-trace-rb) gem. Recently we merged a very small tweak to master to make ensure compilers did the right thing always: https://github.com/ruby/ruby/pull/11036 . It would be really cool to also get this fix on the Ruby versions still under active maintenance (3.2 and 3.3). I've gone ahead and opened PRs backporting this fix: * https://github.com/ruby/ruby/pull/11090 * https://github.com/ruby/ruby/pull/11091 Let me know if I can help any further :) -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118487] [Ruby master Bug#20617] /\pArabic/ character property doesn't match certain Arabic characters
by kytrinyx (Katrina Owen) 09 Jul '24

09 Jul '24

Issue #20617 has been reported by kytrinyx (Katrina Owen). ---------------------------------------- Bug #20617: /\pArabic/ character property doesn't match certain Arabic characters https://bugs.ruby-lang.org/issues/20617 * Author: kytrinyx (Katrina Owen) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin21] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I am not sure this is a bug. On some occasions I have Arabic text, but the Arabic character property rejects it as being Arabic. Example: ``` str = "شغل مرحلة أولى ، جداً؟" /^\p{Arabic}$/.match(str).inspect # => nil str.chars.reject {|char| /\p{Arabic}/.match(char)}.uniq # arabic space, arabic comma, arabic question mark, and arabic fatahan ``` This isn't a problem, since I defined my own regex to include the missing characters, but wanted to raise it in case it is, in fact, a bug. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118269] [Ruby master Bug#20570] Nokey behavior changed since 3.3.
by ksss (Yuki Kurihara) 08 Jul '24

08 Jul '24

Issue #20570 has been reported by ksss (Yuki Kurihara). ---------------------------------------- Bug #20570: Nokey behavior changed since 3.3. https://bugs.ruby-lang.org/issues/20570 * Author: ksss (Yuki Kurihara) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I am using code that transfers the following method call, but found that the behavior has changed since CRuby 3.3. ```ruby receiver_value = Set.new method_name = :merge args = [1] kwargs = {} block = nil receiver_value.__send__(method_name, *args, **kwargs, &block) # => no keywords accepted (ArgumentError) ``` Upon investigation, I found that the behavior of calling a method using `**nil` has changed starting from version 3.3. ``` $ docker run -it --rm rubylang/all-ruby env ALL_RUBY_SINCE=ruby-2.6 ./all-ruby -e 'def foo(*, **nil); end; p foo(*[], **{})' WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ruby-2.6.0 -e:1: syntax error, unexpected nil, expecting ')' def foo(*, **nil); end; p foo(*[], **{}) ^~~ exit 1 ... ruby-2.6.10 -e:1: syntax error, unexpected nil, expecting ')' def foo(*, **nil); end; p foo(*[], **{}) ^~~ exit 1 ruby-2.7.0-preview1 -e:1: syntax error, unexpected `nil', expecting ')' def foo(*, **nil); end; p foo(*[], **{}) ^~~ exit 1 ruby-2.7.0-preview2 nil ... ruby-3.2.4 nil ruby-3.3.0-preview1 -e:1:in `<main>': no keywords accepted (ArgumentError) def foo(*, **nil); end; p foo(*[], **{}) ^^^^^^^^^ exit 1 ... ruby-3.3.2 -e:1:in `<main>': no keywords accepted (ArgumentError) def foo(*, **nil); end; p foo(*[], **{}) ^^^^^^^^^ exit 1 ruby-3.4.0-preview1 -e:1:in '<main>': no keywords accepted (ArgumentError) def foo(*, **nil); end; p foo(*[], **{}) ^^^^^^^^^ exit 1 ``` Is this change intentional? -- https://bugs.ruby-lang.org/

5 7

[ruby-core:116597] [Ruby master Bug#20239] Segmentation fault when using Regex on a large String
by martinsp (Martins Polakovs) 08 Jul '24

08 Jul '24

Issue #20239 has been reported by martinsp (Martins Polakovs). ---------------------------------------- Bug #20239: Segmentation fault when using Regex on a large String https://bugs.ruby-lang.org/issues/20239 * Author: martinsp (Martins Polakovs) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since v3.2.0 ruby crashes with segmentation fault on the following script with a `[BUG] Segmentation fault at ...` ``` ruby require "rbconfig/sizeof" ("\u{0101}" + "a" * RbConfig::LIMITS["INT_MAX"] + "b").match(/b/) ``` Crash can be reproduced on the following ruby versions: - ruby 3.2.0 (2022-12-25 revision a528908271) [aarch64-linux] - ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [aarch64-linux] - ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [aarch64-linux] works as expected It seems that call to `enclen` inside `str_lower_case_match` returns negative offset in this case https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/v3… -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118399] [Ruby master Bug#20598] Corruption of internal encoding string
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20598 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20598: Corruption of internal encoding string https://bugs.ruby-lang.org/issues/20598 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11068 Just like #20595, Encoding#name_list and Encoding#aliases can have their strings corrupted when Encoding.default_internal is set to nil. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118391] [Ruby master Bug#20595] Corruption of encoding name string
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20595 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20595: Corruption of encoding name string https://bugs.ruby-lang.org/issues/20595 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11062 enc_set_default_encoding will free the C string if the encoding is nil, but the C string can be used by the encoding name string. This will cause the encoding name string to be corrupted. Consider the following code: ```ruby Encoding.default_internal = Encoding::ASCII_8BIT names = Encoding.default_internal.names p names Encoding.default_internal = nil p names ``` It outputs: ``` ["ASCII-8BIT", "BINARY", "internal"] ["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"] ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118345] [Ruby master Bug#20585] Size of memory allocated by String.new(:capacity) is different from the specified value
by os (Shigeki OHARA) 08 Jul '24

08 Jul '24

Issue #20585 has been reported by os (Shigeki OHARA). ---------------------------------------- Bug #20585: Size of memory allocated by String.new(:capacity) is different from the specified value https://bugs.ruby-lang.org/issues/20585 * Author: os (Shigeki OHARA) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [x86_64-freebsd14.0] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- IMHO, if :capacity is specified in String.new, capa will be its value. In fact, Ruby 3.2 seems to allocate the size as specified. ``` % cat string_capacity.rb unless /\A3\.[23]\./ =~ RUBY_VERSION raise NotImplementedError, 'Not Supported Ruby Version' end require 'inline' class String def super_inspect self.class.superclass.instance_method(:inspect).bind(self).call end inline do |builder| builder.include '<stdio.h>' builder.add_compile_flags '-Wall' builder.c_raw <<~CODE VALUE capacity(int argc, VALUE *argv, VALUE self) { struct RString *rstring = RSTRING(self); if (! (RBASIC(self)->flags & RSTRING_NOEMBED)) { return rb_to_symbol(rb_str_new_cstr("EMBED")); } else { if (RBASIC(self)->flags & ELTS_SHARED) { return rb_to_symbol(rb_str_new_cstr("SHARED")); } else { return LONG2NUM(rstring->as.heap.aux.capa); } } return Qnil; /* NOTREACHED */ } CODE end end ``` ``` % irb -I. -rstring_capacity irb(main):001:0> [RUBY_PLATFORM, RUBY_VERSION] => ["x86_64-freebsd14.0", "3.2.4"] irb(main):002:0> String.new('', capacity: 1024).capacity => 1024 irb(main):003:0> String.new('*'*1024, capacity: 1024).capacity => 1024 irb(main):004:0> ``` This is what I expect. However, Ruby 3.3 seems to behave differently. ``` % irb -I. -rstring_capacity irb(main):001> [RUBY_PLATFORM, RUBY_VERSION] => ["x86_64-freebsd14.0", "3.3.2"] irb(main):002> String.new('', capacity: 1024).capacity => 1023 irb(main):003> String.new('*'*1024, capacity: 1024).capacity => 2047 irb(main):004> ``` * If only :capacity is specified, one byte less is allocated. * If the initial string and its bytesize are specified, about twice the size is allocated. Is this intentional? -- https://bugs.ruby-lang.org/

4 4

[ruby-core:118320] [Ruby master Bug#20581] Ruby 3.3.3 install has missing deps for bundled net-pop gem
by MatzFan (Bruce Steedman) 08 Jul '24

08 Jul '24

Issue #20581 has been reported by MatzFan (Bruce Steedman). ---------------------------------------- Bug #20581: Ruby 3.3.3 install has missing deps for bundled net-pop gem https://bugs.ruby-lang.org/issues/20581 * Author: MatzFan (Bruce Steedman) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- A stackoverflow question and answer has identified an issue with the Ruby 3.3.3 install which can cause a failure when bundler installs gem dependencies. https://stackoverflow.com/a/78620570/4114896 A workaround has been described. There is a related issue with the net-pop gem here: https://github.com/ruby/net-pop/issues/26 -- https://bugs.ruby-lang.org/

3 3