June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118399] [Ruby master Bug#20598] Corruption of internal encoding string
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20598 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20598: Corruption of internal encoding string https://bugs.ruby-lang.org/issues/20598 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11068 Just like #20595, Encoding#name_list and Encoding#aliases can have their strings corrupted when Encoding.default_internal is set to nil. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118391] [Ruby master Bug#20595] Corruption of encoding name string
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20595 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20595: Corruption of encoding name string https://bugs.ruby-lang.org/issues/20595 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11062 enc_set_default_encoding will free the C string if the encoding is nil, but the C string can be used by the encoding name string. This will cause the encoding name string to be corrupted. Consider the following code: ```ruby Encoding.default_internal = Encoding::ASCII_8BIT names = Encoding.default_internal.names p names Encoding.default_internal = nil p names ``` It outputs: ``` ["ASCII-8BIT", "BINARY", "internal"] ["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"] ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118345] [Ruby master Bug#20585] Size of memory allocated by String.new(:capacity) is different from the specified value
by os (Shigeki OHARA) 08 Jul '24

08 Jul '24

Issue #20585 has been reported by os (Shigeki OHARA). ---------------------------------------- Bug #20585: Size of memory allocated by String.new(:capacity) is different from the specified value https://bugs.ruby-lang.org/issues/20585 * Author: os (Shigeki OHARA) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [x86_64-freebsd14.0] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- IMHO, if :capacity is specified in String.new, capa will be its value. In fact, Ruby 3.2 seems to allocate the size as specified. ``` % cat string_capacity.rb unless /\A3\.[23]\./ =~ RUBY_VERSION raise NotImplementedError, 'Not Supported Ruby Version' end require 'inline' class String def super_inspect self.class.superclass.instance_method(:inspect).bind(self).call end inline do |builder| builder.include '<stdio.h>' builder.add_compile_flags '-Wall' builder.c_raw <<~CODE VALUE capacity(int argc, VALUE *argv, VALUE self) { struct RString *rstring = RSTRING(self); if (! (RBASIC(self)->flags & RSTRING_NOEMBED)) { return rb_to_symbol(rb_str_new_cstr("EMBED")); } else { if (RBASIC(self)->flags & ELTS_SHARED) { return rb_to_symbol(rb_str_new_cstr("SHARED")); } else { return LONG2NUM(rstring->as.heap.aux.capa); } } return Qnil; /* NOTREACHED */ } CODE end end ``` ``` % irb -I. -rstring_capacity irb(main):001:0> [RUBY_PLATFORM, RUBY_VERSION] => ["x86_64-freebsd14.0", "3.2.4"] irb(main):002:0> String.new('', capacity: 1024).capacity => 1024 irb(main):003:0> String.new('*'*1024, capacity: 1024).capacity => 1024 irb(main):004:0> ``` This is what I expect. However, Ruby 3.3 seems to behave differently. ``` % irb -I. -rstring_capacity irb(main):001> [RUBY_PLATFORM, RUBY_VERSION] => ["x86_64-freebsd14.0", "3.3.2"] irb(main):002> String.new('', capacity: 1024).capacity => 1023 irb(main):003> String.new('*'*1024, capacity: 1024).capacity => 2047 irb(main):004> ``` * If only :capacity is specified, one byte less is allocated. * If the initial string and its bytesize are specified, about twice the size is allocated. Is this intentional? -- https://bugs.ruby-lang.org/

4 4

[ruby-core:118320] [Ruby master Bug#20581] Ruby 3.3.3 install has missing deps for bundled net-pop gem
by MatzFan (Bruce Steedman) 08 Jul '24

08 Jul '24

Issue #20581 has been reported by MatzFan (Bruce Steedman). ---------------------------------------- Bug #20581: Ruby 3.3.3 install has missing deps for bundled net-pop gem https://bugs.ruby-lang.org/issues/20581 * Author: MatzFan (Bruce Steedman) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- A stackoverflow question and answer has identified an issue with the Ruby 3.3.3 install which can cause a failure when bundler installs gem dependencies. https://stackoverflow.com/a/78620570/4114896 A workaround has been described. There is a related issue with the net-pop gem here: https://github.com/ruby/net-pop/issues/26 -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118226] [Ruby master Bug#20562] Categorize `RUBY_FREE_AT_EXIT` warning
by nobu (Nobuyoshi Nakada) 08 Jul '24

08 Jul '24

Issue #20562 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20562: Categorize `RUBY_FREE_AT_EXIT` warning https://bugs.ruby-lang.org/issues/20562 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```sh $ RUBY_FREE_AT_EXIT=1 ruby -W:no-experimental -ep ruby: warning: Free at exit is experimental and may be unstable ``` It should be categorized as experimental as its message. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116078] [Ruby master Bug#20162] Memory leak when duplicating too complex object
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20162 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20162: Memory leak when duplicating too complex object https://bugs.ruby-lang.org/issues/20162 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9449 Creating a ST table then calling st_replace leaks memory because the st_replace overwrites the ST table without freeing any of the existing memory. This commit changes it to use st_copy instead. For example: ```ruby RubyVM::Shape.exhaust_shapes o = Object.new o.instance_variable_set(:@a, 0) 10.times do 100_000.times { o.dup } puts `ps -o rss= -p #{$$}` end ``` Before: ``` 23264 33600 42672 52160 61600 71728 81056 90528 100560 109840 ``` After: ``` 14752 14816 15584 15584 15664 15664 15664 15664 15664 15664 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114932] [Ruby master Bug#19907] Method calls with keyword arguments in eval leaks callcache and callinfo objects
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #19907 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19907: Method calls with keyword arguments in eval leaks callcache and callinfo objects https://bugs.ruby-lang.org/issues/19907 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script leaks callcache and callinfo objects: ```ruby def foo(a:); end 10.times do 10_000.times do eval(<<~RUBY) foo(a: 1) RUBY end puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output is: ``` Number of live objects: 46248 Memory usage: 16160 Number of live objects: 65902 Memory usage: 19888 Number of live objects: 92656 Memory usage: 24032 Number of live objects: 126791 Memory usage: 28048 Number of live objects: 132919 Memory usage: 28816 Number of live objects: 180687 Memory usage: 32384 Number of live objects: 181957 Memory usage: 32464 Number of live objects: 227485 Memory usage: 34224 Number of live objects: 256101 Memory usage: 37200 Number of live objects: 274151 Memory usage: 38752 ``` After performing a `ObjectSpace.dump_all`, I found that it is leaking callcache and callinfo objects that is being held on by the `Object` class. ```json {"address":"0x102ecef70", "type":"CLASS", "shape_id":2, "slot_size":160, "class":"0x102ecf8d0", "variation_count":0, "superclass":"0x102ecfd30", "real_class_name":"Object", "singleton":true, "references":[ ... ]} {"address":"0x1030c90a0", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callinfo", "mid":"foo", "memsize":40, "flags":{"wb_protected":true}} {"address":"0x102fad568", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callcache", "memsize":40, "flags":{"wb_protected":true, "old":true, "uncollectible":true, "marked":true}} ``` -- https://bugs.ruby-lang.org/

5 4

[ruby-core:115999] [Ruby master Bug#20145] Memory leak when duplicating identhash
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20145 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20145: Memory leak when duplicating identhash https://bugs.ruby-lang.org/issues/20145 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9411 Both copy_compare_by_id and hash_copy will create a copy of the ST table, so the ST table created in copy_compare_by_id will be leaked. ```ruby h = { 1 => 2 }.compare_by_identity 10.times do 1_000_000.times do h.select { false } end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 110736 204352 300272 395520 460704 476736 542000 604704 682624 770528 ``` After: ``` 15504 16048 16144 16256 16320 16320 16752 16752 16752 16752 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116514] [Ruby master Bug#20228] Memory leak in Regexp timeout
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20228 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20228: Memory leak in Regexp timeout https://bugs.ruby-lang.org/issues/20228 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9765 If a Regexp::TimeoutError is raised, the `stk_base` and `OnigRegion` will leak. For example: ```ruby Regexp.timeout = 0.001 regex = /^(a*)*$/ str = "a" * 1000000 + "x" 10.times do 100.times do begin regex =~ str rescue end end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 328800 632416 934368 1230448 1531088 1831248 2125072 2414384 2703440 2995664 ``` After: ``` 39280 47888 49024 56240 56496 56512 56592 56592 56720 56720 ``` -- https://bugs.ruby-lang.org/

5 5

[ruby-core:111306] [Ruby master Feature#19236] Allow to create hashes with a specific capacity from Ruby
by byroot (Jean Boussier) 07 Jul '24

07 Jul '24

Issue #19236 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19236: Allow to create hashes with a specific capacity from Ruby https://bugs.ruby-lang.org/issues/19236 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.3 ---------------------------------------- Followup on [Feature #18683] which added a C-API for this purpose. Various protocol parsers such as Redis `RESP3` or `msgpack`, have to create hashes, and they know the size in advance. For efficiency, it would be preferable if they could directly allocate a Hash of the necessary size, so that large hashes wouldn't cause many re-alloccations and re-hash. `String` and `Array` both already offer similar APIs: ```ruby String.new(capacity: XXX) Array.new(XX) / rb_ary_new_capa(long) ``` However there's no such public API for Hashes in Ruby land. ### Proposal I think `Hash` should have a way to create a new hash with a `capacity` parameter. The logical signature of `Hash.new(capacity: 1000)` was deemed too incompatible in [Feature #18683]. @Eregon proposed to add `Hash.create(capacity: 1000)`. -- https://bugs.ruby-lang.org/

8 14