January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:114792] [Ruby master Bug#19887] RUBYOPT doesn't work unless starting with `-`
by nobu (Nobuyoshi Nakada) 20 Jul '24

20 Jul '24

Issue #19887 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19887: RUBYOPT doesn't work unless starting with `-` https://bugs.ruby-lang.org/issues/19887 * Author: nobu (Nobuyoshi Nakada) * Status: Assigned * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- It was working till 1.8. ```sh-session $ for ruby in ruby{1.{8..9},2.{0..7},3.{0..3}}; do RUBYOPT=w $ruby -e 'puts "'$ruby': #{$VERBOSE}"'; done ruby1.8: true ruby1.9: false ruby2.0: false ruby2.1: false ruby2.2: false ruby2.3: false ruby2.4: false ruby2.5: false ruby2.6: false ruby2.7: false ruby3.0: false ruby3.1: false ruby3.2: false ruby3.3: false ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:115728] [Ruby master Bug#20062] Numbered parameters are broken in Ruby 3.3-dev
by osyo (manga osyo) 20 Jul '24

20 Jul '24

Issue #20062 has been reported by osyo (manga osyo). ---------------------------------------- Bug #20062: Numbered parameters are broken in Ruby 3.3-dev https://bugs.ruby-lang.org/issues/20062 * Author: osyo (manga osyo) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-13T13:16:11Z master f390c51b15) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Define Numbered parameters and a method in one block and then reference Numbered parameters in another block, you will get a SyntaxError in Ruby 3.3. This is not a problem in Ruby 3.2. ```ruby 'a'.tap do p _1 def f1; end end 'a'.tap do p _1 end __END__ output: Ruby 3.2.2 => "a" "a" Ruby 3.3.0dev(2023-12-13T13:16:11Z master f390c51b15) => /path/to/test.rb: /path/to/test.rb:8: numbered parameter is already used in (SyntaxError) /path/to/test.rb:2: outer block here ``` I noticed this while checking the operation of https://bugs.ruby-lang.org/issues/18966. I haven't checked, but https://bugs.ruby-lang.org/issues/19619 may be related. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:114258] [Ruby master Bug#19781] Stack consistency error with tailcall and YJIT enabled
by k0kubun (Takashi Kokubun) 20 Jul '24

20 Jul '24

Issue #19781 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #19781: Stack consistency error with tailcall and YJIT enabled https://bugs.ruby-lang.org/issues/19781 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal * Assignee: yjit * Backport: 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- ```rb src = <<-EOS def apply_one_and_two(&block) [1, p(1)] end def add_one_and_two apply_one_and_two(&:+) end EOS RubyVM::InstructionSequence.new( "proc {|_|_.class_eval {#{src}}}", __FILE__, __FILE__, 1, tailcall_optimization: true, trace_instruction: false, ).eval[self.singleton_class] def entry add_one_and_two end entry ``` This script causes "Stack consistency error" with `ruby --yjit-call-threshold=1` on any YJIT-enabled build (3.1, 3.2, and master). -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115549] [Ruby master Bug#20030] `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string.
by tompng (tomoya ishida) 15 Jul '24

15 Jul '24

Issue #20030 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20030: `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string. https://bugs.ruby-lang.org/issues/20030 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-11-30T16:23:25Z master d048bae96b) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ~~~ruby Ripper.tokenize '"\\C-あ"' # or Ripper.tokenize "\"\\C-\u3042\"" # => ["\"", "\x81", "\x82", "\""] ~~~ I expect all tokens to be valid_encoding if the source string is valid_encoding. This is causing IRB crash when typing `"\C-あ"`. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116114] [Ruby master Bug#20169] `GC.compact` can raises `EFAULT` on IO
by ko1 (Koichi Sasada) 15 Jul '24

15 Jul '24

Issue #20169 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20169: `GC.compact` can raises `EFAULT` on IO https://bugs.ruby-lang.org/issues/20169 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- 1. `GC.compact` introduces read barriers to detect read accesses to the pages. 2. I/O operations release GVL to pass the control while their execution, and another thread can call `GC.compact` (or auto compact feature I guess, but not checked yet). 3. Call `write(ptr)` can return `EFAULT` when `GC.compact` is running because `ptr` points read-barrier protected pages. Reproducible steps: Apply the following patch to increase possibility: ```patch diff --git a/io.c b/io.c index f6cd2c1a56..83d67ba2dc 100644 --- a/io.c +++ b/io.c @@ -1212,8 +1212,12 @@ internal_write_func(void *ptr) } } + int cnt = 0; retry: - do_write_retry(write(iis->fd, iis->buf, iis->capa)); + for (; cnt < 1000; cnt++) { + do_write_retry(write(iis->fd, iis->buf, iis->capa)); + if (result <= 0) break; + } if (result < 0 && !iis->nonblock) { int e = errno; ``` Run the following code: ```ruby t1 = Thread.new{ 10_000.times.map{"#{_1}"}; GC.compact while true } t2 = Thread.new{ i=0 $stdout.write "<#{i+=1}>" while true } t2.join ``` and ``` $ make run (snip) 4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4>#<Thread:0x00007fa61b4dd758 ../../src/trunk/test.rb:3 run> terminated with exception (report_on_exception is true): ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' make: *** [uncommon.mk:1383: run] Error 1 ``` I think this is why we get `EFAULT` on CI. To increase possibilities running many busy processes (`ruby -e 'loop{}'` for example) will help (and on CI environment there are such busy processes accidentally). -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116078] [Ruby master Bug#20162] Memory leak when duplicating too complex object
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20162 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20162: Memory leak when duplicating too complex object https://bugs.ruby-lang.org/issues/20162 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9449 Creating a ST table then calling st_replace leaks memory because the st_replace overwrites the ST table without freeing any of the existing memory. This commit changes it to use st_copy instead. For example: ```ruby RubyVM::Shape.exhaust_shapes o = Object.new o.instance_variable_set(:@a, 0) 10.times do 100_000.times { o.dup } puts `ps -o rss= -p #{$$}` end ``` Before: ``` 23264 33600 42672 52160 61600 71728 81056 90528 100560 109840 ``` After: ``` 14752 14816 15584 15584 15664 15664 15664 15664 15664 15664 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114932] [Ruby master Bug#19907] Method calls with keyword arguments in eval leaks callcache and callinfo objects
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #19907 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19907: Method calls with keyword arguments in eval leaks callcache and callinfo objects https://bugs.ruby-lang.org/issues/19907 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script leaks callcache and callinfo objects: ```ruby def foo(a:); end 10.times do 10_000.times do eval(<<~RUBY) foo(a: 1) RUBY end puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output is: ``` Number of live objects: 46248 Memory usage: 16160 Number of live objects: 65902 Memory usage: 19888 Number of live objects: 92656 Memory usage: 24032 Number of live objects: 126791 Memory usage: 28048 Number of live objects: 132919 Memory usage: 28816 Number of live objects: 180687 Memory usage: 32384 Number of live objects: 181957 Memory usage: 32464 Number of live objects: 227485 Memory usage: 34224 Number of live objects: 256101 Memory usage: 37200 Number of live objects: 274151 Memory usage: 38752 ``` After performing a `ObjectSpace.dump_all`, I found that it is leaking callcache and callinfo objects that is being held on by the `Object` class. ```json {"address":"0x102ecef70", "type":"CLASS", "shape_id":2, "slot_size":160, "class":"0x102ecf8d0", "variation_count":0, "superclass":"0x102ecfd30", "real_class_name":"Object", "singleton":true, "references":[ ... ]} {"address":"0x1030c90a0", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callinfo", "mid":"foo", "memsize":40, "flags":{"wb_protected":true}} {"address":"0x102fad568", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callcache", "memsize":40, "flags":{"wb_protected":true, "old":true, "uncollectible":true, "marked":true}} ``` -- https://bugs.ruby-lang.org/

5 4

[ruby-core:115999] [Ruby master Bug#20145] Memory leak when duplicating identhash
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20145 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20145: Memory leak when duplicating identhash https://bugs.ruby-lang.org/issues/20145 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9411 Both copy_compare_by_id and hash_copy will create a copy of the ST table, so the ST table created in copy_compare_by_id will be leaked. ```ruby h = { 1 => 2 }.compare_by_identity 10.times do 1_000_000.times do h.select { false } end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 110736 204352 300272 395520 460704 476736 542000 604704 682624 770528 ``` After: ``` 15504 16048 16144 16256 16320 16320 16752 16752 16752 16752 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116514] [Ruby master Bug#20228] Memory leak in Regexp timeout
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20228 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20228: Memory leak in Regexp timeout https://bugs.ruby-lang.org/issues/20228 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9765 If a Regexp::TimeoutError is raised, the `stk_base` and `OnigRegion` will leak. For example: ```ruby Regexp.timeout = 0.001 regex = /^(a*)*$/ str = "a" * 1000000 + "x" 10.times do 100.times do begin regex =~ str rescue end end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 328800 632416 934368 1230448 1531088 1831248 2125072 2414384 2703440 2995664 ``` After: ``` 39280 47888 49024 56240 56496 56512 56592 56592 56720 56720 ``` -- https://bugs.ruby-lang.org/

5 5

[ruby-core:111306] [Ruby master Feature#19236] Allow to create hashes with a specific capacity from Ruby
by byroot (Jean Boussier) 07 Jul '24

07 Jul '24

Issue #19236 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19236: Allow to create hashes with a specific capacity from Ruby https://bugs.ruby-lang.org/issues/19236 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.3 ---------------------------------------- Followup on [Feature #18683] which added a C-API for this purpose. Various protocol parsers such as Redis `RESP3` or `msgpack`, have to create hashes, and they know the size in advance. For efficiency, it would be preferable if they could directly allocate a Hash of the necessary size, so that large hashes wouldn't cause many re-alloccations and re-hash. `String` and `Array` both already offer similar APIs: ```ruby String.new(capacity: XXX) Array.new(XX) / rb_ary_new_capa(long) ``` However there's no such public API for Hashes in Ruby land. ### Proposal I think `Hash` should have a way to create a new hash with a `capacity` parameter. The logical signature of `Hash.new(capacity: 1000)` was deemed too incompatible in [Feature #18683]. @Eregon proposed to add `Hash.create(capacity: 1000)`. -- https://bugs.ruby-lang.org/

8 14