February 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116114] [Ruby master Bug#20169] `GC.compact` can raises `EFAULT` on IO
by ko1 (Koichi Sasada) 15 Jul '24

15 Jul '24

Issue #20169 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20169: `GC.compact` can raises `EFAULT` on IO https://bugs.ruby-lang.org/issues/20169 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- 1. `GC.compact` introduces read barriers to detect read accesses to the pages. 2. I/O operations release GVL to pass the control while their execution, and another thread can call `GC.compact` (or auto compact feature I guess, but not checked yet). 3. Call `write(ptr)` can return `EFAULT` when `GC.compact` is running because `ptr` points read-barrier protected pages. Reproducible steps: Apply the following patch to increase possibility: ```patch diff --git a/io.c b/io.c index f6cd2c1a56..83d67ba2dc 100644 --- a/io.c +++ b/io.c @@ -1212,8 +1212,12 @@ internal_write_func(void *ptr) } } + int cnt = 0; retry: - do_write_retry(write(iis->fd, iis->buf, iis->capa)); + for (; cnt < 1000; cnt++) { + do_write_retry(write(iis->fd, iis->buf, iis->capa)); + if (result <= 0) break; + } if (result < 0 && !iis->nonblock) { int e = errno; ``` Run the following code: ```ruby t1 = Thread.new{ 10_000.times.map{"#{_1}"}; GC.compact while true } t2 = Thread.new{ i=0 $stdout.write "<#{i+=1}>" while true } t2.join ``` and ``` $ make run (snip) 4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4>#<Thread:0x00007fa61b4dd758 ../../src/trunk/test.rb:3 run> terminated with exception (report_on_exception is true): ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' make: *** [uncommon.mk:1383: run] Error 1 ``` I think this is why we get `EFAULT` on CI. To increase possibilities running many busy processes (`ruby -e 'loop{}'` for example) will help (and on CI environment there are such busy processes accidentally). -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116908] [Ruby master Bug#20292] Abort ruby by `String#initialize`
by ksss (Yuki Kurihara) 15 Jul '24

15 Jul '24

Issue #20292 has been reported by ksss (Yuki Kurihara). ---------------------------------------- Bug #20292: Abort ruby by `String#initialize` https://bugs.ruby-lang.org/issues/20292 * Author: ksss (Yuki Kurihara) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found a code that causes an anomaly. ``` $ uname -a Darwin 20208671n 22.6.0 Darwin Kernel Version 22.6.0: Wed Oct 4 21:26:55 PDT 2023; root:xnu-8796.141.3.701.17~4/RELEASE_ARM64_T6020 arm64 $ ruby -v ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: -1) }' ruby(13817,0x1f22b2080) malloc: Heap corruption detected, free list is damaged at 0x600000e27bc0 *** Incorrect guard value: 29830901955328 ruby(13817,0x1f22b2080) malloc: *** set a breakpoint in malloc_error_break to debug [1] 13817 abort ruby -e $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: 0) }' # never return .... ``` My research indicates the following conditions. - String must be at least 64 characters long - Call `#initialize` with capacity: 0 or less - Repeat this several times. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116930] [Ruby master Bug#20296] Complex(:sym, exception: false) generate exception with weird timing
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20296 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20296: Complex(:sym, exception: false) generate exception with weird timing https://bugs.ruby-lang.org/issues/20296 * Author: MaxLap (Maxime Lapointe) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Happens in Ruby 3.2.2, 3.2.3 and 3.3.0 (didn't test others). Add this to a file and execute it as-is: ``` 3.times { Complex(:sym, exception: false) } puts "hi" ``` The output will be: ``` $ ruby local2.rb hi local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' ``` So the exception is still reported (instead of being swallow), but the puts, which is after, had the time to execute first. I can even put a `sleep()` in between and the exceptions are still reported after the puts. In a IRB, there is no exception at all and everything seems to work as expected. Happens if I pass in an Array too. Doesn't seem to happen with bad strings. There is also a quadratic behavior somewhere in there. Running the code in an IRB (which doesn't show exceptions), if you double the number of `times` in the loop, it takes 4x as long. This is probably related... -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116965] [Ruby master Bug#20307] `Hash#update` from compare_by_identity hash can have unfrozen string keys
by nobu (Nobuyoshi Nakada) 13 Jul '24

13 Jul '24

Issue #20307 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20307: `Hash#update` from compare_by_identity hash can have unfrozen string keys https://bugs.ruby-lang.org/issues/20307 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- I don't think this behavior is expected. ```ruby i = Hash.new.compare_by_identity k = "a" i[k] = 0 h = {}.update(i) p h["a"] # => 0 k.upcase! p h.fetch(k) # in 'Hash#fetch': key not found: "A" (KeyError) ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:116643] [Ruby master Bug#20250] Crash with "Object ID seen, but not in mapping table: proc" error
by zetter (Chris Zetter) 13 Jul '24

13 Jul '24

Issue #20250 has been reported by zetter (Chris Zetter). ---------------------------------------- Bug #20250: Crash with "Object ID seen, but not in mapping table: proc" error https://bugs.ruby-lang.org/issues/20250 * Author: zetter (Chris Zetter) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hello, I experienced a crash which I was able to reliably reproduce with the following: ``` ruby require 'bundler/inline' gemfile(true) do source 'https://rubygems.org' gem 'activesupport', '7.1.3' end require 'active_support' logger = ActiveSupport::Logger.new('log/log.log', 1, 100 * 1024 * 1024) logger.formatter = proc {|_, _, _, message| "#{message}\n" } logger = ActiveSupport::TaggedLogging.new(logger) logger.tagged("TAG").info "hello" logger.tagged("TAG").info "hello" # usually crashes here GC.start # sometimes need to trigger crash with GC ``` It looks like this is caused by the interaction with the formatter proc and the tagged logging features of activesupport. Let me know if a more minimal example would be useful. I can reproduce this with: * ruby 3.3.0 on arm64-darwin23 * ruby 3.3.0 on x86_64-linux * latest ruby HEAD (5e12b75716) on arm64-darwin23 I cannot reproduce this on ruby 3.2.2. Thanks for any help ---Files-------------------------------- ruby-2024-02-09-092829.ips (13.2 KB) output.log (142 KB) crash_test_2.rb (465 Bytes) -- https://bugs.ruby-lang.org/

4 5

[ruby-core:116597] [Ruby master Bug#20239] Segmentation fault when using Regex on a large String
by martinsp (Martins Polakovs) 09 Jul '24

09 Jul '24

Issue #20239 has been reported by martinsp (Martins Polakovs). ---------------------------------------- Bug #20239: Segmentation fault when using Regex on a large String https://bugs.ruby-lang.org/issues/20239 * Author: martinsp (Martins Polakovs) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since v3.2.0 ruby crashes with segmentation fault on the following script with a `[BUG] Segmentation fault at ...` ``` ruby require "rbconfig/sizeof" ("\u{0101}" + "a" * RbConfig::LIMITS["INT_MAX"] + "b").match(/b/) ``` Crash can be reproduced on the following ruby versions: - ruby 3.2.0 (2022-12-25 revision a528908271) [aarch64-linux] - ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [aarch64-linux] - ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [aarch64-linux] works as expected It seems that call to `enclen` inside `str_lower_case_match` returns negative offset in this case https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/v3… -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116078] [Ruby master Bug#20162] Memory leak when duplicating too complex object
by peterzhu2118 (Peter Zhu) 08 Jul '24

08 Jul '24

Issue #20162 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20162: Memory leak when duplicating too complex object https://bugs.ruby-lang.org/issues/20162 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9449 Creating a ST table then calling st_replace leaks memory because the st_replace overwrites the ST table without freeing any of the existing memory. This commit changes it to use st_copy instead. For example: ```ruby RubyVM::Shape.exhaust_shapes o = Object.new o.instance_variable_set(:@a, 0) 10.times do 100_000.times { o.dup } puts `ps -o rss= -p #{$$}` end ``` Before: ``` 23264 33600 42672 52160 61600 71728 81056 90528 100560 109840 ``` After: ``` 14752 14816 15584 15584 15664 15664 15664 15664 15664 15664 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114932] [Ruby master Bug#19907] Method calls with keyword arguments in eval leaks callcache and callinfo objects
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #19907 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19907: Method calls with keyword arguments in eval leaks callcache and callinfo objects https://bugs.ruby-lang.org/issues/19907 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script leaks callcache and callinfo objects: ```ruby def foo(a:); end 10.times do 10_000.times do eval(<<~RUBY) foo(a: 1) RUBY end puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output is: ``` Number of live objects: 46248 Memory usage: 16160 Number of live objects: 65902 Memory usage: 19888 Number of live objects: 92656 Memory usage: 24032 Number of live objects: 126791 Memory usage: 28048 Number of live objects: 132919 Memory usage: 28816 Number of live objects: 180687 Memory usage: 32384 Number of live objects: 181957 Memory usage: 32464 Number of live objects: 227485 Memory usage: 34224 Number of live objects: 256101 Memory usage: 37200 Number of live objects: 274151 Memory usage: 38752 ``` After performing a `ObjectSpace.dump_all`, I found that it is leaking callcache and callinfo objects that is being held on by the `Object` class. ```json {"address":"0x102ecef70", "type":"CLASS", "shape_id":2, "slot_size":160, "class":"0x102ecf8d0", "variation_count":0, "superclass":"0x102ecfd30", "real_class_name":"Object", "singleton":true, "references":[ ... ]} {"address":"0x1030c90a0", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callinfo", "mid":"foo", "memsize":40, "flags":{"wb_protected":true}} {"address":"0x102fad568", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callcache", "memsize":40, "flags":{"wb_protected":true, "old":true, "uncollectible":true, "marked":true}} ``` -- https://bugs.ruby-lang.org/

5 4

[ruby-core:115999] [Ruby master Bug#20145] Memory leak when duplicating identhash
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20145 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20145: Memory leak when duplicating identhash https://bugs.ruby-lang.org/issues/20145 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9411 Both copy_compare_by_id and hash_copy will create a copy of the ST table, so the ST table created in copy_compare_by_id will be leaked. ```ruby h = { 1 => 2 }.compare_by_identity 10.times do 1_000_000.times do h.select { false } end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 110736 204352 300272 395520 460704 476736 542000 604704 682624 770528 ``` After: ``` 15504 16048 16144 16256 16320 16320 16752 16752 16752 16752 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116514] [Ruby master Bug#20228] Memory leak in Regexp timeout
by peterzhu2118 (Peter Zhu) 07 Jul '24

07 Jul '24

Issue #20228 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20228: Memory leak in Regexp timeout https://bugs.ruby-lang.org/issues/20228 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9765 If a Regexp::TimeoutError is raised, the `stk_base` and `OnigRegion` will leak. For example: ```ruby Regexp.timeout = 0.001 regex = /^(a*)*$/ str = "a" * 1000000 + "x" 10.times do 100.times do begin regex =~ str rescue end end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 328800 632416 934368 1230448 1531088 1831248 2125072 2414384 2703440 2995664 ``` After: ``` 39280 47888 49024 56240 56496 56512 56592 56592 56720 56720 ``` -- https://bugs.ruby-lang.org/

5 5