June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:117200] [Ruby master Bug#20342] Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load
by nobu (Nobuyoshi Nakada) 15 Jul '24

15 Jul '24

Issue #20342 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20342: Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load https://bugs.ruby-lang.org/issues/20342 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- With this file: ```ruby # load.rb public def f = :ok ``` It is OK when `require`d. ```sh-session $ ruby -r ./load.rb -e 'p f' :ok ``` Simple `load` is OK too. ```sh-session $ ruby -e 'load ARGV[0]; p f' load.rb :ok ``` Wrapped `load` fails. ```sh-session $ ruby -e 'load ARGV[0], true' load.rb load.rb:1:in 'public': undefined method 'f' for class 'Object' (NameError) public def f = :ok ^^^^^^ from load.rb:1:in '<top (required)>' from -e:1:in 'Kernel#load' from -e:1:in '<main>' ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:117697] [Ruby master Bug#20453] Pointer being freed was not allocated in Regexp timeout
by dodecadaniel (Daniel Colson) 15 Jul '24

15 Jul '24

Issue #20453 has been reported by dodecadaniel (Daniel Colson). ---------------------------------------- Bug #20453: Pointer being freed was not allocated in Regexp timeout https://bugs.ruby-lang.org/issues/20453 * Author: dodecadaniel (Daniel Colson) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- https://bugs.ruby-lang.org/issues/20228 frees `stk_base` to avoid a memory leak, but `stk_base` is sometimes stack allocated ([see `xalloca`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1177-L1181)). So the free only works if the regex stack grows enough that it needs to double ([see `xmalloc` and `xrealloc` in `stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1210-L1249). Reproduction: ```ruby Regexp.timeout = 0.001 /^(a*)x$/ =~ "a" * 1000000 + "x"' ``` I'll open a PR shortly. https://bugs.ruby-lang.org/issues/20228 was backported to 3.3.1, so this bug affects that version as well. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118299] [Ruby master Feature#20576] Add MatchData#bytebegin and MatchData#byteend
by shugo (Shugo Maeda) 15 Jul '24

15 Jul '24

Issue #20576 has been reported by shugo (Shugo Maeda). ---------------------------------------- Feature #20576: Add MatchData#bytebegin and MatchData#byteend https://bugs.ruby-lang.org/issues/20576 * Author: shugo (Shugo Maeda) * Status: Open * Target version: 3.4 ---------------------------------------- I'd like to propose MatchData#bytebegin and MatchData#byteend. These methods are similar to MatchData#begin and MatchData#end, but returns offsets in bytes instead of codepoints. Pull request: https://github.com/ruby/ruby/pull/10973 One of the use cases is scanning strings: https://github.com/ruby/net-imap/pull/286/files MatchData#byteend is faster than MatchData#byteoffset because there is no need to allocate an Array. Here's a benchmark result: ``` voyager:ruby$ cat b.rb require "benchmark" require "strscan" text = "あ" * 100000 Benchmark.bmbm do |b| b.report("byteoffset(0)[1]") do pos = 0 while text.byteindex(/\G./, pos) pos = $~.byteoffset(0)[1] end end b.report("byteend(0)") do pos = 0 while text.byteindex(/\G./, pos) pos = $~.byteend(0) end end end voyager:ruby$ ./tool/runruby.rb b.rb Rehearsal ---------------------------------------------------- byteoffset(0)[1] 0.020558 0.000393 0.020951 ( 0.020963) byteend(0) 0.018149 0.000000 0.018149 ( 0.018151) ------------------------------------------- total: 0.039100sec user system total real byteoffset(0)[1] 0.020821 0.000000 0.020821 ( 0.020822) byteend(0) 0.017455 0.000000 0.017455 ( 0.017455) ``` -- https://bugs.ruby-lang.org/

3 5

[ruby-core:116950] [Ruby master Bug#20304] Memory leak when setting Encoding.default_internal
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20304 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20304: Memory leak when setting Encoding.default_internal https://bugs.ruby-lang.org/issues/20304 * Author: MaxLap (Maxime Lapointe) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Problem is present in Ruby 3.2.2, 3.2.3, 3.3.0. Didn't check before. Put this in a file: ``` 10.times do 100000.times do Encoding.default_internal = nil end puts `ps -o rss= -p #{$$}`.to_i end ``` Result: ``` $ ruby local.rb 27044 30212 33116 36284 39452 42620 45788 48956 51860 55028 ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:117037] [Ruby master Bug#20322] rb_enc_interned_str_cstr doesn't accept null pointer for encoding
by thomasmarshall (Thomas Marshall) 15 Jul '24

15 Jul '24

Issue #20322 has been reported by thomasmarshall (Thomas Marshall). ---------------------------------------- Bug #20322: rb_enc_interned_str_cstr doesn't accept null pointer for encoding https://bugs.ruby-lang.org/issues/20322 * Author: thomasmarshall (Thomas Marshall) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [header documentation](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd… for `rb_enc_interned_str_cstr` notes: > `enc` can be a null pointer. When [adding support for this function to TruffleRuby](https://github.com/oracle/truffleruby/pull/3427) we noticed that the behaviour did not match that documentation. I think this did work previously, but it currently causes a segmentation fault when trying to [autoload the encoding](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd1281b… because it calls `rb_enc_mbmaxlen` which [expects a non-null encoding](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd1281b…. I'm not sure how important this behaviour is, whether extensions ever actually call this with a null pointer, or if the documentation is instead just incorrect. I think it's a straightforward fix to make the behaviour match the documentation, so I would like to open a PR, but also happy to just remove the note if that is preferable. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:117531] [Ruby master Bug#20431] Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1
by shubham_yadav (Shubham Yadav) 15 Jul '24

15 Jul '24

Issue #20431 has been reported by shubham_yadav (Shubham Yadav). ---------------------------------------- Bug #20431: Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1 https://bugs.ruby-lang.org/issues/20431 * Author: shubham_yadav (Shubham Yadav) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I'm trying to build ruby from scratch with openss1.1.0. But it's failing with below error. In file included from ./include/ruby/internal/arithmetic/char.h:23, from ./include/ruby/internal/arithmetic.h:24, from ./include/ruby/ruby.h:28, from ./include/ruby/encoding.h:19, from ./include/ruby/io.h:16, from io_buffer.c:9: ./include/ruby/internal/arithmetic/int.h: In function ‘rb_int2num_inline’: ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h: In function ‘rb_uint2num_inline’: ./include/ruby/internal/arithmetic/int.h:256: warning: comparison is always true due to limited range of data type io_buffer.c: In function ‘io_buffer_read_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c:1872: error: (Each undeclared identifier is reported only once io_buffer.c:1872: error: for each function it appears in.) io_buffer.c: In function ‘io_buffer_write_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘Init_IO_Buffer’: io_buffer.c:3702: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) At top level: cc1: warning: unrecognized command line option "-Wno-maybe-uninitialized" cc1: warning: unrecognized command line option "-Wno-tautological-compare" cc1: warning: unrecognized command line option "-Wno-self-assign" cc1: warning: unrecognized command line option "-Wno-parentheses-equality" cc1: warning: unrecognized command line option "-Wno-constant-logical-operand" cc1: warning: unrecognized command line option "-Wno-cast-function-type" make: *** [io_buffer.o] Error 1 -- https://bugs.ruby-lang.org/

6 10

[ruby-core:116114] [Ruby master Bug#20169] `GC.compact` can raises `EFAULT` on IO
by ko1 (Koichi Sasada) 15 Jul '24

15 Jul '24

Issue #20169 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20169: `GC.compact` can raises `EFAULT` on IO https://bugs.ruby-lang.org/issues/20169 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- 1. `GC.compact` introduces read barriers to detect read accesses to the pages. 2. I/O operations release GVL to pass the control while their execution, and another thread can call `GC.compact` (or auto compact feature I guess, but not checked yet). 3. Call `write(ptr)` can return `EFAULT` when `GC.compact` is running because `ptr` points read-barrier protected pages. Reproducible steps: Apply the following patch to increase possibility: ```patch diff --git a/io.c b/io.c index f6cd2c1a56..83d67ba2dc 100644 --- a/io.c +++ b/io.c @@ -1212,8 +1212,12 @@ internal_write_func(void *ptr) } } + int cnt = 0; retry: - do_write_retry(write(iis->fd, iis->buf, iis->capa)); + for (; cnt < 1000; cnt++) { + do_write_retry(write(iis->fd, iis->buf, iis->capa)); + if (result <= 0) break; + } if (result < 0 && !iis->nonblock) { int e = errno; ``` Run the following code: ```ruby t1 = Thread.new{ 10_000.times.map{"#{_1}"}; GC.compact while true } t2 = Thread.new{ i=0 $stdout.write "<#{i+=1}>" while true } t2.join ``` and ``` $ make run (snip) 4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4>#<Thread:0x00007fa61b4dd758 ../../src/trunk/test.rb:3 run> terminated with exception (report_on_exception is true): ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' make: *** [uncommon.mk:1383: run] Error 1 ``` I think this is why we get `EFAULT` on CI. To increase possibilities running many busy processes (`ruby -e 'loop{}'` for example) will help (and on CI environment there are such busy processes accidentally). -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116908] [Ruby master Bug#20292] Abort ruby by `String#initialize`
by ksss (Yuki Kurihara) 15 Jul '24

15 Jul '24

Issue #20292 has been reported by ksss (Yuki Kurihara). ---------------------------------------- Bug #20292: Abort ruby by `String#initialize` https://bugs.ruby-lang.org/issues/20292 * Author: ksss (Yuki Kurihara) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found a code that causes an anomaly. ``` $ uname -a Darwin 20208671n 22.6.0 Darwin Kernel Version 22.6.0: Wed Oct 4 21:26:55 PDT 2023; root:xnu-8796.141.3.701.17~4/RELEASE_ARM64_T6020 arm64 $ ruby -v ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: -1) }' ruby(13817,0x1f22b2080) malloc: Heap corruption detected, free list is damaged at 0x600000e27bc0 *** Incorrect guard value: 29830901955328 ruby(13817,0x1f22b2080) malloc: *** set a breakpoint in malloc_error_break to debug [1] 13817 abort ruby -e $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: 0) }' # never return .... ``` My research indicates the following conditions. - String must be at least 64 characters long - Call `#initialize` with capacity: 0 or less - Repeat this several times. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116930] [Ruby master Bug#20296] Complex(:sym, exception: false) generate exception with weird timing
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20296 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20296: Complex(:sym, exception: false) generate exception with weird timing https://bugs.ruby-lang.org/issues/20296 * Author: MaxLap (Maxime Lapointe) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Happens in Ruby 3.2.2, 3.2.3 and 3.3.0 (didn't test others). Add this to a file and execute it as-is: ``` 3.times { Complex(:sym, exception: false) } puts "hi" ``` The output will be: ``` $ ruby local2.rb hi local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' ``` So the exception is still reported (instead of being swallow), but the puts, which is after, had the time to execute first. I can even put a `sleep()` in between and the exceptions are still reported after the puts. In a IRB, there is no exception at all and everything seems to work as expected. Happens if I pass in an Array too. Doesn't seem to happen with bad strings. There is also a quadratic behavior somewhere in there. Running the code in an IRB (which doesn't show exceptions), if you double the number of `times` in the loop, it takes 4x as long. This is probably related... -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116965] [Ruby master Bug#20307] `Hash#update` from compare_by_identity hash can have unfrozen string keys
by nobu (Nobuyoshi Nakada) 13 Jul '24

13 Jul '24

Issue #20307 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20307: `Hash#update` from compare_by_identity hash can have unfrozen string keys https://bugs.ruby-lang.org/issues/20307 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- I don't think this behavior is expected. ```ruby i = Hash.new.compare_by_identity k = "a" i[k] = 0 h = {}.update(i) p h["a"] # => 0 k.upcase! p h.fetch(k) # in 'Hash#fetch': key not found: "A" (KeyError) ``` -- https://bugs.ruby-lang.org/

5 6