May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:115549] [Ruby master Bug#20030] `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string.
by tompng (tomoya ishida) 15 Jul '24

15 Jul '24

Issue #20030 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20030: `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string. https://bugs.ruby-lang.org/issues/20030 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-11-30T16:23:25Z master d048bae96b) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ~~~ruby Ripper.tokenize '"\\C-あ"' # or Ripper.tokenize "\"\\C-\u3042\"" # => ["\"", "\x81", "\x82", "\""] ~~~ I expect all tokens to be valid_encoding if the source string is valid_encoding. This is causing IRB crash when typing `"\C-あ"`. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117992] [Ruby master Bug#20505] Reassigning the block argument in method body keeps old block when calling super with implicit arguments
by Earlopain (A S) 15 Jul '24

15 Jul '24

Issue #20505 has been reported by Earlopain (A S). ---------------------------------------- Bug #20505: Reassigning the block argument in method body keeps old block when calling super with implicit arguments https://bugs.ruby-lang.org/issues/20505 * Author: Earlopain (A S) * Status: Open * ruby -v: 3.3.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- You can call super without arguments and parenthesis to pass along all the enclosing method arguments to the parent method. You can modify positional and keyword arguments before the call to super, and the parent method will recieve these modified values. With the block arg however that isn't the case: ```rb class A def positional_arg(a) puts a end def block_arg(&block) yield end end class B < A def positional_arg(a = nil) a = 'b' super end def block_arg(&block) block = proc { puts 'b' } super end end B.new.positional_arg('a') B.new.positional_arg B.new.block_arg { puts 'a' } B.new.block_arg ``` I would expect this snippet to print `b` four times. The actual output is `b` `b` `a` and `LocalJumpError`. To get the desired output I must pass the block along explicitly with `super(&block)`. I hope my example explains the issue good enough. I have looked through issues here and searched for documentation and haven't found any mention of this behaviour. Sorry if I missed something somewhere. -- https://bugs.ruby-lang.org/

5 6

[ruby-core:117200] [Ruby master Bug#20342] Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load
by nobu (Nobuyoshi Nakada) 15 Jul '24

15 Jul '24

Issue #20342 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20342: Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load https://bugs.ruby-lang.org/issues/20342 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- With this file: ```ruby # load.rb public def f = :ok ``` It is OK when `require`d. ```sh-session $ ruby -r ./load.rb -e 'p f' :ok ``` Simple `load` is OK too. ```sh-session $ ruby -e 'load ARGV[0]; p f' load.rb :ok ``` Wrapped `load` fails. ```sh-session $ ruby -e 'load ARGV[0], true' load.rb load.rb:1:in 'public': undefined method 'f' for class 'Object' (NameError) public def f = :ok ^^^^^^ from load.rb:1:in '<top (required)>' from -e:1:in 'Kernel#load' from -e:1:in '<main>' ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:117697] [Ruby master Bug#20453] Pointer being freed was not allocated in Regexp timeout
by dodecadaniel (Daniel Colson) 15 Jul '24

15 Jul '24

Issue #20453 has been reported by dodecadaniel (Daniel Colson). ---------------------------------------- Bug #20453: Pointer being freed was not allocated in Regexp timeout https://bugs.ruby-lang.org/issues/20453 * Author: dodecadaniel (Daniel Colson) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- https://bugs.ruby-lang.org/issues/20228 frees `stk_base` to avoid a memory leak, but `stk_base` is sometimes stack allocated ([see `xalloca`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1177-L1181)). So the free only works if the regex stack grows enough that it needs to double ([see `xmalloc` and `xrealloc` in `stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1210-L1249). Reproduction: ```ruby Regexp.timeout = 0.001 /^(a*)x$/ =~ "a" * 1000000 + "x"' ``` I'll open a PR shortly. https://bugs.ruby-lang.org/issues/20228 was backported to 3.3.1, so this bug affects that version as well. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116950] [Ruby master Bug#20304] Memory leak when setting Encoding.default_internal
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20304 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20304: Memory leak when setting Encoding.default_internal https://bugs.ruby-lang.org/issues/20304 * Author: MaxLap (Maxime Lapointe) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Problem is present in Ruby 3.2.2, 3.2.3, 3.3.0. Didn't check before. Put this in a file: ``` 10.times do 100000.times do Encoding.default_internal = nil end puts `ps -o rss= -p #{$$}`.to_i end ``` Result: ``` $ ruby local.rb 27044 30212 33116 36284 39452 42620 45788 48956 51860 55028 ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:117037] [Ruby master Bug#20322] rb_enc_interned_str_cstr doesn't accept null pointer for encoding
by thomasmarshall (Thomas Marshall) 15 Jul '24

15 Jul '24

Issue #20322 has been reported by thomasmarshall (Thomas Marshall). ---------------------------------------- Bug #20322: rb_enc_interned_str_cstr doesn't accept null pointer for encoding https://bugs.ruby-lang.org/issues/20322 * Author: thomasmarshall (Thomas Marshall) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [header documentation](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd… for `rb_enc_interned_str_cstr` notes: > `enc` can be a null pointer. When [adding support for this function to TruffleRuby](https://github.com/oracle/truffleruby/pull/3427) we noticed that the behaviour did not match that documentation. I think this did work previously, but it currently causes a segmentation fault when trying to [autoload the encoding](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd1281b… because it calls `rb_enc_mbmaxlen` which [expects a non-null encoding](https://github.com/ruby/ruby/blob/93556d46203545bc2364b1c0dd1281b…. I'm not sure how important this behaviour is, whether extensions ever actually call this with a null pointer, or if the documentation is instead just incorrect. I think it's a straightforward fix to make the behaviour match the documentation, so I would like to open a PR, but also happy to just remove the note if that is preferable. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:117531] [Ruby master Bug#20431] Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1
by shubham_yadav (Shubham Yadav) 15 Jul '24

15 Jul '24

Issue #20431 has been reported by shubham_yadav (Shubham Yadav). ---------------------------------------- Bug #20431: Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1 https://bugs.ruby-lang.org/issues/20431 * Author: shubham_yadav (Shubham Yadav) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I'm trying to build ruby from scratch with openss1.1.0. But it's failing with below error. In file included from ./include/ruby/internal/arithmetic/char.h:23, from ./include/ruby/internal/arithmetic.h:24, from ./include/ruby/ruby.h:28, from ./include/ruby/encoding.h:19, from ./include/ruby/io.h:16, from io_buffer.c:9: ./include/ruby/internal/arithmetic/int.h: In function ‘rb_int2num_inline’: ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h: In function ‘rb_uint2num_inline’: ./include/ruby/internal/arithmetic/int.h:256: warning: comparison is always true due to limited range of data type io_buffer.c: In function ‘io_buffer_read_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c:1872: error: (Each undeclared identifier is reported only once io_buffer.c:1872: error: for each function it appears in.) io_buffer.c: In function ‘io_buffer_write_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘Init_IO_Buffer’: io_buffer.c:3702: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) At top level: cc1: warning: unrecognized command line option "-Wno-maybe-uninitialized" cc1: warning: unrecognized command line option "-Wno-tautological-compare" cc1: warning: unrecognized command line option "-Wno-self-assign" cc1: warning: unrecognized command line option "-Wno-parentheses-equality" cc1: warning: unrecognized command line option "-Wno-constant-logical-operand" cc1: warning: unrecognized command line option "-Wno-cast-function-type" make: *** [io_buffer.o] Error 1 -- https://bugs.ruby-lang.org/

6 10

[ruby-core:116114] [Ruby master Bug#20169] `GC.compact` can raises `EFAULT` on IO
by ko1 (Koichi Sasada) 15 Jul '24

15 Jul '24

Issue #20169 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20169: `GC.compact` can raises `EFAULT` on IO https://bugs.ruby-lang.org/issues/20169 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- 1. `GC.compact` introduces read barriers to detect read accesses to the pages. 2. I/O operations release GVL to pass the control while their execution, and another thread can call `GC.compact` (or auto compact feature I guess, but not checked yet). 3. Call `write(ptr)` can return `EFAULT` when `GC.compact` is running because `ptr` points read-barrier protected pages. Reproducible steps: Apply the following patch to increase possibility: ```patch diff --git a/io.c b/io.c index f6cd2c1a56..83d67ba2dc 100644 --- a/io.c +++ b/io.c @@ -1212,8 +1212,12 @@ internal_write_func(void *ptr) } } + int cnt = 0; retry: - do_write_retry(write(iis->fd, iis->buf, iis->capa)); + for (; cnt < 1000; cnt++) { + do_write_retry(write(iis->fd, iis->buf, iis->capa)); + if (result <= 0) break; + } if (result < 0 && !iis->nonblock) { int e = errno; ``` Run the following code: ```ruby t1 = Thread.new{ 10_000.times.map{"#{_1}"}; GC.compact while true } t2 = Thread.new{ i=0 $stdout.write "<#{i+=1}>" while true } t2.join ``` and ``` $ make run (snip) 4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4>#<Thread:0x00007fa61b4dd758 ../../src/trunk/test.rb:3 run> terminated with exception (report_on_exception is true): ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' make: *** [uncommon.mk:1383: run] Error 1 ``` I think this is why we get `EFAULT` on CI. To increase possibilities running many busy processes (`ruby -e 'loop{}'` for example) will help (and on CI environment there are such busy processes accidentally). -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116908] [Ruby master Bug#20292] Abort ruby by `String#initialize`
by ksss (Yuki Kurihara) 15 Jul '24

15 Jul '24

Issue #20292 has been reported by ksss (Yuki Kurihara). ---------------------------------------- Bug #20292: Abort ruby by `String#initialize` https://bugs.ruby-lang.org/issues/20292 * Author: ksss (Yuki Kurihara) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found a code that causes an anomaly. ``` $ uname -a Darwin 20208671n 22.6.0 Darwin Kernel Version 22.6.0: Wed Oct 4 21:26:55 PDT 2023; root:xnu-8796.141.3.701.17~4/RELEASE_ARM64_T6020 arm64 $ ruby -v ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: -1) }' ruby(13817,0x1f22b2080) malloc: Heap corruption detected, free list is damaged at 0x600000e27bc0 *** Incorrect guard value: 29830901955328 ruby(13817,0x1f22b2080) malloc: *** set a breakpoint in malloc_error_break to debug [1] 13817 abort ruby -e $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: 0) }' # never return .... ``` My research indicates the following conditions. - String must be at least 64 characters long - Call `#initialize` with capacity: 0 or less - Repeat this several times. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116930] [Ruby master Bug#20296] Complex(:sym, exception: false) generate exception with weird timing
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20296 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20296: Complex(:sym, exception: false) generate exception with weird timing https://bugs.ruby-lang.org/issues/20296 * Author: MaxLap (Maxime Lapointe) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Happens in Ruby 3.2.2, 3.2.3 and 3.3.0 (didn't test others). Add this to a file and execute it as-is: ``` 3.times { Complex(:sym, exception: false) } puts "hi" ``` The output will be: ``` $ ruby local2.rb hi local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' ``` So the exception is still reported (instead of being swallow), but the puts, which is after, had the time to execute first. I can even put a `sleep()` in between and the exceptions are still reported after the puts. In a IRB, there is no exception at all and everything seems to work as expected. Happens if I pass in an Array too. Doesn't seem to happen with bad strings. There is also a quadratic behavior somewhere in there. Running the code in an IRB (which doesn't show exceptions), if you double the number of `times` in the loop, it takes 4x as long. This is probably related... -- https://bugs.ruby-lang.org/

3 2