May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:117453] [Ruby master Bug#20413] Enumerator can block fiber scheduler.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20413 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20413: Enumerator can block fiber scheduler. https://bugs.ruby-lang.org/issues/20413 * Author: ioquatix (Samuel Williams) * Status: Closed * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Using `Enumerator` in the event loop can cause problems as the fiber created by `rb_fiber_new` is blocking by default. It should be non-blocking. See <https://github.com/ruby/ruby/pull/10478> for the fix. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117458] [Ruby master Bug#20414] `Fiber#raise` should recurse to `resumed_fiber` rather than failing.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20414 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20414: `Fiber#raise` should recurse to `resumed_fiber` rather than failing. https://bugs.ruby-lang.org/issues/20414 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following program will fail with `FiberError`, and is difficult to properly clean up: ```ruby root_fiber = Fiber.current f1 = Fiber.new do root_fiber.transfer end f2 = Fiber.new do f1.resume end f2.transfer f2.raise("error") # => `raise': attempt to transfer to a resuming fiber (FiberError) ``` This program deliberately set's up a scenario where `f2` is resuming `f1`. Trying to raise an exception on `f2` is impossible, because the only way control flow can return to it, is when `f1` yields or exits. We can avoid this problem, by raising the exception on f1, and we can do this automatically using the following logic: ```c static VALUE fiber_raise(rb_fiber_t *fiber, VALUE exception) { // Add this recursive step: if (fiber->resuming_fiber) { return fiber_raise(fiber->resuming_fiber, exception); } // Existing code ... else if (FIBER_SUSPENDED_P(fiber) && !fiber->yielding) { return fiber_transfer_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } else { return fiber_resume_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } } ``` This makes `Fiber#raise` much more robust and useful for the purpose of stopping fibers, without knowing exactly what they are doing. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:117947] [Ruby master Bug#20499] Ruby builds on macOS store absolute paths for AR and NM in rbconfig since Ruby 3.2.3/3.3.0
by Bo98 (Bo Anderson) 06 Jul '24

06 Jul '24

Issue #20499 has been reported by Bo98 (Bo Anderson). ---------------------------------------- Bug #20499: Ruby builds on macOS store absolute paths for AR and NM in rbconfig since Ruby 3.2.3/3.3.0 https://bugs.ruby-lang.org/issues/20499 * Author: Bo98 (Bo Anderson) * Status: Open * ruby -v: ruby 3.2.4 (2024-04-23 revision af471c0e01) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is a regression introduced by https://github.com/ruby/ruby/commit/038f9ade3c4d965415e4956561975454cf9eeb21 and causes various downstream problems such as https://github.com/ruby/prism/issues/2716. Development tools on macOS are relocatable - Xcode.app can exist anywhere on the system so it cannot be assumed that the location on the build machine matches the location on the running machine. If an absolute path must be used, `/usr/bin/nm` and `/usr/bin/ar` are acceptable. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115937] [Ruby master Bug#20099] Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux)
by jirutka (Jakub Jirutka) 01 Jul '24

01 Jul '24

Issue #20099 has been reported by jirutka (Jakub Jirutka). ---------------------------------------- Bug #20099: Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux) https://bugs.ruby-lang.org/issues/20099 * Author: jirutka (Jakub Jirutka) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr * exec prefix: ${prefix} * arch: s390x-linux-musl * site arch: ${arch} * RUBY_BASE_NAME: ruby * enable shared: yes * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: linux-musl * compiler: gcc * with thread: pthread * with coroutine: ucontext * enable shared libs: yes * dynamic library ext: so * CFLAGS: ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -Wl,--as-needed,-O1,--sort-common \ -fstack-protector-strong -rdynamic \ -Wl,-export-dynamic -Wl,--no-as-needed * DLDFLAGS: -Wl,--as-needed,-O1,--sort-common \ -Wl,--compress-debug-sections=zlib * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wdeprecated-declarations \ -Wdiv-by-zero -Wduplicated-cond \ -Wimplicit-function-declaration -Wimplicit-int \ -Wpointer-arith -Wwrite-strings \ -Wold-style-definition -Wimplicit-fallthrough=0 \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings \ -Wno-packed-bitfield-compat \ -Wno-parentheses-equality -Wno-self-assign \ -Wno-tautological-compare -Wno-unused-parameter \ -Wno-unused-value -Wsuggest-attribute=format \ -Wsuggest-attribute=noreturn -Wunused-variable \ -Wmisleading-indentation -Wundef * strip command: strip -S -x * install doc: rdoc * YJIT support: no * RJIT support: no * man page type: man * search path: /usr/lib/site_ruby/$(ruby_ver)/s390x-linux --- BASERUBY = ./tool/missing-baseruby.bat CC = gcc LD = ld LDSHARED = gcc -shared CFLAGS = -O2 -fstack-clash-protection -Wformat -Werror=format-security -g -fno-omit-frame-pointer -fno-strict-aliasing -fPIC XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/s390x-linux-musl -I./include -I. -I./prism -I./enc/unicode/15.0.0 CPPFLAGS = -fno-omit-frame-pointer -fno-strict-aliasing DLDFLAGS = -Wl,--as-needed,-O1,--sort-common -Wl,--compress-debug-sections=zlib -Wl,-soname,libruby.so.3.3 -fstack-protector-strong SOLIBS = -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread -lucontext LANG = LC_ALL = LC_CTYPE = MFLAGS = -j8 --jobserver-auth=fifo:/tmp/GMfifo7323 RUSTC = no YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/builds/alpine/aports/main/ruby/src/ruby-3.3.0/yjit/target/release/' ./yjit/src/lib.rs ``` ``` 2023-12-26 20:45:17 +0000 Driver is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] Target is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] .............................................................................. .............................................................................. .............................................................................. .............................................................................. ................................................................EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:18Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:18Z", "-ex", "bt", "-batch"] ...EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:19Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:19Z", "-ex", "bt", "-batch"] .....Fstderr output is not empty [BUG] Segmentation fault at 0x000007fece67a000 ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] -- Control frame information ----------------------------------------------- c:0001 p:---- s:0003 e:000002 DUMMY [FINISH] -- Threading information --------------------------------------------------- Total ractor count: 2 Ruby thread count for this ractor: 1 -- Other runtime information ----------------------------------------------- * Loaded script: bootstraptest.test_ractor.rb_244_1238.rb * Loaded features: 0 enumerator.so 1 thread.rb 2 fiber.so 3 rational.so 4 complex.so 5 ruby2_keywords.rb 6 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 7 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so * Process memory map: 2aa25e00000-2aa25e01000 r-xp 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e01000-2aa25e02000 r--p 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e02000-2aa25e03000 rw-p 00001000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa2765d000-2aa2765e000 ---p 00000000 00:00 0 [heap] 2aa2765e000-2aa27664000 rw-p 00000000 00:00 0 [heap] 3ff572a8000-3ff572aa000 ---p 00000000 00:00 0 3ff572aa000-3ff572d0000 rw-p 00000000 00:00 0 3ff572d0000-3ff572d2000 ---p 00000000 00:00 0 3ff572d2000-3ff573f9000 rw-p 00000000 00:00 0 3ff573f9000-3ff573fa000 ---p 00000000 00:00 0 3ff573fa000-3ff5751a000 rw-p 00000000 00:00 0 3ff5751a000-3ff5751b000 ---p 00000000 00:00 0 3ff5751b000-3ff77300000 rw-p 00000000 00:00 0 3ff77300000-3ff77302000 r-xp 00000000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77302000-3ff77303000 r--p 00002000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77303000-3ff77304000 rw-p 00003000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77307000-3ff77309000 rw-p 00000000 00:00 0 3ff7730b000-3ff7730e000 rw-p 00000000 00:00 0 3ff77310000-3ff77330000 rw-p 00000000 00:00 0 3ff77332000-3ff77337000 rw-p 00000000 00:00 0 3ff77339000-3ff7733a000 rw-p 00000000 00:00 0 3ff7733b000-3ff7733e000 rw-p 00000000 00:00 0 3ff77340000-3ff77380000 rw-p 00000000 00:00 0 3ff77380000-3ff77382000 r-xp 00000000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77382000-3ff77383000 r--p 00001000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77383000-3ff77384000 rw-p 00002000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77384000-3ff7739c000 rw-p 00000000 00:00 0 3ff7739c000-3ff7739d000 ---p 00000000 00:00 0 3ff7739d000-3ff773de000 rw-p 00000000 00:00 0 3ff773de000-3ff773df000 ---p 00000000 00:00 0 3ff773df000-3ff77420000 rw-p 00000000 00:00 0 3ff77420000-3ff77421000 ---p 00000000 00:00 0 3ff77421000-3ff77462000 rw-p 00000000 00:00 0 3ff77462000-3ff77463000 ---p 00000000 00:00 0 3ff77463000-3ff774a4000 rw-p 00000000 00:00 0 3ff774a4000-3ff774a5000 ---p 00000000 00:00 0 3ff774a5000-3ff774e6000 rw-p 00000000 00:00 0 3ff774e6000-3ff774e7000 ---p 00000000 00:00 0 3ff774e7000-3ff77528000 rw-p 00000000 00:00 0 3ff77528000-3ff77529000 ---p 00000000 00:00 0 3ff77529000-3ff7756a000 rw-p 00000000 00:00 0 3ff7756a000-3ff7756b000 ---p 00000000 00:00 0 3ff7756b000-3ff775ac000 rw-p 00000000 00:00 0 3ff775ac000-3ff775ad000 ---p 00000000 00:00 0 3ff775ad000-3ff775ee000 rw-p 00000000 00:00 0 3ff775ee000-3ff775ef000 ---p 00000000 00:00 0 3ff775ef000-3ff77630000 rw-p 00000000 00:00 0 3ff77630000-3ff77631000 ---p 00000000 00:00 0 3ff77631000-3ff77672000 rw-p 00000000 00:00 0 3ff77672000-3ff77673000 ---p 00000000 00:00 0 3ff77673000-3ff776b4000 rw-p 00000000 00:00 0 3ff776b4000-3ff776b5000 ---p 00000000 00:00 0 3ff776b5000-3ff776f6000 rw-p 00000000 00:00 0 3ff776f6000-3ff776f7000 ---p 00000000 00:00 0 3ff776f7000-3ff77738000 rw-p 00000000 00:00 0 3ff77738000-3ff77739000 ---p 00000000 00:00 0 3ff77739000-3ff7777a000 rw-p 00000000 00:00 0 3ff7777a000-3ff7777b000 ---p 00000000 00:00 0 3ff7777b000-3ff777bc000 rw-p 00000000 00:00 0 3ff777bc000-3ff777bd000 ---p 00000000 00:00 0 3ff777bd000-3ff777fe000 rw-p 00000000 00:00 0 3ff777fe000-3ff777ff000 ---p 00000000 00:00 0 3ff777ff000-3ff77840000 rw-p 00000000 00:00 0 3ff77840000-3ff77841000 ---p 00000000 00:00 0 3ff77841000-3ff77882000 rw-p 00000000 00:00 0 3ff77882000-3ff77883000 ---p 00000000 00:00 0 3ff77883000-3ff778c4000 rw-p 00000000 00:00 0 3ff778c4000-3ff778c5000 ---p 00000000 00:00 0 3ff778c5000-3ff77906000 rw-p 00000000 00:00 0 3ff77906000-3ff77907000 ---p 00000000 00:00 0 3ff77907000-3ff77948000 rw-p 00000000 00:00 0 3ff77948000-3ff77949000 ---p 00000000 00:00 0 3ff77949000-3ff7798a000 rw-p 00000000 00:00 0 3ff7798a000-3ff7798b000 ---p 00000000 00:00 0 3ff7798b000-3ff779cc000 rw-p 00000000 00:00 0 3ff779cc000-3ff779cd000 ---p 00000000 00:00 0 3ff779cd000-3ff77a0e000 rw-p 00000000 00:00 0 3ff77a0e000-3ff77a0f000 ---p 00000000 00:00 0 3ff77a0f000-3ff77a50000 rw-p 00000000 00:00 0 3ff77a50000-3ff77a51000 ---p 00000000 00:00 0 3ff77a51000-3ff77a92000 rw-p 00000000 00:00 0 3ff77a92000-3ff77a93000 ---p 00000000 00:00 0 3ff77a93000-3ff77ad4000 rw-p 00000000 00:00 0 3ff77ad4000-3ff77ad5000 ---p 00000000 00:00 0 3ff77ad5000-3ff77b16000 rw-p 00000000 00:00 0 3ff77b16000-3ff77b17000 ---p 00000000 00:00 0 3ff77b17000-3ff77b58000 rw-p 00000000 00:00 0 3ff77b58000-3ff77b59000 ---p 00000000 00:00 0 3ff77b59000-3ff77b9a000 rw-p 00000000 00:00 0 3ff77b9a000-3ff77b9b000 ---p 00000000 00:00 0 3ff77b9b000-3ff77bdc000 rw-p 00000000 00:00 0 3ff77bdc000-3ff77bde000 ---p 00000000 00:00 0 3ff77bde000-3ff91100000 rw-p 00000000 00:00 0 3ff91100000-3ff91101000 r-xp 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91101000-3ff91102000 r--p 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91102000-3ff91103000 rw-p 00001000 fe:00 2758431 /lib/libucontext.so.1 3ff91103000-3ff9114b000 rw-p 00000000 00:00 0 3ff9114f000-3ff91153000 rw-p 00000000 00:00 0 3ff91155000-3ff91180000 rw-p 00000000 00:00 0 3ff91180000-3ff911fd000 r-xp 00000000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911fd000-3ff911ff000 r--p 0007c000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911ff000-3ff91200000 rw-p 0007e000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff91200000-3ff91219000 r-xp 00000000 fe:00 2754145 /lib/libz.so.1.3 3ff91219000-3ff9121a000 r--p 00018000 fe:00 2754145 /lib/libz.so.1.3 3ff9121a000-3ff9121b000 rw-p 00019000 fe:00 2754145 /lib/libz.so.1.3 3ff9121b000-3ff91280000 rw-p 00000000 00:00 0 3ff91280000-3ff91700000 r-xp 00000000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff91700000-3ff9170c000 r--p 00480000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170c000-3ff9170d000 rw-p 0048c000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170d000-3ff9171f000 rw-p 00000000 00:00 0 3ff9171f000-3ff91780000 rw-p 00000000 00:00 0 3ff91780000-3ff91830000 r-xp 00000000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91830000-3ff91831000 r--p 000b0000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91831000-3ff91832000 rw-p 000b1000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91832000-3ff91835000 rw-p 00000000 00:00 0 3ff91835000-3ff9187c000 rw-p 00000000 00:00 0 3ff9187c000-3ff9187e000 r--p 00000000 00:00 0 [vvar] 3ff9187e000-3ff91880000 r-xp 00000000 00:00 0 [vdso] 3ffe2381000-3ffe2b80000 rw-p 00000000 00:00 0 [stack] ``` …and many more tests fail like this. ---Files-------------------------------- ruby-3.3.0-s390x.log (281 KB) -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118128] [Ruby master Bug#20516] The version of rexml in ruby 3.3.2 has not been updated since 3.2.6.
by naitoh (Jun NAITOH) 21 Jun '24

21 Jun '24

Issue #20516 has been reported by naitoh (Jun NAITOH). ---------------------------------------- Bug #20516: The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. https://bugs.ruby-lang.org/issues/20516 * Author: naitoh (Jun NAITOH) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. This is still a DoS vulnerable version. https://www.ruby-lang.org/en/news/2024/05/16/dos-rexml-cve-2024-35176/ ``` $ ruby -v ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] $ gem list rexml *** LOCAL GEMS *** rexml (3.2.6) ``` -- https://bugs.ruby-lang.org/

5 12

[ruby-core:117746] [Ruby master Bug#20462] Native threads are no longer reused
by tenderlovemaking (Aaron Patterson) 20 Jun '24

20 Jun '24

Issue #20462 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20462: Native threads are no longer reused https://bugs.ruby-lang.org/issues/20462 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ruby used to reuse native threads in order to amortize the cost of making a pthread. For example this program: ```ruby ntids = 1000.times.map { Thread.new { Thread.current.native_thread_id }.value } p ntids.uniq.length ``` With Ruby 3.2.0, this would return 1. With Ruby 3.3.x, it returns 1000. It means we cannot amortize the cost of a pthread for short lived threads. I was able to bisect this to commit be1bbd5b7d40ad863ab35097765d3754726bbd54. But the change is big so I don't know how to fix it. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:117498] [Ruby master Feature#20425] Optimize forwarding callers and callees
by tenderlovemaking (Aaron Patterson) 18 Jun '24

18 Jun '24

Issue #20425 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #20425: Optimize forwarding callers and callees https://bugs.ruby-lang.org/issues/20425 * Author: tenderlovemaking (Aaron Patterson) * Status: Open ---------------------------------------- [This PR](https://github.com/ruby/ruby/pull/10510) optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` ## How does it work? This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. ## Why? I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. -- https://bugs.ruby-lang.org/

2 13

[ruby-core:117918] [Ruby master Bug#20494] Non-default directories are not searched when checking for a gmp header
by lish82 (Hiroki Katagiri) 15 Jun '24

15 Jun '24

Issue #20494 has been reported by lish82 (Hiroki Katagiri). ---------------------------------------- Bug #20494: Non-default directories are not searched when checking for a gmp header https://bugs.ruby-lang.org/issues/20494 * Author: lish82 (Hiroki Katagiri) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is my very first issue submission I would be happy to know if I missed anything --- I found that the ruby installed with `ruby-build` is not linked to the gmp library Environment: - OS: macOS Sonoma 14.5 - Directory that gmp is installed: /opt/homebrew (= installed via homebrew) I believe it was able to link to gmp before, so I tried to investigate the cause of the problem As a result, I found the followings: - During the execution of `./configure`, `checking for gmp.h...` is showing `no` - Specifying `--with-gmp-dir=/opt/homebrew` does not work - It has been occurring since 3.2.3 - When I built it on linux, it was successfully linked to gmp - gmp was installed in the default directory (`/usr/include`) - Revert the following changes and the problem goes away - https://github.com/ruby/ruby/pull/8449 Based on these observations, I believe the issue is that the directories specified with `--with-opt-dir` or `--with-gmp-dir` are not being included in the search path when checking for the gmp header I was able to solve the problem by changing the build process as follows ```patch diff --git a/configure.ac b/configure.ac index 18b4247..a440de6 100644 --- a/configure.ac +++ b/configure.ac @@ -1363,7 +1363,11 @@ AS_CASE("$target_cpu", [x64|x86_64|i[3-6]86*], [ RUBY_UNIVERSAL_CHECK_HEADER([x86_64, i386], x86intrin.h) AS_IF([test "x$with_gmp" != xno], - [AC_CHECK_HEADERS(gmp.h) + [original_CPPFLAGS="${CPPFLAGS}" + CPPFLAGS="${INCFLAGS} ${CPPFLAGS}" + AC_CHECK_HEADERS(gmp.h) + CPPFLAGS="${original_CPPFLAGS}" + unset original_CPPFLAGS AS_IF([test "x$ac_cv_header_gmp_h" != xno], AC_SEARCH_LIBS([__gmpz_init], [gmp], [AC_DEFINE(HAVE_LIBGMP, 1)]))]) ``` However, I am not familiar with ruby's build process or autotools, so I'm not sure if this is the correct approach --- I would appreciate it if someone could consider resolving this issue -- https://bugs.ruby-lang.org/

7 9

[ruby-core:118110] [Ruby master Bug#20515] --with-gmp is not working - GMP support won't be built
by sorah (Sorah Fukumori) 15 Jun '24

15 Jun '24

Issue #20515 has been reported by sorah (Sorah Fukumori). ---------------------------------------- Bug #20515: --with-gmp is not working - GMP support won't be built https://bugs.ruby-lang.org/issues/20515 * Author: sorah (Sorah Fukumori) * Status: Open * Assignee: sorah (Sorah Fukumori) * Target version: 3.3 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- gmp integration has been broken since https://bugs.ruby-lang.org/issues/20494 18eaf0be905e3e251423b42d6f4e56b7cae1bc3b including just released 3.3.2 due to missing HAVE_GMP_H. -- https://bugs.ruby-lang.org/

3 7

[ruby-core:117905] [Ruby master Bug#20493] Segfault on rb_io_getline_fast
by josegomezr (Jose Gomez) 15 Jun '24

15 Jun '24

Issue #20493 has been reported by josegomezr (Jose Gomez). ---------------------------------------- Bug #20493: Segfault on rb_io_getline_fast https://bugs.ruby-lang.org/issues/20493 * Author: josegomezr (Jose Gomez) * Status: Open * ruby -v: 3.3.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- We've spotted a consistent segfault when running bundle install with `--jobs 4` When running: `bundle install -j 4` we'd get a Segfault at: ``` /usr/lib64/ruby/3.3.0/rubygems/ext/builder.rb:93: [BUG] Segmentation fault at 0x0000000000000014 ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux-gnu] ``` Full [log is available here][0]. I could not find a shorter reproducer besides using bundler with `--jobs 4` or `--jobs 8`. Here's a sample command to trigger the behavior (it creates the Gemfile and calls bundler) [1]. We installed all debug symbols and narrowed down the location of the segfault to `rb_io_getline_fast` in io.c At [line 4001][3] `str` is `T_NONE`, which makes further usage down the line in [`io_enc_str`][4] raise a null pointer dereference. With the notes from [extension.rdoc - Appendix E. RB_GC_GUARD to protect from premature GC][8] I've prepared a patched ruby 3.3.1 package that does not segfault. It's on [OBS Project home:josegomezr:branches:ruby/ruby3.3][6]. Adding a `RB_GC_GUARD` on `rb_io_getline_fast` @ `io.c:4004` just before the return ```diff --- ruby3.3.orig/ruby-3.3.1/io.c +++ ruby3.3/ruby-3.3.1/io.c @@ -4004,6 +4004,7 @@ rb_io_getline_fast(rb_io_t *fptr, rb_enc ENC_CODERANGE_SET(str, cr); fptr->lineno++; + RB_GC_GUARD(str); return str; } ``` Fixes the segfault in our tests. `bundle` finish the installation and the image is built. I've set up a project in OBS to provide reproduceables. - [ruby3.3.1 package][5]. - [ruby3.3.1 base image with enough dependencies to reproduce][7] with [the reproducer script][1]. And the corresponding container is exported in the `containers-patched` repository. Here I leave the docker images generated by OBS: - 3.3.1 [without patches, segfaults.] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers/b… ``` - 3.3.1 [with patch, does not fail] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers-p… ``` [0]: https://gist.github.com/josegomezr/441c271cc731b0ec57213cb98743a699 [1]: https://gist.github.com/josegomezr/e17129bf2df33f3bea60e84a616a8322 [2]: https://gist.github.com/josegomezr/6f81878c979af334efee59b8f2225e58 [3]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4001 [4]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4003 [5]: https://build.opensuse.org/package/show/devel:languages:ruby/ruby3.3 [6]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby/ruby3… [7]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby:conta… [8]: https://github.com/ruby/ruby/blob/master/doc/extension.rdoc#label-Appendix+… -- https://bugs.ruby-lang.org/

4 13