April 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:117453] [Ruby master Bug#20413] Enumerator can block fiber scheduler.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20413 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20413: Enumerator can block fiber scheduler. https://bugs.ruby-lang.org/issues/20413 * Author: ioquatix (Samuel Williams) * Status: Closed * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Using `Enumerator` in the event loop can cause problems as the fiber created by `rb_fiber_new` is blocking by default. It should be non-blocking. See <https://github.com/ruby/ruby/pull/10478> for the fix. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117458] [Ruby master Bug#20414] `Fiber#raise` should recurse to `resumed_fiber` rather than failing.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20414 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20414: `Fiber#raise` should recurse to `resumed_fiber` rather than failing. https://bugs.ruby-lang.org/issues/20414 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following program will fail with `FiberError`, and is difficult to properly clean up: ```ruby root_fiber = Fiber.current f1 = Fiber.new do root_fiber.transfer end f2 = Fiber.new do f1.resume end f2.transfer f2.raise("error") # => `raise': attempt to transfer to a resuming fiber (FiberError) ``` This program deliberately set's up a scenario where `f2` is resuming `f1`. Trying to raise an exception on `f2` is impossible, because the only way control flow can return to it, is when `f1` yields or exits. We can avoid this problem, by raising the exception on f1, and we can do this automatically using the following logic: ```c static VALUE fiber_raise(rb_fiber_t *fiber, VALUE exception) { // Add this recursive step: if (fiber->resuming_fiber) { return fiber_raise(fiber->resuming_fiber, exception); } // Existing code ... else if (FIBER_SUSPENDED_P(fiber) && !fiber->yielding) { return fiber_transfer_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } else { return fiber_resume_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } } ``` This makes `Fiber#raise` much more robust and useful for the purpose of stopping fibers, without knowing exactly what they are doing. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:115937] [Ruby master Bug#20099] Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux)
by jirutka (Jakub Jirutka) 01 Jul '24

01 Jul '24

Issue #20099 has been reported by jirutka (Jakub Jirutka). ---------------------------------------- Bug #20099: Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux) https://bugs.ruby-lang.org/issues/20099 * Author: jirutka (Jakub Jirutka) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr * exec prefix: ${prefix} * arch: s390x-linux-musl * site arch: ${arch} * RUBY_BASE_NAME: ruby * enable shared: yes * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: linux-musl * compiler: gcc * with thread: pthread * with coroutine: ucontext * enable shared libs: yes * dynamic library ext: so * CFLAGS: ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -Wl,--as-needed,-O1,--sort-common \ -fstack-protector-strong -rdynamic \ -Wl,-export-dynamic -Wl,--no-as-needed * DLDFLAGS: -Wl,--as-needed,-O1,--sort-common \ -Wl,--compress-debug-sections=zlib * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wdeprecated-declarations \ -Wdiv-by-zero -Wduplicated-cond \ -Wimplicit-function-declaration -Wimplicit-int \ -Wpointer-arith -Wwrite-strings \ -Wold-style-definition -Wimplicit-fallthrough=0 \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings \ -Wno-packed-bitfield-compat \ -Wno-parentheses-equality -Wno-self-assign \ -Wno-tautological-compare -Wno-unused-parameter \ -Wno-unused-value -Wsuggest-attribute=format \ -Wsuggest-attribute=noreturn -Wunused-variable \ -Wmisleading-indentation -Wundef * strip command: strip -S -x * install doc: rdoc * YJIT support: no * RJIT support: no * man page type: man * search path: /usr/lib/site_ruby/$(ruby_ver)/s390x-linux --- BASERUBY = ./tool/missing-baseruby.bat CC = gcc LD = ld LDSHARED = gcc -shared CFLAGS = -O2 -fstack-clash-protection -Wformat -Werror=format-security -g -fno-omit-frame-pointer -fno-strict-aliasing -fPIC XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/s390x-linux-musl -I./include -I. -I./prism -I./enc/unicode/15.0.0 CPPFLAGS = -fno-omit-frame-pointer -fno-strict-aliasing DLDFLAGS = -Wl,--as-needed,-O1,--sort-common -Wl,--compress-debug-sections=zlib -Wl,-soname,libruby.so.3.3 -fstack-protector-strong SOLIBS = -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread -lucontext LANG = LC_ALL = LC_CTYPE = MFLAGS = -j8 --jobserver-auth=fifo:/tmp/GMfifo7323 RUSTC = no YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/builds/alpine/aports/main/ruby/src/ruby-3.3.0/yjit/target/release/' ./yjit/src/lib.rs ``` ``` 2023-12-26 20:45:17 +0000 Driver is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] Target is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] .............................................................................. .............................................................................. .............................................................................. .............................................................................. ................................................................EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:18Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:18Z", "-ex", "bt", "-batch"] ...EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:19Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:19Z", "-ex", "bt", "-batch"] .....Fstderr output is not empty [BUG] Segmentation fault at 0x000007fece67a000 ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] -- Control frame information ----------------------------------------------- c:0001 p:---- s:0003 e:000002 DUMMY [FINISH] -- Threading information --------------------------------------------------- Total ractor count: 2 Ruby thread count for this ractor: 1 -- Other runtime information ----------------------------------------------- * Loaded script: bootstraptest.test_ractor.rb_244_1238.rb * Loaded features: 0 enumerator.so 1 thread.rb 2 fiber.so 3 rational.so 4 complex.so 5 ruby2_keywords.rb 6 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 7 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so * Process memory map: 2aa25e00000-2aa25e01000 r-xp 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e01000-2aa25e02000 r--p 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e02000-2aa25e03000 rw-p 00001000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa2765d000-2aa2765e000 ---p 00000000 00:00 0 [heap] 2aa2765e000-2aa27664000 rw-p 00000000 00:00 0 [heap] 3ff572a8000-3ff572aa000 ---p 00000000 00:00 0 3ff572aa000-3ff572d0000 rw-p 00000000 00:00 0 3ff572d0000-3ff572d2000 ---p 00000000 00:00 0 3ff572d2000-3ff573f9000 rw-p 00000000 00:00 0 3ff573f9000-3ff573fa000 ---p 00000000 00:00 0 3ff573fa000-3ff5751a000 rw-p 00000000 00:00 0 3ff5751a000-3ff5751b000 ---p 00000000 00:00 0 3ff5751b000-3ff77300000 rw-p 00000000 00:00 0 3ff77300000-3ff77302000 r-xp 00000000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77302000-3ff77303000 r--p 00002000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77303000-3ff77304000 rw-p 00003000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77307000-3ff77309000 rw-p 00000000 00:00 0 3ff7730b000-3ff7730e000 rw-p 00000000 00:00 0 3ff77310000-3ff77330000 rw-p 00000000 00:00 0 3ff77332000-3ff77337000 rw-p 00000000 00:00 0 3ff77339000-3ff7733a000 rw-p 00000000 00:00 0 3ff7733b000-3ff7733e000 rw-p 00000000 00:00 0 3ff77340000-3ff77380000 rw-p 00000000 00:00 0 3ff77380000-3ff77382000 r-xp 00000000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77382000-3ff77383000 r--p 00001000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77383000-3ff77384000 rw-p 00002000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77384000-3ff7739c000 rw-p 00000000 00:00 0 3ff7739c000-3ff7739d000 ---p 00000000 00:00 0 3ff7739d000-3ff773de000 rw-p 00000000 00:00 0 3ff773de000-3ff773df000 ---p 00000000 00:00 0 3ff773df000-3ff77420000 rw-p 00000000 00:00 0 3ff77420000-3ff77421000 ---p 00000000 00:00 0 3ff77421000-3ff77462000 rw-p 00000000 00:00 0 3ff77462000-3ff77463000 ---p 00000000 00:00 0 3ff77463000-3ff774a4000 rw-p 00000000 00:00 0 3ff774a4000-3ff774a5000 ---p 00000000 00:00 0 3ff774a5000-3ff774e6000 rw-p 00000000 00:00 0 3ff774e6000-3ff774e7000 ---p 00000000 00:00 0 3ff774e7000-3ff77528000 rw-p 00000000 00:00 0 3ff77528000-3ff77529000 ---p 00000000 00:00 0 3ff77529000-3ff7756a000 rw-p 00000000 00:00 0 3ff7756a000-3ff7756b000 ---p 00000000 00:00 0 3ff7756b000-3ff775ac000 rw-p 00000000 00:00 0 3ff775ac000-3ff775ad000 ---p 00000000 00:00 0 3ff775ad000-3ff775ee000 rw-p 00000000 00:00 0 3ff775ee000-3ff775ef000 ---p 00000000 00:00 0 3ff775ef000-3ff77630000 rw-p 00000000 00:00 0 3ff77630000-3ff77631000 ---p 00000000 00:00 0 3ff77631000-3ff77672000 rw-p 00000000 00:00 0 3ff77672000-3ff77673000 ---p 00000000 00:00 0 3ff77673000-3ff776b4000 rw-p 00000000 00:00 0 3ff776b4000-3ff776b5000 ---p 00000000 00:00 0 3ff776b5000-3ff776f6000 rw-p 00000000 00:00 0 3ff776f6000-3ff776f7000 ---p 00000000 00:00 0 3ff776f7000-3ff77738000 rw-p 00000000 00:00 0 3ff77738000-3ff77739000 ---p 00000000 00:00 0 3ff77739000-3ff7777a000 rw-p 00000000 00:00 0 3ff7777a000-3ff7777b000 ---p 00000000 00:00 0 3ff7777b000-3ff777bc000 rw-p 00000000 00:00 0 3ff777bc000-3ff777bd000 ---p 00000000 00:00 0 3ff777bd000-3ff777fe000 rw-p 00000000 00:00 0 3ff777fe000-3ff777ff000 ---p 00000000 00:00 0 3ff777ff000-3ff77840000 rw-p 00000000 00:00 0 3ff77840000-3ff77841000 ---p 00000000 00:00 0 3ff77841000-3ff77882000 rw-p 00000000 00:00 0 3ff77882000-3ff77883000 ---p 00000000 00:00 0 3ff77883000-3ff778c4000 rw-p 00000000 00:00 0 3ff778c4000-3ff778c5000 ---p 00000000 00:00 0 3ff778c5000-3ff77906000 rw-p 00000000 00:00 0 3ff77906000-3ff77907000 ---p 00000000 00:00 0 3ff77907000-3ff77948000 rw-p 00000000 00:00 0 3ff77948000-3ff77949000 ---p 00000000 00:00 0 3ff77949000-3ff7798a000 rw-p 00000000 00:00 0 3ff7798a000-3ff7798b000 ---p 00000000 00:00 0 3ff7798b000-3ff779cc000 rw-p 00000000 00:00 0 3ff779cc000-3ff779cd000 ---p 00000000 00:00 0 3ff779cd000-3ff77a0e000 rw-p 00000000 00:00 0 3ff77a0e000-3ff77a0f000 ---p 00000000 00:00 0 3ff77a0f000-3ff77a50000 rw-p 00000000 00:00 0 3ff77a50000-3ff77a51000 ---p 00000000 00:00 0 3ff77a51000-3ff77a92000 rw-p 00000000 00:00 0 3ff77a92000-3ff77a93000 ---p 00000000 00:00 0 3ff77a93000-3ff77ad4000 rw-p 00000000 00:00 0 3ff77ad4000-3ff77ad5000 ---p 00000000 00:00 0 3ff77ad5000-3ff77b16000 rw-p 00000000 00:00 0 3ff77b16000-3ff77b17000 ---p 00000000 00:00 0 3ff77b17000-3ff77b58000 rw-p 00000000 00:00 0 3ff77b58000-3ff77b59000 ---p 00000000 00:00 0 3ff77b59000-3ff77b9a000 rw-p 00000000 00:00 0 3ff77b9a000-3ff77b9b000 ---p 00000000 00:00 0 3ff77b9b000-3ff77bdc000 rw-p 00000000 00:00 0 3ff77bdc000-3ff77bde000 ---p 00000000 00:00 0 3ff77bde000-3ff91100000 rw-p 00000000 00:00 0 3ff91100000-3ff91101000 r-xp 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91101000-3ff91102000 r--p 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91102000-3ff91103000 rw-p 00001000 fe:00 2758431 /lib/libucontext.so.1 3ff91103000-3ff9114b000 rw-p 00000000 00:00 0 3ff9114f000-3ff91153000 rw-p 00000000 00:00 0 3ff91155000-3ff91180000 rw-p 00000000 00:00 0 3ff91180000-3ff911fd000 r-xp 00000000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911fd000-3ff911ff000 r--p 0007c000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911ff000-3ff91200000 rw-p 0007e000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff91200000-3ff91219000 r-xp 00000000 fe:00 2754145 /lib/libz.so.1.3 3ff91219000-3ff9121a000 r--p 00018000 fe:00 2754145 /lib/libz.so.1.3 3ff9121a000-3ff9121b000 rw-p 00019000 fe:00 2754145 /lib/libz.so.1.3 3ff9121b000-3ff91280000 rw-p 00000000 00:00 0 3ff91280000-3ff91700000 r-xp 00000000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff91700000-3ff9170c000 r--p 00480000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170c000-3ff9170d000 rw-p 0048c000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170d000-3ff9171f000 rw-p 00000000 00:00 0 3ff9171f000-3ff91780000 rw-p 00000000 00:00 0 3ff91780000-3ff91830000 r-xp 00000000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91830000-3ff91831000 r--p 000b0000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91831000-3ff91832000 rw-p 000b1000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91832000-3ff91835000 rw-p 00000000 00:00 0 3ff91835000-3ff9187c000 rw-p 00000000 00:00 0 3ff9187c000-3ff9187e000 r--p 00000000 00:00 0 [vvar] 3ff9187e000-3ff91880000 r-xp 00000000 00:00 0 [vdso] 3ffe2381000-3ffe2b80000 rw-p 00000000 00:00 0 [stack] ``` …and many more tests fail like this. ---Files-------------------------------- ruby-3.3.0-s390x.log (281 KB) -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117498] [Ruby master Feature#20425] Optimize forwarding callers and callees
by tenderlovemaking (Aaron Patterson) 18 Jun '24

18 Jun '24

Issue #20425 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #20425: Optimize forwarding callers and callees https://bugs.ruby-lang.org/issues/20425 * Author: tenderlovemaking (Aaron Patterson) * Status: Open ---------------------------------------- [This PR](https://github.com/ruby/ruby/pull/10510) optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` ## How does it work? This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. ## Why? I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. -- https://bugs.ruby-lang.org/

2 13

[ruby-core:116182] [Ruby master Bug#20180] Inconsistent evaluation of `**{}` depending on position in array
by ozydingo (Andrew Schwartz) 15 Jun '24

15 Jun '24

Issue #20180 has been reported by ozydingo (Andrew Schwartz). ---------------------------------------- Bug #20180: Inconsistent evaluation of `**{}` depending on position in array https://bugs.ruby-lang.org/issues/20180 * Author: ozydingo (Andrew Schwartz) * Status: Open * Priority: Normal * ruby -v: 3.3 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduced on ruby:3.3 docker container The evaluation of `**{}` differs if it appears alone (evaluates as empty / no content) in an array vs after another element (evaluates as an empty Hash). ```rb args = []; kwargs = {} [*args] # => [] [**kwargs] # => [] [*args, **kwargs] # => [{}] [*args] + [**kwargs] == [*args, **kwargs] => false ``` I claim this violates the Principle of Least Surprise. I will admit that beyond a thin example I will give below, I am struggling to come up with a more convincing pragmatic reason that this should be addressed, but due to how surprising it is and the bugs it cause our team I wanted to submit it for tracking. This may be related to https://bugs.ruby-lang.org/issues/20064?tab=notes though the issues are distinct. Specifically, in my use case, I am writing a class responsible for adapting arguments in one form to another (json object to method args and vice versa). My tests broken when adding support for keyword args due to expectations of no args: ``` # RSpec expect(foo).to have_received(:bar).with([]) # This now need `with([], {})` ``` Again, this is a bit thin as by itself this isn't problematic as I can add the `{}`. However, this does require that my test knows more about the implementation that it should. That is, my implementation might be ``` if kwargs.present? call(*args, **kwargs) else call(*args) end ``` This change does not change the behavior of the class, but will break the test. I therefore think this behavior of `**kwargs` is problematic. -- https://bugs.ruby-lang.org/

6 7

[ruby-core:116838] [Ruby master Bug#20280] Error at invalid encoding symbols
by nobu (Nobuyoshi Nakada) 15 Jun '24

15 Jun '24

Issue #20280 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20280: Error at invalid encoding symbols https://bugs.ruby-lang.org/issues/20280 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Parsing an invalid encoding symbol raises an `EncodingError`. It must be a `SyntaxError`. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116790] [Ruby master Bug#20270] Options with `--parser=prism`
by nobu (Nobuyoshi Nakada) 11 Jun '24

11 Jun '24

Issue #20270 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20270: Options with `--parser=prism` https://bugs.ruby-lang.org/issues/20270 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ### `--dump` option Currently `parsetree` and `prism_parsetree` bits are separated, but it seems meaningless as far as `--parser` option selects only one parser. Why doesn't simply `--dump=parsetree` dump AST by parse.y, or PRISM AST if `--parser=prism` is given? Actually, `ruby --dump=parsetree --parser=prism` just segfaults, because these bits and `--parser` option are separately handled. ### streaming code from stdin ```sh-session $ echo end | ruby --parser=prism - ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker. -: warning: Prism support for streaming code from stdin is not currently supported ``` Nothing reported is fine, since it is not currently supported. I was a bit curious that it looks like trying something after the warning. ```C if (strcmp(opt->script, "-") == 0) { int xflag = opt->xflag; VALUE rb_source = open_load_file(opt->script_name, &xflag); opt->xflag = xflag != 0; rb_warn("Prism support for streaming code from stdin is not currently supported"); error = pm_parse_string(&result, rb_source, opt->script_name); } ``` Note that `open_load_file` returns `rb_stdin` when `-` is given. This object is a `RFile`, not a `RString`, and `RSTRING_PTR(rb_source)` accesses after the object boundary. It should just bail out, not try obviously wrong thing. ### Fix https://github.com/nobu/ruby/tree/options-refactor -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116039] [Ruby master Bug#20154] aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment
by jprokop (Jarek Prokop) 09 Jun '24

09 Jun '24

Issue #20154 has been reported by jprokop (Jarek Prokop). ---------------------------------------- Bug #20154: aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment https://bugs.ruby-lang.org/issues/20154 * Author: jprokop (Jarek Prokop) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Recently a GH PR was merged <https://github.com/ruby/ruby/pull/9306> For PAC/BTI support on ARM CPUs for Coroutine.S. Without proper compilation support in configure.ac it segfaults Ruby with fibers on CPUs where PAC is supported: https://bugs.ruby-lang.org/issues/20085 At the time of writing, configure.ac appends the first option from a list for flag `-mbranch-protection` that successfully compiles a program <https://github.com/ruby/ruby/blob/master/configure.ac#L829>, to XCFLAGS and now also ASFLAGS to fix issue 20085 for Ruby master. This is suboptimal for Fedora as we set -mbranch-protection=standard by default in C{,XX}FLAGS: ``` CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ' export CFLAGS CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' export CXXFLAGS ``` And the appended flag overrides distribution's compilation configuration, which in this case ends up omitting BTI instructions and only using PAC. Would it make sense to check if such flags exist and not overwrite them if they do? Serious proposals: 1. Simplest fix that does not overwrite what is set in the distribution and results in higher security is simply prepending the list of options with `-mbranch-protection=standard`, it should cause no problems on ARMv8 CPUs and forward, BTI similarly to PAC instructions result into NOP, it is only extending the capability. See attached 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch 2. Other fix that sounds more sane IMO and dodges this kind of guessing where are all the correct places for the flag is what another Fedora contributor Florian Weimer suggested: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… "The reliable way to do this would be to compile a C file and check whether that enables __ARM_FEATURE_PAC_DEFAULT, and if that's the case, define a *different* macro for use in the assembler implementation. This way, you don't need to care about the exact name of the option." IOW instead of using __ARM_FEATURE_* directly in that code, define a macro in the style of "USE_PAC" with value of the feature if it is defined, I think that way we shouldn't need to append ASFLAGS anymore. However it's also important to catch the value of those macros as their values have meaning, I have an idea how to do that but I'd get on that monday earliest. ---Files-------------------------------- 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch (1004 Bytes) -- https://bugs.ruby-lang.org/

4 10

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:116117] [Ruby master Misc#20170] Drop support for GCC < 11
by kddnewton (Kevin Newton) 07 Jun '24

07 Jun '24

Issue #20170 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20170: Drop support for GCC < 11 https://bugs.ruby-lang.org/issues/20170 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers. -- https://bugs.ruby-lang.org/

4 5