April 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:111306] [Ruby master Feature#19236] Allow to create hashes with a specific capacity from Ruby
by byroot (Jean Boussier) 07 Jul '24

07 Jul '24

Issue #19236 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19236: Allow to create hashes with a specific capacity from Ruby https://bugs.ruby-lang.org/issues/19236 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.3 ---------------------------------------- Followup on [Feature #18683] which added a C-API for this purpose. Various protocol parsers such as Redis `RESP3` or `msgpack`, have to create hashes, and they know the size in advance. For efficiency, it would be preferable if they could directly allocate a Hash of the necessary size, so that large hashes wouldn't cause many re-alloccations and re-hash. `String` and `Array` both already offer similar APIs: ```ruby String.new(capacity: XXX) Array.new(XX) / rb_ary_new_capa(long) ``` However there's no such public API for Hashes in Ruby land. ### Proposal I think `Hash` should have a way to create a new hash with a `capacity` parameter. The logical signature of `Hash.new(capacity: 1000)` was deemed too incompatible in [Feature #18683]. @Eregon proposed to add `Hash.create(capacity: 1000)`. -- https://bugs.ruby-lang.org/

8 14

[ruby-core:116960] [Ruby master Bug#20305] commit 1d2d25dcadda0764f303183ac091d0c87b432566 breaks grapheme_clusters
by fablestales (Fable Tales) 06 Jul '24

06 Jul '24

Issue #20305 has been reported by fablestales (Fable Tales). ---------------------------------------- Bug #20305: commit 1d2d25dcadda0764f303183ac091d0c87b432566 breaks grapheme_clusters https://bugs.ruby-lang.org/issues/20305 * Author: fablestales (Fable Tales) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- given a script: ``` #script.rb p "안녕".byteslice(0, 4).grapheme_clusters ``` The commit 1d2d25dcadda0764f303183ac091d0c87b432566 (https://github.com/ruby/ruby/commit/1d2d25dcadda0764f303183ac091d0c87b432566) breaks the grapheme_clusters method on a byte slice ``` (commit 1d2d25dcadda0764f303183ac091d0c87b432566) ((HEAD detached at 1d2d25dcad)) $ ./ruby --disable=gems script.rb ["안", "\xEB"] ((HEAD detached at 1d2d25dcad)) $ git checkout HEAD^ (114e71d06280f9c57b9859ee4405ae89a989ddb6) ((HEAD detached at 114e71d062)) $ make -j ... ((HEAD detached at 114e71d062)) $ ./ruby --disable=gems script.rb ["안"] ((HEAD detached at 114e71d062)) $ cat script.rb p "안녕".byteslice(0, 4).grapheme_clusters ``` the expected result here is almost certainly the latter output, and not the former. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:116894] [Ruby master Bug#20288] `rb_fiber_scheduler_close` exceptions are not handled in `rb_fiber_scheduler_set`.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20288 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20288: `rb_fiber_scheduler_close` exceptions are not handled in `rb_fiber_scheduler_set`. https://bugs.ruby-lang.org/issues/20288 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The code responsible for setting a new scheduler on a thread does not properly handle exceptions raised by `rb_fiber_scheduler_close`. If `rb_fiber_scheduler_close` raised an exception, the assignment `thread->scheduler = scheduler` would not be executed. This leaves the thread in an undefined state because it might still hold a reference to the old scheduler, which was supposed to be closed and replaced. ## Steps to Reproduce: 1. Define a custom fiber scheduler that raises an exception in its close method. 2. Set the custom scheduler on a thread. 3. Attempt to replace the custom scheduler with a different scheduler. 4. Observe that if the close method of the original scheduler raises an exception, the thread's scheduler reference is not updated. ## Proposed Fix: The use of `rb_ensure` can be introduced to wrap the call to `rb_fiber_scheduler_close` to ensure that, regardless of whether an exception is raised, `thread->scheduler` is set to `Qnil`, and then `thread->scheduler = scheduler` is safely executed to update the thread's scheduler reference. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:117321] [Ruby master Bug#20393] `after_fork_ruby` clears all pending interrupts for both parent and child process.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20393 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20393: `after_fork_ruby` clears all pending interrupts for both parent and child process. https://bugs.ruby-lang.org/issues/20393 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- In the following program, the behaviour of the parent process is affected by whether `Process.fork` is invoked or not. ```ruby Thread.handle_interrupt(RuntimeError => :never) do Thread.current.raise(RuntimeError, "Queued error") puts "Pending interrupt: #{Thread.pending_interrupt?}" # true pid = Process.fork do puts "Pending interrupt (child process): #{Thread.pending_interrupt?}" Thread.handle_interrupt(RuntimeError => :immediate){} end _, status = Process.waitpid2(pid) puts "Child process status: #{status.inspect}" puts "Pending interrupt: #{Thread.pending_interrupt?}" # false end puts "Exiting..." ``` I don't think the parent process pending interrupts should be cleared by `after_fork_ruby`: ```c static void after_fork_ruby(rb_pid_t pid) { rb_threadptr_pending_interrupt_clear(GET_THREAD()); if (pid == 0) { // child clear_pid_cache(); rb_thread_atfork(); } else { // parent after_exec(); } } ``` How about this implementation: ```c static void after_fork_ruby(rb_pid_t pid) { if (pid == 0) { // child rb_threadptr_pending_interrupt_clear(GET_THREAD()); clear_pid_cache(); rb_thread_atfork(); } else { // parent after_exec(); } } ``` cc @ko1 -- https://bugs.ruby-lang.org/

3 5

[ruby-core:116890] [Ruby master Bug#20286] TracePoint does not emit `thread_end` event when thread exits with exception
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20286 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20286: TracePoint does not emit `thread_end` event when thread exits with exception https://bugs.ruby-lang.org/issues/20286 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Using TracePoint to trace `thread_begin` and `thread_end` events fails to emit the `thread_end` event when an exception (e.g., Interrupt) is raised within a thread. This behavior occurs because the exception handling bypasses the internal thread finishing logic, including trace point and fiber scheduler cleanup code. This issue affects the ability to accurately monitor thread lifecycle events in scenarios involving exception handling or abrupt thread terminations. ## Steps to Reproduce: 1. Set up `TracePoint` to trace `thread_begin` and `thread_end` events. 2. Create a new thread that raises an exception. 3. Join the thread and observe that only the `thread_begin` event is emitted without a corresponding `thread_end` event. ## Expected Behavior: The `TracePoint` should emit both `thread_begin` and `thread_end` events, accurately reflecting the lifecycle of the thread, even when an exception is raised within the thread. ## Actual Behavior: The `TracePoint` emits the `thread_begin` event but fails to emit the `thread_end` event when an exception is raised within the thread, indicating an incomplete tracing of thread lifecycle events. I've confirmed this as far back as Ruby 2.6. ## Example Code ```ruby TracePoint.trace(:thread_begin, :thread_end) do |tp| p [tp.event, tp.lineno, tp.path, tp.defined_class, tp.method_id] end thread = Thread.new do raise Interrupt end thread.join ``` ## Possible Fix Changing the implementation of `thread_do_start` to have what amounts to an "ensure" block. ```c static void thread_do_start(rb_thread_t *th) { native_set_thread_name(th); VALUE result = Qundef; rb_execution_context_t *ec = th->ec; int state; EXEC_EVENT_HOOK(ec, RUBY_EVENT_THREAD_BEGIN, th->self, 0, 0, 0, Qundef); EC_PUSH_TAG(ec); if ((state = EC_EXEC_TAG()) == TAG_NONE) { switch (th->invoke_type) { case thread_invoke_type_proc: result = thread_do_start_proc(th); break; case thread_invoke_type_ractor_proc: result = thread_do_start_proc(th); rb_ractor_atexit(th->ec, result); break; case thread_invoke_type_func: result = (*th->invoke_arg.func.func)(th->invoke_arg.func.arg); break; case thread_invoke_type_none: rb_bug("unreachable"); } } EC_POP_TAG(); VALUE errinfo = ec->errinfo; if (!NIL_P(errinfo) && !RB_TYPE_P(errinfo, T_OBJECT)) { ec->errinfo = Qnil; } rb_fiber_scheduler_set(Qnil); EXEC_EVENT_HOOK(th->ec, RUBY_EVENT_THREAD_END, th->self, 0, 0, 0, Qundef); ec->errinfo = errinfo; if (state) EC_JUMP_TAG(ec, state); th->value = result; } ``` It's possible `rb_fiber_scheduler_set(Qnil);` can emit an exception itself. How do we write the code to handle that case? -- https://bugs.ruby-lang.org/

3 5

[ruby-core:117453] [Ruby master Bug#20413] Enumerator can block fiber scheduler.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20413 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20413: Enumerator can block fiber scheduler. https://bugs.ruby-lang.org/issues/20413 * Author: ioquatix (Samuel Williams) * Status: Closed * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Using `Enumerator` in the event loop can cause problems as the fiber created by `rb_fiber_new` is blocking by default. It should be non-blocking. See <https://github.com/ruby/ruby/pull/10478> for the fix. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117458] [Ruby master Bug#20414] `Fiber#raise` should recurse to `resumed_fiber` rather than failing.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20414 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20414: `Fiber#raise` should recurse to `resumed_fiber` rather than failing. https://bugs.ruby-lang.org/issues/20414 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following program will fail with `FiberError`, and is difficult to properly clean up: ```ruby root_fiber = Fiber.current f1 = Fiber.new do root_fiber.transfer end f2 = Fiber.new do f1.resume end f2.transfer f2.raise("error") # => `raise': attempt to transfer to a resuming fiber (FiberError) ``` This program deliberately set's up a scenario where `f2` is resuming `f1`. Trying to raise an exception on `f2` is impossible, because the only way control flow can return to it, is when `f1` yields or exits. We can avoid this problem, by raising the exception on f1, and we can do this automatically using the following logic: ```c static VALUE fiber_raise(rb_fiber_t *fiber, VALUE exception) { // Add this recursive step: if (fiber->resuming_fiber) { return fiber_raise(fiber->resuming_fiber, exception); } // Existing code ... else if (FIBER_SUSPENDED_P(fiber) && !fiber->yielding) { return fiber_transfer_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } else { return fiber_resume_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } } ``` This makes `Fiber#raise` much more robust and useful for the purpose of stopping fibers, without knowing exactly what they are doing. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:115937] [Ruby master Bug#20099] Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux)
by jirutka (Jakub Jirutka) 01 Jul '24

01 Jul '24

Issue #20099 has been reported by jirutka (Jakub Jirutka). ---------------------------------------- Bug #20099: Ruby 3.3.0 segfaults on s390x musl libc (Alpine Linux) https://bugs.ruby-lang.org/issues/20099 * Author: jirutka (Jakub Jirutka) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr * exec prefix: ${prefix} * arch: s390x-linux-musl * site arch: ${arch} * RUBY_BASE_NAME: ruby * enable shared: yes * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: linux-musl * compiler: gcc * with thread: pthread * with coroutine: ucontext * enable shared libs: yes * dynamic library ext: so * CFLAGS: ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -Wl,--as-needed,-O1,--sort-common \ -fstack-protector-strong -rdynamic \ -Wl,-export-dynamic -Wl,--no-as-needed * DLDFLAGS: -Wl,--as-needed,-O1,--sort-common \ -Wl,--compress-debug-sections=zlib * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wdeprecated-declarations \ -Wdiv-by-zero -Wduplicated-cond \ -Wimplicit-function-declaration -Wimplicit-int \ -Wpointer-arith -Wwrite-strings \ -Wold-style-definition -Wimplicit-fallthrough=0 \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings \ -Wno-packed-bitfield-compat \ -Wno-parentheses-equality -Wno-self-assign \ -Wno-tautological-compare -Wno-unused-parameter \ -Wno-unused-value -Wsuggest-attribute=format \ -Wsuggest-attribute=noreturn -Wunused-variable \ -Wmisleading-indentation -Wundef * strip command: strip -S -x * install doc: rdoc * YJIT support: no * RJIT support: no * man page type: man * search path: /usr/lib/site_ruby/$(ruby_ver)/s390x-linux --- BASERUBY = ./tool/missing-baseruby.bat CC = gcc LD = ld LDSHARED = gcc -shared CFLAGS = -O2 -fstack-clash-protection -Wformat -Werror=format-security -g -fno-omit-frame-pointer -fno-strict-aliasing -fPIC XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/s390x-linux-musl -I./include -I. -I./prism -I./enc/unicode/15.0.0 CPPFLAGS = -fno-omit-frame-pointer -fno-strict-aliasing DLDFLAGS = -Wl,--as-needed,-O1,--sort-common -Wl,--compress-debug-sections=zlib -Wl,-soname,libruby.so.3.3 -fstack-protector-strong SOLIBS = -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread -lucontext LANG = LC_ALL = LC_CTYPE = MFLAGS = -j8 --jobserver-auth=fifo:/tmp/GMfifo7323 RUSTC = no YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/builds/alpine/aports/main/ruby/src/ruby-3.3.0/yjit/target/release/' ./yjit/src/lib.rs ``` ``` 2023-12-26 20:45:17 +0000 Driver is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] Target is ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] .............................................................................. .............................................................................. .............................................................................. .............................................................................. ................................................................EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:18Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:18Z", "-ex", "bt", "-batch"] ...EA core file is found. Saving it at: "/tmp/bootstraptest-core.2023-12-26T20:45:19Z" ["gdb", "/builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby -I/builds/alpine/aports/main/ruby/src/ruby-3.3.0/lib --disable-gems", "-c", "/tmp/bootstraptest-core.2023-12-26T20:45:19Z", "-ex", "bt", "-batch"] .....Fstderr output is not empty [BUG] Segmentation fault at 0x000007fece67a000 ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [s390x-linux-musl] -- Control frame information ----------------------------------------------- c:0001 p:---- s:0003 e:000002 DUMMY [FINISH] -- Threading information --------------------------------------------------- Total ractor count: 2 Ruby thread count for this ractor: 1 -- Other runtime information ----------------------------------------------- * Loaded script: bootstraptest.test_ractor.rb_244_1238.rb * Loaded features: 0 enumerator.so 1 thread.rb 2 fiber.so 3 rational.so 4 complex.so 5 ruby2_keywords.rb 6 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 7 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so * Process memory map: 2aa25e00000-2aa25e01000 r-xp 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e01000-2aa25e02000 r--p 00000000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa25e02000-2aa25e03000 rw-p 00001000 fe:00 2520910 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/ruby 2aa2765d000-2aa2765e000 ---p 00000000 00:00 0 [heap] 2aa2765e000-2aa27664000 rw-p 00000000 00:00 0 [heap] 3ff572a8000-3ff572aa000 ---p 00000000 00:00 0 3ff572aa000-3ff572d0000 rw-p 00000000 00:00 0 3ff572d0000-3ff572d2000 ---p 00000000 00:00 0 3ff572d2000-3ff573f9000 rw-p 00000000 00:00 0 3ff573f9000-3ff573fa000 ---p 00000000 00:00 0 3ff573fa000-3ff5751a000 rw-p 00000000 00:00 0 3ff5751a000-3ff5751b000 ---p 00000000 00:00 0 3ff5751b000-3ff77300000 rw-p 00000000 00:00 0 3ff77300000-3ff77302000 r-xp 00000000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77302000-3ff77303000 r--p 00002000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77303000-3ff77304000 rw-p 00003000 fe:00 2520334 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/trans/transdb.so 3ff77307000-3ff77309000 rw-p 00000000 00:00 0 3ff7730b000-3ff7730e000 rw-p 00000000 00:00 0 3ff77310000-3ff77330000 rw-p 00000000 00:00 0 3ff77332000-3ff77337000 rw-p 00000000 00:00 0 3ff77339000-3ff7733a000 rw-p 00000000 00:00 0 3ff7733b000-3ff7733e000 rw-p 00000000 00:00 0 3ff77340000-3ff77380000 rw-p 00000000 00:00 0 3ff77380000-3ff77382000 r-xp 00000000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77382000-3ff77383000 r--p 00001000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77383000-3ff77384000 rw-p 00002000 fe:00 2520245 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/.ext/s390x-linux-musl/enc/encdb.so 3ff77384000-3ff7739c000 rw-p 00000000 00:00 0 3ff7739c000-3ff7739d000 ---p 00000000 00:00 0 3ff7739d000-3ff773de000 rw-p 00000000 00:00 0 3ff773de000-3ff773df000 ---p 00000000 00:00 0 3ff773df000-3ff77420000 rw-p 00000000 00:00 0 3ff77420000-3ff77421000 ---p 00000000 00:00 0 3ff77421000-3ff77462000 rw-p 00000000 00:00 0 3ff77462000-3ff77463000 ---p 00000000 00:00 0 3ff77463000-3ff774a4000 rw-p 00000000 00:00 0 3ff774a4000-3ff774a5000 ---p 00000000 00:00 0 3ff774a5000-3ff774e6000 rw-p 00000000 00:00 0 3ff774e6000-3ff774e7000 ---p 00000000 00:00 0 3ff774e7000-3ff77528000 rw-p 00000000 00:00 0 3ff77528000-3ff77529000 ---p 00000000 00:00 0 3ff77529000-3ff7756a000 rw-p 00000000 00:00 0 3ff7756a000-3ff7756b000 ---p 00000000 00:00 0 3ff7756b000-3ff775ac000 rw-p 00000000 00:00 0 3ff775ac000-3ff775ad000 ---p 00000000 00:00 0 3ff775ad000-3ff775ee000 rw-p 00000000 00:00 0 3ff775ee000-3ff775ef000 ---p 00000000 00:00 0 3ff775ef000-3ff77630000 rw-p 00000000 00:00 0 3ff77630000-3ff77631000 ---p 00000000 00:00 0 3ff77631000-3ff77672000 rw-p 00000000 00:00 0 3ff77672000-3ff77673000 ---p 00000000 00:00 0 3ff77673000-3ff776b4000 rw-p 00000000 00:00 0 3ff776b4000-3ff776b5000 ---p 00000000 00:00 0 3ff776b5000-3ff776f6000 rw-p 00000000 00:00 0 3ff776f6000-3ff776f7000 ---p 00000000 00:00 0 3ff776f7000-3ff77738000 rw-p 00000000 00:00 0 3ff77738000-3ff77739000 ---p 00000000 00:00 0 3ff77739000-3ff7777a000 rw-p 00000000 00:00 0 3ff7777a000-3ff7777b000 ---p 00000000 00:00 0 3ff7777b000-3ff777bc000 rw-p 00000000 00:00 0 3ff777bc000-3ff777bd000 ---p 00000000 00:00 0 3ff777bd000-3ff777fe000 rw-p 00000000 00:00 0 3ff777fe000-3ff777ff000 ---p 00000000 00:00 0 3ff777ff000-3ff77840000 rw-p 00000000 00:00 0 3ff77840000-3ff77841000 ---p 00000000 00:00 0 3ff77841000-3ff77882000 rw-p 00000000 00:00 0 3ff77882000-3ff77883000 ---p 00000000 00:00 0 3ff77883000-3ff778c4000 rw-p 00000000 00:00 0 3ff778c4000-3ff778c5000 ---p 00000000 00:00 0 3ff778c5000-3ff77906000 rw-p 00000000 00:00 0 3ff77906000-3ff77907000 ---p 00000000 00:00 0 3ff77907000-3ff77948000 rw-p 00000000 00:00 0 3ff77948000-3ff77949000 ---p 00000000 00:00 0 3ff77949000-3ff7798a000 rw-p 00000000 00:00 0 3ff7798a000-3ff7798b000 ---p 00000000 00:00 0 3ff7798b000-3ff779cc000 rw-p 00000000 00:00 0 3ff779cc000-3ff779cd000 ---p 00000000 00:00 0 3ff779cd000-3ff77a0e000 rw-p 00000000 00:00 0 3ff77a0e000-3ff77a0f000 ---p 00000000 00:00 0 3ff77a0f000-3ff77a50000 rw-p 00000000 00:00 0 3ff77a50000-3ff77a51000 ---p 00000000 00:00 0 3ff77a51000-3ff77a92000 rw-p 00000000 00:00 0 3ff77a92000-3ff77a93000 ---p 00000000 00:00 0 3ff77a93000-3ff77ad4000 rw-p 00000000 00:00 0 3ff77ad4000-3ff77ad5000 ---p 00000000 00:00 0 3ff77ad5000-3ff77b16000 rw-p 00000000 00:00 0 3ff77b16000-3ff77b17000 ---p 00000000 00:00 0 3ff77b17000-3ff77b58000 rw-p 00000000 00:00 0 3ff77b58000-3ff77b59000 ---p 00000000 00:00 0 3ff77b59000-3ff77b9a000 rw-p 00000000 00:00 0 3ff77b9a000-3ff77b9b000 ---p 00000000 00:00 0 3ff77b9b000-3ff77bdc000 rw-p 00000000 00:00 0 3ff77bdc000-3ff77bde000 ---p 00000000 00:00 0 3ff77bde000-3ff91100000 rw-p 00000000 00:00 0 3ff91100000-3ff91101000 r-xp 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91101000-3ff91102000 r--p 00000000 fe:00 2758431 /lib/libucontext.so.1 3ff91102000-3ff91103000 rw-p 00001000 fe:00 2758431 /lib/libucontext.so.1 3ff91103000-3ff9114b000 rw-p 00000000 00:00 0 3ff9114f000-3ff91153000 rw-p 00000000 00:00 0 3ff91155000-3ff91180000 rw-p 00000000 00:00 0 3ff91180000-3ff911fd000 r-xp 00000000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911fd000-3ff911ff000 r--p 0007c000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff911ff000-3ff91200000 rw-p 0007e000 fe:00 2755903 /usr/lib/libgmp.so.10.5.0 3ff91200000-3ff91219000 r-xp 00000000 fe:00 2754145 /lib/libz.so.1.3 3ff91219000-3ff9121a000 r--p 00018000 fe:00 2754145 /lib/libz.so.1.3 3ff9121a000-3ff9121b000 rw-p 00019000 fe:00 2754145 /lib/libz.so.1.3 3ff9121b000-3ff91280000 rw-p 00000000 00:00 0 3ff91280000-3ff91700000 r-xp 00000000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff91700000-3ff9170c000 r--p 00480000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170c000-3ff9170d000 rw-p 0048c000 fe:00 2519847 /builds/alpine/aports/main/ruby/src/ruby-3.3.0/libruby.so.3.3.0 3ff9170d000-3ff9171f000 rw-p 00000000 00:00 0 3ff9171f000-3ff91780000 rw-p 00000000 00:00 0 3ff91780000-3ff91830000 r-xp 00000000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91830000-3ff91831000 r--p 000b0000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91831000-3ff91832000 rw-p 000b1000 fe:00 2758095 /lib/ld-musl-s390x.so.1 3ff91832000-3ff91835000 rw-p 00000000 00:00 0 3ff91835000-3ff9187c000 rw-p 00000000 00:00 0 3ff9187c000-3ff9187e000 r--p 00000000 00:00 0 [vvar] 3ff9187e000-3ff91880000 r-xp 00000000 00:00 0 [vdso] 3ffe2381000-3ffe2b80000 rw-p 00000000 00:00 0 [stack] ``` …and many more tests fail like this. ---Files-------------------------------- ruby-3.3.0-s390x.log (281 KB) -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117498] [Ruby master Feature#20425] Optimize forwarding callers and callees
by tenderlovemaking (Aaron Patterson) 18 Jun '24

18 Jun '24

Issue #20425 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #20425: Optimize forwarding callers and callees https://bugs.ruby-lang.org/issues/20425 * Author: tenderlovemaking (Aaron Patterson) * Status: Open ---------------------------------------- [This PR](https://github.com/ruby/ruby/pull/10510) optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` ## How does it work? This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. ## Why? I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. -- https://bugs.ruby-lang.org/

2 13

[ruby-core:116182] [Ruby master Bug#20180] Inconsistent evaluation of `**{}` depending on position in array
by ozydingo (Andrew Schwartz) 15 Jun '24

15 Jun '24

Issue #20180 has been reported by ozydingo (Andrew Schwartz). ---------------------------------------- Bug #20180: Inconsistent evaluation of `**{}` depending on position in array https://bugs.ruby-lang.org/issues/20180 * Author: ozydingo (Andrew Schwartz) * Status: Open * Priority: Normal * ruby -v: 3.3 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduced on ruby:3.3 docker container The evaluation of `**{}` differs if it appears alone (evaluates as empty / no content) in an array vs after another element (evaluates as an empty Hash). ```rb args = []; kwargs = {} [*args] # => [] [**kwargs] # => [] [*args, **kwargs] # => [{}] [*args] + [**kwargs] == [*args, **kwargs] => false ``` I claim this violates the Principle of Least Surprise. I will admit that beyond a thin example I will give below, I am struggling to come up with a more convincing pragmatic reason that this should be addressed, but due to how surprising it is and the bugs it cause our team I wanted to submit it for tracking. This may be related to https://bugs.ruby-lang.org/issues/20064?tab=notes though the issues are distinct. Specifically, in my use case, I am writing a class responsible for adapting arguments in one form to another (json object to method args and vice versa). My tests broken when adding support for keyword args due to expectations of no args: ``` # RSpec expect(foo).to have_received(:bar).with([]) # This now need `with([], {})` ``` Again, this is a bit thin as by itself this isn't problematic as I can add the `{}`. However, this does require that my test knows more about the implementation that it should. That is, my implementation might be ``` if kwargs.present? call(*args, **kwargs) else call(*args) end ``` This change does not change the behavior of the class, but will break the test. I therefore think this behavior of `**kwargs` is problematic. -- https://bugs.ruby-lang.org/

6 7