June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116960] [Ruby master Bug#20305] commit 1d2d25dcadda0764f303183ac091d0c87b432566 breaks grapheme_clusters
by fablestales (Fable Tales) 06 Jul '24

06 Jul '24

Issue #20305 has been reported by fablestales (Fable Tales). ---------------------------------------- Bug #20305: commit 1d2d25dcadda0764f303183ac091d0c87b432566 breaks grapheme_clusters https://bugs.ruby-lang.org/issues/20305 * Author: fablestales (Fable Tales) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- given a script: ``` #script.rb p "안녕".byteslice(0, 4).grapheme_clusters ``` The commit 1d2d25dcadda0764f303183ac091d0c87b432566 (https://github.com/ruby/ruby/commit/1d2d25dcadda0764f303183ac091d0c87b432566) breaks the grapheme_clusters method on a byte slice ``` (commit 1d2d25dcadda0764f303183ac091d0c87b432566) ((HEAD detached at 1d2d25dcad)) $ ./ruby --disable=gems script.rb ["안", "\xEB"] ((HEAD detached at 1d2d25dcad)) $ git checkout HEAD^ (114e71d06280f9c57b9859ee4405ae89a989ddb6) ((HEAD detached at 114e71d062)) $ make -j ... ((HEAD detached at 114e71d062)) $ ./ruby --disable=gems script.rb ["안"] ((HEAD detached at 114e71d062)) $ cat script.rb p "안녕".byteslice(0, 4).grapheme_clusters ``` the expected result here is almost certainly the latter output, and not the former. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:116894] [Ruby master Bug#20288] `rb_fiber_scheduler_close` exceptions are not handled in `rb_fiber_scheduler_set`.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20288 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20288: `rb_fiber_scheduler_close` exceptions are not handled in `rb_fiber_scheduler_set`. https://bugs.ruby-lang.org/issues/20288 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The code responsible for setting a new scheduler on a thread does not properly handle exceptions raised by `rb_fiber_scheduler_close`. If `rb_fiber_scheduler_close` raised an exception, the assignment `thread->scheduler = scheduler` would not be executed. This leaves the thread in an undefined state because it might still hold a reference to the old scheduler, which was supposed to be closed and replaced. ## Steps to Reproduce: 1. Define a custom fiber scheduler that raises an exception in its close method. 2. Set the custom scheduler on a thread. 3. Attempt to replace the custom scheduler with a different scheduler. 4. Observe that if the close method of the original scheduler raises an exception, the thread's scheduler reference is not updated. ## Proposed Fix: The use of `rb_ensure` can be introduced to wrap the call to `rb_fiber_scheduler_close` to ensure that, regardless of whether an exception is raised, `thread->scheduler` is set to `Qnil`, and then `thread->scheduler = scheduler` is safely executed to update the thread's scheduler reference. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:117321] [Ruby master Bug#20393] `after_fork_ruby` clears all pending interrupts for both parent and child process.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20393 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20393: `after_fork_ruby` clears all pending interrupts for both parent and child process. https://bugs.ruby-lang.org/issues/20393 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- In the following program, the behaviour of the parent process is affected by whether `Process.fork` is invoked or not. ```ruby Thread.handle_interrupt(RuntimeError => :never) do Thread.current.raise(RuntimeError, "Queued error") puts "Pending interrupt: #{Thread.pending_interrupt?}" # true pid = Process.fork do puts "Pending interrupt (child process): #{Thread.pending_interrupt?}" Thread.handle_interrupt(RuntimeError => :immediate){} end _, status = Process.waitpid2(pid) puts "Child process status: #{status.inspect}" puts "Pending interrupt: #{Thread.pending_interrupt?}" # false end puts "Exiting..." ``` I don't think the parent process pending interrupts should be cleared by `after_fork_ruby`: ```c static void after_fork_ruby(rb_pid_t pid) { rb_threadptr_pending_interrupt_clear(GET_THREAD()); if (pid == 0) { // child clear_pid_cache(); rb_thread_atfork(); } else { // parent after_exec(); } } ``` How about this implementation: ```c static void after_fork_ruby(rb_pid_t pid) { if (pid == 0) { // child rb_threadptr_pending_interrupt_clear(GET_THREAD()); clear_pid_cache(); rb_thread_atfork(); } else { // parent after_exec(); } } ``` cc @ko1 -- https://bugs.ruby-lang.org/

3 5

[ruby-core:116890] [Ruby master Bug#20286] TracePoint does not emit `thread_end` event when thread exits with exception
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20286 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20286: TracePoint does not emit `thread_end` event when thread exits with exception https://bugs.ruby-lang.org/issues/20286 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Using TracePoint to trace `thread_begin` and `thread_end` events fails to emit the `thread_end` event when an exception (e.g., Interrupt) is raised within a thread. This behavior occurs because the exception handling bypasses the internal thread finishing logic, including trace point and fiber scheduler cleanup code. This issue affects the ability to accurately monitor thread lifecycle events in scenarios involving exception handling or abrupt thread terminations. ## Steps to Reproduce: 1. Set up `TracePoint` to trace `thread_begin` and `thread_end` events. 2. Create a new thread that raises an exception. 3. Join the thread and observe that only the `thread_begin` event is emitted without a corresponding `thread_end` event. ## Expected Behavior: The `TracePoint` should emit both `thread_begin` and `thread_end` events, accurately reflecting the lifecycle of the thread, even when an exception is raised within the thread. ## Actual Behavior: The `TracePoint` emits the `thread_begin` event but fails to emit the `thread_end` event when an exception is raised within the thread, indicating an incomplete tracing of thread lifecycle events. I've confirmed this as far back as Ruby 2.6. ## Example Code ```ruby TracePoint.trace(:thread_begin, :thread_end) do |tp| p [tp.event, tp.lineno, tp.path, tp.defined_class, tp.method_id] end thread = Thread.new do raise Interrupt end thread.join ``` ## Possible Fix Changing the implementation of `thread_do_start` to have what amounts to an "ensure" block. ```c static void thread_do_start(rb_thread_t *th) { native_set_thread_name(th); VALUE result = Qundef; rb_execution_context_t *ec = th->ec; int state; EXEC_EVENT_HOOK(ec, RUBY_EVENT_THREAD_BEGIN, th->self, 0, 0, 0, Qundef); EC_PUSH_TAG(ec); if ((state = EC_EXEC_TAG()) == TAG_NONE) { switch (th->invoke_type) { case thread_invoke_type_proc: result = thread_do_start_proc(th); break; case thread_invoke_type_ractor_proc: result = thread_do_start_proc(th); rb_ractor_atexit(th->ec, result); break; case thread_invoke_type_func: result = (*th->invoke_arg.func.func)(th->invoke_arg.func.arg); break; case thread_invoke_type_none: rb_bug("unreachable"); } } EC_POP_TAG(); VALUE errinfo = ec->errinfo; if (!NIL_P(errinfo) && !RB_TYPE_P(errinfo, T_OBJECT)) { ec->errinfo = Qnil; } rb_fiber_scheduler_set(Qnil); EXEC_EVENT_HOOK(th->ec, RUBY_EVENT_THREAD_END, th->self, 0, 0, 0, Qundef); ec->errinfo = errinfo; if (state) EC_JUMP_TAG(ec, state); th->value = result; } ``` It's possible `rb_fiber_scheduler_set(Qnil);` can emit an exception itself. How do we write the code to handle that case? -- https://bugs.ruby-lang.org/

3 5

[ruby-core:117453] [Ruby master Bug#20413] Enumerator can block fiber scheduler.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20413 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20413: Enumerator can block fiber scheduler. https://bugs.ruby-lang.org/issues/20413 * Author: ioquatix (Samuel Williams) * Status: Closed * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Using `Enumerator` in the event loop can cause problems as the fiber created by `rb_fiber_new` is blocking by default. It should be non-blocking. See <https://github.com/ruby/ruby/pull/10478> for the fix. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117458] [Ruby master Bug#20414] `Fiber#raise` should recurse to `resumed_fiber` rather than failing.
by ioquatix (Samuel Williams) 06 Jul '24

06 Jul '24

Issue #20414 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20414: `Fiber#raise` should recurse to `resumed_fiber` rather than failing. https://bugs.ruby-lang.org/issues/20414 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following program will fail with `FiberError`, and is difficult to properly clean up: ```ruby root_fiber = Fiber.current f1 = Fiber.new do root_fiber.transfer end f2 = Fiber.new do f1.resume end f2.transfer f2.raise("error") # => `raise': attempt to transfer to a resuming fiber (FiberError) ``` This program deliberately set's up a scenario where `f2` is resuming `f1`. Trying to raise an exception on `f2` is impossible, because the only way control flow can return to it, is when `f1` yields or exits. We can avoid this problem, by raising the exception on f1, and we can do this automatically using the following logic: ```c static VALUE fiber_raise(rb_fiber_t *fiber, VALUE exception) { // Add this recursive step: if (fiber->resuming_fiber) { return fiber_raise(fiber->resuming_fiber, exception); } // Existing code ... else if (FIBER_SUSPENDED_P(fiber) && !fiber->yielding) { return fiber_transfer_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } else { return fiber_resume_kw(fiber, -1, &exception, RB_NO_KEYWORDS); } } ``` This makes `Fiber#raise` much more robust and useful for the purpose of stopping fibers, without knowing exactly what they are doing. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:117947] [Ruby master Bug#20499] Ruby builds on macOS store absolute paths for AR and NM in rbconfig since Ruby 3.2.3/3.3.0
by Bo98 (Bo Anderson) 06 Jul '24

06 Jul '24

Issue #20499 has been reported by Bo98 (Bo Anderson). ---------------------------------------- Bug #20499: Ruby builds on macOS store absolute paths for AR and NM in rbconfig since Ruby 3.2.3/3.3.0 https://bugs.ruby-lang.org/issues/20499 * Author: Bo98 (Bo Anderson) * Status: Open * ruby -v: ruby 3.2.4 (2024-04-23 revision af471c0e01) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is a regression introduced by https://github.com/ruby/ruby/commit/038f9ade3c4d965415e4956561975454cf9eeb21 and causes various downstream problems such as https://github.com/ruby/prism/issues/2716. Development tools on macOS are relocatable - Xcode.app can exist anywhere on the system so it cannot be assumed that the location on the build machine matches the location on the running machine. If an absolute path must be used, `/usr/bin/nm` and `/usr/bin/ar` are acceptable. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118398] [Ruby master Bug#20597] `eval('break if false')` should raise SyntaxError but retuns nil
by tompng (tomoya ishida) 03 Jul '24

03 Jul '24

Issue #20597 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20597: `eval('break if false')` should raise SyntaxError but retuns nil https://bugs.ruby-lang.org/issues/20597 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-06-27T13:47:22Z master c6a0d03649) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- These are all SyntaxError (Invalid break, compile error (SyntaxError)) ~~~ruby ruby -ce "break if false" ruby -ce "break if (false)" ruby -ce "break if nil" ruby -ce "break if (nil)" ruby -ce "break if 0>1" ~~~ But when it is passed to `eval`, some of them does not raise SyntaxError but return nil. ~~~ruby eval('break if false') #=> nil eval('break if (false)') #=> nil eval('break if nil') #=> nil eval('break if (nil)') #=> Can't escape from eval with break (SyntaxError) eval('break if 0>1') #=> Can't escape from eval with break (SyntaxError) ~~~ Same behavior with `next` `redo` and `yield` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118365] [Ruby master Feature#20589] Resize array in `rb_ary_freeze` and use `rb_ary_freeze` internally for arrays
by eileencodes (Eileen Uchitelle) 03 Jul '24

03 Jul '24

Issue #20589 has been reported by eileencodes (Eileen Uchitelle). ---------------------------------------- Feature #20589: Resize array in `rb_ary_freeze` and use `rb_ary_freeze` internally for arrays https://bugs.ruby-lang.org/issues/20589 * Author: eileencodes (Eileen Uchitelle) * Status: Open ---------------------------------------- GitHub PR https://github.com/ruby/ruby/pull/11030 This is a redo of https://github.com/ruby/ruby/pull/2640 and a new issue for the array portion of https://bugs.ruby-lang.org/issues/16291 because both are stale. This change proposes the following: 1) Call `ary_shrink_capa` from `rb_ary_freeze` to resize arrays before freezing (if they are not embedded, not shared, and not a shared root). 2) Update callers to use `rb_ary_freeze` instead of `rb_obj_freeze` internally in CRuby/ 3) Add an assertion to `ary_heap_realloc` that ensures frozen arrays are not being reallocated.\ The orignal issue implemented this for performance reasons, which are still valid. However, additionally this ensures that frozen arrays are not being passed to `ary_heap_realloc` and also ensures the capacity is set with `ARY_SET_CAPA` in `ary_shrink_capa`. Previously, because `ARY_SET_CAPA` was not called, `ary_heap_realloc` would get called after the array was frozen (when that is unnecessary). Array memsize before and after this change: Before: ```ruby $ require 'objspace' => false $ a = (1..5).to_a => [1, 2, 3, 4, 5] $ ObjectSpace.memsize_of(a) => 200 $ a.freeze => [1, 2, 3, 4, 5] $ ObjectSpace.memsize_of(a) => 200 ``` After: ```ruby $ require 'objspace' => false $ a = (1..5).to_a => [1, 2, 3, 4, 5] $ ObjectSpace.memsize_of(a) => 200 $ a.freeze => [1, 2, 3, 4, 5] $ ObjectSpace.memsize_of(a) => 80 ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118401] [Ruby master Bug#20599] TypeError: #<BaseMailer:0x00000000008070> is not a symbol nor a string since 4cbc41d5e5cb6793174d5964975fdb4470323ca1 with YJIT enabled
by yahonda (Yasuo Honda) 01 Jul '24

01 Jul '24

Issue #20599 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #20599: TypeError: #<BaseMailer:0x00000000008070> is not a symbol nor a string since 4cbc41d5e5cb6793174d5964975fdb4470323ca1 with YJIT enabled https://bugs.ruby-lang.org/issues/20599 * Author: yahonda (Yasuo Honda) * Status: Open * ruby -v: ruby 3.4.0dev (2024-06-26T20:01:26Z :detached: 4cbc41d5e5) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- https://buildkite.com/rails/rails-nightly/builds/711#01905b80-6b82-460c-908… ### Steps to reproduce ``` git clone https://github.com/rails/rails cd rails/actionmailer bundle install RUBY_YJIT_ENABLE=1 bin/test test/base_test.rb --seed 51482 ``` ### Expected behavior It should pass. ### Actual behavior It raises `TypeError: #<BaseMailer:0x00000000006cc0> is not a symbol nor a string` as follows. ```ruby ``` $ RUBY_YJIT_ENABLE=1 bin/test test/base_test.rb --seed 51482 Run options: --seed 51482 # Running: .............................E Error: BaseTest#test_explicit_multipart_with_one_template_has_the_expected_format: TypeError: #<BaseMailer:0x00000000006cc0> is not a symbol nor a string /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/base.rb:226:in 'AbstractController::Base#process_action' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/callbacks.rb:261:in 'block in AbstractController::Callbacks#process_action' /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/callbacks.rb:101:in 'ActiveSupport::Callbacks#run_callbacks' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/callbacks.rb:260:in 'AbstractController::Callbacks#process_action' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/base.rb:163:in 'AbstractController::Base#process' lib/action_mailer/rescuable.rb:29:in 'block in ActionMailer::Rescuable#process' lib/action_mailer/rescuable.rb:21:in 'ActionMailer::Rescuable#handle_exceptions' lib/action_mailer/rescuable.rb:28:in 'ActionMailer::Rescuable#process' /home/yahonda/src/github.com/rails/rails/actionview/lib/action_view/rendering.rb:40:in 'ActionView::Rendering#process' lib/action_mailer/base.rb:657:in 'block in ActionMailer::Base#process' /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/notifications.rb:212:in 'ActiveSupport::Notifications.instrument' lib/action_mailer/base.rb:656:in 'ActionMailer::Base#process' lib/action_mailer/message_delivery.rb:136:in 'block in ActionMailer::MessageDelivery#processed_mailer' <internal:kernel>:91:in 'Kernel#tap' lib/action_mailer/message_delivery.rb:135:in 'ActionMailer::MessageDelivery#processed_mailer' lib/action_mailer/message_delivery.rb:32:in 'ActionMailer::MessageDelivery#__getobj__' /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.4.0+0/delegate.rb:84:in 'Delegator#method_missing' test/base_test.rb:538:in 'block in <class:BaseTest>' bin/test test/base_test.rb:536 E Error: BaseTest#test_explicit_multipart: TypeError: #<BaseMailer:0x00000000006d10> is not a symbol nor a string /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/base.rb:226:in 'AbstractController::Base#process_action' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/callbacks.rb:261:in 'block in AbstractController::Callbacks#process_action' /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/callbacks.rb:101:in 'ActiveSupport::Callbacks#run_callbacks' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/callbacks.rb:260:in 'AbstractController::Callbacks#process_action' /home/yahonda/src/github.com/rails/rails/actionpack/lib/abstract_controller/base.rb:163:in 'AbstractController::Base#process' lib/action_mailer/rescuable.rb:29:in 'block in ActionMailer::Rescuable#process' lib/action_mailer/rescuable.rb:21:in 'ActionMailer::Rescuable#handle_exceptions' lib/action_mailer/rescuable.rb:28:in 'ActionMailer::Rescuable#process' /home/yahonda/src/github.com/rails/rails/actionview/lib/action_view/rendering.rb:40:in 'ActionView::Rendering#process' lib/action_mailer/base.rb:657:in 'block in ActionMailer::Base#process' /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/notifications.rb:212:in 'ActiveSupport::Notifications.instrument' lib/action_mailer/base.rb:656:in 'ActionMailer::Base#process' lib/action_mailer/message_delivery.rb:136:in 'block in ActionMailer::MessageDelivery#processed_mailer' <internal:kernel>:91:in 'Kernel#tap' lib/action_mailer/message_delivery.rb:135:in 'ActionMailer::MessageDelivery#processed_mailer' lib/action_mailer/message_delivery.rb:32:in 'ActionMailer::MessageDelivery#__getobj__' /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.4.0+0/delegate.rb:84:in 'Delegator#method_missing' test/base_test.rb:468:in 'block in <class:BaseTest>' bin/test test/base_test.rb:466 .E ... snip ... 100 runs, 119 assertions, 1 failures, 52 errors, 0 skips ``` According to git bisect, this type error reported since 4cbc41d5e5cb6793174d5964975fdb4470323ca1 It does not reproduce without YJIT enabled like `$ bin/test test/base_test.rb --seed 51482` -- https://bugs.ruby-lang.org/

2 2