November 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119636] [Ruby master Feature#20852] Anonymous HEREDOC blocks
by bradgessler (Brad Gessler) 07 Nov '24

07 Nov '24

Issue #20852 has been reported by bradgessler (Brad Gessler). ---------------------------------------- Feature #20852: Anonymous HEREDOC blocks https://bugs.ruby-lang.org/issues/20852 * Author: bradgessler (Brad Gessler) * Status: Open ---------------------------------------- Consider the following code. ```ruby # Consider the following code. Markdown.render <<~MARKDOWN # Hello there This is a Markdown file. See? 1. This is a list 2. With items 3. And more items MARKDOWN ``` There's a lot of "Markdown" that's repeated. ```ruby # Or this SQL.execute <<~SQL SELECT * FROM users WHERE id = 1; SQL ``` It's fine. It works. The key feature is that it understands the text in that block is indented. Not requiring the name could make the syntax less redundant and free developers up from having to name the string. Here's what that could look like: ```ruby # Now with 75% less markdown! Markdown.render <<~ # Hello there This is a Markdown file. See? 1. This is a list 2. With items 3. And more items ~>> # We can't sell SQL symbols fast enough! SQL.execute <<~ SELECT * FROM users WHERE id = 1; ~>> ``` I implement a lot of my application UI in Phlex. The dream is that I can build pages out that look like this: ```ruby class Form < ApplicationForm def view_template input(type: "text", name: "name", placeholder: "Name") fieldset{ h2 { "Agreement" } article(class: "prose"){ markdown <<~ ## Terms and Conditions By submitting this form, you agree to the terms and conditions of this website. # Privacy Policy We will not share your information with anyone. ... ~>> } } button{ "Submit" } end end ``` The closest I can get to that today is if I do this: ```ruby SQL.execute <<~_ SELECT * FROM users WHERE id = 1; _ ``` I've proposed a similar feature a while back and chimed in on similar proposals, but after thinking about it for a bit I realized "Anonymous HEREDOC blocks" is a slightly different approach that could be a different possible path forward. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119502] [Ruby master Feature#20793] Allow Multiple Arguments for the .is_a? Method
by artemb (Artem Borodkin) 07 Nov '24

07 Nov '24

Issue #20793 has been reported by artemb (Artem Borodkin). ---------------------------------------- Feature #20793: Allow Multiple Arguments for the .is_a? Method https://bugs.ruby-lang.org/issues/20793 * Author: artemb (Artem Borodkin) * Status: Open ---------------------------------------- I propose allowing multiple arguments to be passed to the .is_a? Method imply "OR" semantics: ``` ruby name.is_a? String, Symbol ``` Currently, we need to write the following to achieve the same functionality: ``` ruby [String, Symbol].include?(name.class) ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:119469] [Ruby master Feature#20786] Flow chaining with "then" keyword
by lpogic 07 Nov '24

07 Nov '24

Issue #20786 has been reported by lpogic (Łukasz Pomietło). ---------------------------------------- Feature #20786: Flow chaining with "then" keyword https://bugs.ruby-lang.org/issues/20786 * Author: lpogic (Łukasz Pomietło) * Status: Open ---------------------------------------- Hi, I would like to propose using the *"then"* keyword to create chained flows. ### Background: Original idea: https://bugs.ruby-lang.org/issues/20770#change-110056 However, the concept has changed slightly since then. I now assume that rhs is only calculated when lhs is true, which seems more compatible with the current usage of *"then"* keyword. I would also like to move the discussion here as it may cause unnecessary chaos there. ### Basics: In the canonical version, *"then"* is used in a *"begin..end"* block to split it into steps. When the result of the last expression before *"then"* is not *"false"* or *"nil"*, it is passed as an argument to the next step. Otherwise, all subsequent steps are skipped. The result of the *"begin..end"* block is the result of the last expression, or *"false"*/*"nil"* if any step was skipped. ### Example: ``` ruby # original code source: https://github.com/ruby/ruby/blob/a6383fbe1628bdaa9ec6a30b3baa60dd7430b461/… def include?(other) other = coerce_other(other) return false unless other.family == family begin_addr <= other.begin_addr && end_addr >= other.end_addr end # with chained flow def include?(other) begin coerce_other(other) then => coerced # then => name is my proposition of argument naming syntax return false unless coerced.family == family coerced then => it begin_addr <= it.begin_addr && end_addr >= it.end_addr end end ``` Please treat the canonical form as a starting point. As further improvements I would see: - implicit block argument name - implicit begin - beginless *"then"*, when lhs is just expression ### Example with futher improvements: ``` ruby def include?(other) coerce_other(other) then it.family == family ? it : false then begin_addr <= it.begin_addr && end_addr >= it.end_addr end # or def include?(other) coerce_other other then return false unless it.family == family begin_addr <= it.begin_addr && end_addr >= it.end_addr end end ``` ### *"then"* keyword as the pipeline operator: Under certain conditions, "then" could offer similar capabilities to the pipeline operator of functional languages: ``` ruby # assuming that "foo" and "bar" never return "false" or "nil" # instead of writing this: baz("string", bar(foo(), 2), 5) # you could write this: begin foo() then bar(it, 2) then baz("string", it, 5) end ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119670] [Ruby master Feature#20859] Make Base64 to core class
by hsbt (Hiroshi SHIBATA) 07 Nov '24

07 Nov '24

Issue #20859 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Feature #20859: Make Base64 to core class https://bugs.ruby-lang.org/issues/20859 * Author: hsbt (Hiroshi SHIBATA) * Status: Assigned * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- From https://bugs.ruby-lang.org/issues/20857#note-12 I also heard that `base64` gem has some issue for eco-system like https://github.com/ddnexus/pagy/pull/618. My fist intention is simply added `base64` gem into gem dependency by `Gem::Specification#add_dependency`. But some people rewrite `Base64.encode64` to `Array#pack`. It means gemification of `base64` is miss-direciton by me. I withdraw that and propose to make `Base64` into core class. https://github.com/ruby/ruby/pull/11977 It resolve dependency issues of `base64` as the default gems. TODO: * To release new version of `base64` gem that skip to load if `Base64` is available. * make `require "base64"` same as `require "enumlator"` -- https://bugs.ruby-lang.org/

6 7

[ruby-core:119050] [Ruby master Feature#20715] `Ractor.[]` and `Ractor.[]=` to access Ractor local storage
by ko1 (Koichi Sasada) 07 Nov '24

07 Nov '24

Issue #20715 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #20715: `Ractor.[]` and `Ractor.[]=` to access Ractor local storage https://bugs.ruby-lang.org/issues/20715 * Author: ko1 (Koichi Sasada) * Status: Open * Assignee: ko1 (Koichi Sasada) ---------------------------------------- Trivial proposal. Now `Ractor#[]/#[]=` is supported to access current ractor local storage. However, it doesn't allow to access local storage of other ractors. ```ruby Ractor.current[:foo] = 1 Ractor.new{ p Ractor.main[:foo] #=> nil }.take ``` So providing `Ractor.[]/[]=` is more reasonable. Further more, we don't need to use `Ractor#current` and it is slightly faster. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:119646] [Ruby master Feature#20855] Introduce `Fiber::Scheduler#blocking_region` to avoid stalling the event loop.
by ioquatix (Samuel Williams) 06 Nov '24

06 Nov '24

Issue #20855 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #20855: Introduce `Fiber::Scheduler#blocking_region` to avoid stalling the event loop. https://bugs.ruby-lang.org/issues/20855 * Author: ioquatix (Samuel Williams) * Status: Open ---------------------------------------- The fiber scheduler performance can be negatively affected when performing blocking operations that cannot be deferred to the event loopThe current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking IO for efficient task execution. ## Problem Description Fibers in Ruby are designed to improve performance and responsiveness by allowing concurrent tasks to proceed without blocking one another. However, certain operations inherently block the Fiber Scheduler, leading to delayed execution across other Fibers. When blocking operations are inevitable, such as system or CPU bound operations without event-loop support, they create bottlenecks that degrade the Scheduler's performance. ## Proposed Solution The proposed solution in PR #11963 introduces a blocking_region hook in the Fiber Scheduler to improve handling of blocking operations. This addition allows code that releases the GVL (Global VM Lock) code to be lifted out of the event loop, reducing the performance impact on the Scheduler during blocking calls. By isolating these operations from the primary event loop, this enhancement aims to improve Fiber-based concurrency, especially in cases where non-deferrable blocking operations would otherwise delay the Fiber Scheduler. The new Fiber Scheduler hook blocking_region is designed to accept an opaque callable object, which encapsulates work that can be offloaded to a thread pool for execution. By isolating these blocking tasks from the main event loop, this hook allows the Scheduler to maintain better responsiveness. This addition offers an efficient way to handle tasks that cannot run asynchronously within the event loop itself, leveraging multi-threading where necessary to keep Fibers running smoothly and minimize blocking impact. -- https://bugs.ruby-lang.org/

5 9

[ruby-core:119208] [Ruby master Bug#20745] IO::Buffer#copy triggers UB when src/dest buffers overlap
by hanazuki (Kasumi Hanazuki) 06 Nov '24

06 Nov '24

Issue #20745 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #20745: IO::Buffer#copy triggers UB when src/dest buffers overlap https://bugs.ruby-lang.org/issues/20745 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * ruby -v: ruby 3.4.0dev (2024-09-15T01:06:11Z master 532af89e3b) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The current implementation of `IO::Buffer#copy` uses `memcpy` to copy data between the two memory regions. `memcpy` has a requirement that the source and destination must not overlap; otherwise the behavior is undefined. When copying between the same instance of `IO::Buffer` (or slices sharing the same underlying memory), the rule can be violated, and the data is corrupted with some libc implementation / architecture combinations (note that Alpine uses musl libc). ```shell-session % docker run --platform=linux/amd64 --rm ruby:3.3.5-alpine3.20 ruby -e 'b=IO::Buffer.new(10); b.set_string("0123456789"); b.copy(b, 3, 7, 0); p b' -e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! #<IO::Buffer 0x00007fb439d2c450+10 INTERNAL> 0x00000000 30 31 32 30 31 32 30 31 32 30 0120120120 % docker run --platform=linux/arm64 --rm ruby:3.3.5-alpine3.20 ruby -e 'b=IO::Buffer.new(10); b.set_string("0123456789"); b.copy(b, 3, 7, 0); p b' -e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! #<IO::Buffer 0x00007fcc5c580360+10 INTERNAL> 0x00000000 30 31 32 30 31 32 33 34 35 36 0120123456 ``` Copying "0123456789" three bytes behind onto itself, "0120123456" is the expected result. --- This error can also be detected with ASAN. ```shell-session % CXX=clang++ CC=clang ./configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" % ./ruby -e 'b=IO::Buffer.new(10); b.copy(b, 0, 9, 1)' `RubyGems' were not loaded. `error_highlight' was not loaded. `did_you_mean' was not loaded. `syntax_suggest' was not loaded. -e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! ================================================================= ==1655425==ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges [0x5020000107b0,0x5020000107b9) and [0x5020000107b1, 0x5020000107ba) overlap #0 0x55dcfbb72d90 in __asan_memcpy (/home/kasumi/.local/src/github.com/ruby/ruby/ruby+0x1fdd90) (BuildId: 2591ca8e9e713537a8f388383df19d1f4284b722) #1 0x55dcfbcb31e3 in ruby_nonempty_memcpy /home/kasumi/.local/src/github.com/ruby/ruby/./include/ruby/internal/memory.h:662:16 #2 0x55dcfbcb3867 in io_buffer_memcpy /home/kasumi/.local/src/github.com/ruby/ruby/io_buffer.c:2347:5 #3 0x55dcfbcb354a in io_buffer_copy_from /home/kasumi/.local/src/github.com/ruby/ruby/io_buffer.c:2384:5 #4 0x55dcfbcafd2f in io_buffer_copy /home/kasumi/.local/src/github.com/ruby/ruby/io_buffer.c:2490:12 #5 0x55dcfc0b955f in ractor_safe_call_cfunc_m1 /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:3597:12 #6 0x55dcfc09ca28 in vm_call_cfunc_with_frame_ /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:3788:11 #7 0x55dcfc09cf2a in vm_call_cfunc_with_frame /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:3834:12 #8 0x55dcfc09c0e9 in vm_call_cfunc_other /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:3860:16 #9 0x55dcfc081811 in vm_call_cfunc /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:3942:12 #10 0x55dcfc07f347 in vm_call_method_each_type /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:4766:16 #11 0x55dcfc07ebc2 in vm_call_method /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:4892:20 #12 0x55dcfc02d9a4 in vm_call_general /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:4936:12 #13 0x55dcfc02fb24 in vm_sendish /home/kasumi/.local/src/github.com/ruby/ruby/./vm_insnhelper.c:5955:15 #14 0x55dcfc03eb00 in vm_exec_core /home/kasumi/.local/src/github.com/ruby/ruby/insns.def:898:11 #15 0x55dcfc0306a2 in rb_vm_exec /home/kasumi/.local/src/github.com/ruby/ruby/vm.c:2564:22 #16 0x55dcfc06ee9f in rb_iseq_eval_main /home/kasumi/.local/src/github.com/ruby/ruby/vm.c:2830:11 #17 0x55dcfbbb7a84 in rb_ec_exec_node /home/kasumi/.local/src/github.com/ruby/ruby/eval.c:281:9 #18 0x55dcfbbb74b2 in ruby_run_node /home/kasumi/.local/src/github.com/ruby/ruby/eval.c:319:30 #19 0x55dcfbbaf81e in rb_main /home/kasumi/.local/src/github.com/ruby/ruby/./main.c:43:12 #20 0x55dcfbbaf699 in main /home/kasumi/.local/src/github.com/ruby/ruby/./main.c:62:12 #21 0x7fb604914db9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #22 0x7fb604914e74 in __libc_start_main csu/../csu/libc-start.c:360:3 #23 0x55dcfbad8fe0 in _start (/home/kasumi/.local/src/github.com/ruby/ruby/ruby+0x163fe0) (BuildId: 2591ca8e9e713537a8f388383df19d1f4284b722) [snip] ``` -- https://bugs.ruby-lang.org/

5 7

[ruby-core:119432] [Ruby master Misc#20781] DevMeeting-2024-11-07
by mame (Yusuke Endoh) 06 Nov '24

06 Nov '24

Issue #20781 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20781: DevMeeting-2024-11-07 https://bugs.ruby-lang.org/issues/20781 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- # The next dev meeting **Date: 2024/11/07 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/11/04. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

8 10

[ruby-core:119757] [Ruby master Bug#20872] Undefined RUBY_API_VERSION_* macros used in <ruby/backward/cxxanyargs.hpp>
by hanazuki (Kasumi Hanazuki) 06 Nov '24

06 Nov '24

Issue #20872 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #20872: Undefined RUBY_API_VERSION_* macros used in <ruby/backward/cxxanyargs.hpp> https://bugs.ruby-lang.org/issues/20872 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * ruby -v: ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I just noticed that C++ compilers warn for undefined RUBY_API_VERSION_MAJOR/RUBY_API_VERSION_MINOR macros in `<ruby/backward/cxxanyargs.hpp>` when preprocessing `<ruby.h>`. ``` % cat test.cpp #include <ruby.h> % clang++ -Wundef -I ~/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0 -I ~/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/x86_64-linux -c test.cpp -o /dev/null In file included from test.cpp:1: In file included from /home/kasumi/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/ruby.h:38: In file included from /home/kasumi/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/ruby/ruby.h:27: In file included from /home/kasumi/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/ruby/internal/anyargs.h:83: /home/kasumi/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/ruby/backward/cxxanyargs.hpp:540:6: warning: 'RUBY_API_VERSION_MAJOR' is not defined, evaluates to 0 [-Wundef] #if (RUBY_API_VERSION_MAJOR * 100 + RUBY_API_VERSION_MINOR) >= 301 ^ /home/kasumi/.rbenv/versions/3.4.0-preview2/include/ruby-3.4.0+0/ruby/backward/cxxanyargs.hpp:540:37: warning: 'RUBY_API_VERSION_MINOR' is not defined, evaluates to 0 [-Wundef] #if (RUBY_API_VERSION_MAJOR * 100 + RUBY_API_VERSION_MINOR) >= 301 ^ 2 warnings generated. ``` The offending code was introduced in commit:01825e8bffd. The accompanying code comment says a deprecation warning should be emitted for some use-cases (passing ANYARGS function pointers to `rb_define_method`-series functions) with Ruby 3.x, which looks not working as intended for the current Ruby versions. This seems effectively doing no harm but just disturbing for extension developers. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119767] [Ruby master Bug#20874] Difference in documentation for `irb -U` between `man irb` and `irb --help`
by kyanagi (Kouhei Yanagita) 06 Nov '24

06 Nov '24

Issue #20874 has been reported by kyanagi (Kouhei Yanagita). ---------------------------------------- Bug #20874: Difference in documentation for `irb -U` between `man irb` and `irb --help` https://bugs.ruby-lang.org/issues/20874 * Author: kyanagi (Kouhei Yanagita) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- There is a difference in documentation for `irb -U` between `man irb` and `irb --help`. * `man irb`: Same as `ruby -U' . Sets the default value for internal encodings (Encoding.default_internal) to UTF-8. * `irb --help`: Set external and internal encodings to UTF-8. Which is correct? The implementation matches that of `--help`. However, it also seems to have been implemented with the intention of behaving the same as `ruby -U`. -- https://bugs.ruby-lang.org/

1 0