April 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:121692] [Ruby Feature#3388] regexp support for start_with? and end_with?
by baweaver (Brandon Weaver) 20 Apr '25

20 Apr '25

Issue #3388 has been updated by baweaver (Brandon Weaver). I have admittedly found this confusing recently when I had attempted to use `end_with?(/regex/)` in a book example. Would it be acceptable to consider a non-ideal implementation of this feature, and then continue to optimize it in future releases? I believe the potential confusion relative to its dual method `start_with?` warrants some consideration of consistency in implementation here, even if not enough distinct cases exist exclusively for `end_with?` and regular expressions. ---------------------------------------- Feature #3388: regexp support for start_with? and end_with? https://bugs.ruby-lang.org/issues/3388#change-112743 * Author: trans (Thomas Sawyer) * Status: Feedback * Assignee: naruse (Yui NARUSE) ---------------------------------------- =begin ruby-1.9.2-head > "aBcdeFghIj".start_with?(/abc/i) => false In my implementation of start_with? it is easy enough to utilize #index, which works fine: def start_with?(pattern) index(pattern) == 0 end But #end_with? is more difficult, and I had to use regular expressions. def end_with?(suffix) suffix = Regexp.escape(suffix) if String===suffix /#{suffix}$/.match(self) ? true : false end However, we might get rid of the '? true : false' and return the MatchData, since that could be useful information. =end ---Files-------------------------------- 0001-string.c-rb_str_start_with-rb_str_end_with-allow-Reg.patch (4.08 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:121688] [Ruby Feature#21275] Update to Unicode 17.0
by duerst 19 Apr '25

19 Apr '25

Issue #21275 has been reported by duerst (Martin Dürst). ---------------------------------------- Feature #21275: Update to Unicode 17.0 https://bugs.ruby-lang.org/issues/21275 * Author: duerst (Martin Dürst) * Status: Open * Assignee: duerst (Martin Dürst) * Target version: 3.5 ---------------------------------------- Unicode 17.0 is currently in alpha. Beta is planned for late May, release for September 2025. We just updated to Unicode 16.0, so we can start testing with alpha/beta. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:121672] [Ruby Feature#21269] Opt in to raise for modification of chilled strings returned from `Symbol#to_s`
by Earlopain (Earlopain _) 17 Apr '25

17 Apr '25

Issue #21269 has been reported by Earlopain (Earlopain _). ---------------------------------------- Feature #21269: Opt in to raise for modification of chilled strings returned from `Symbol#to_s` https://bugs.ruby-lang.org/issues/21269 * Author: Earlopain (Earlopain _) * Status: Open ---------------------------------------- Ref https://bugs.ruby-lang.org/issues/20350 ```rb begin :foo.to_s << "bar" puts "ok sym" rescue puts "fail sym" end begin "foo" << "bar" puts "ok str" rescue puts "failed str" end ``` Output with `RUBYOPT="--enable=frozen-string-literal -W"`: ``` test.rb:2: warning: string returned by :foo.to_s will be frozen in the future ok sym failed str ``` `--enable=frozen-string-literal` does something for string literals, I would like the same for `Symbol#to_s`. I would suggest simply reusing `--enable=frozen-string-literal` but its name seems very clear about being about string literals, which doesn't really fit for what's going on with symbols here. For convenience, it would be very nice to have just a single switch for this though. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:121670] [Ruby Feature#21268] Implement a lock-free hash set for fstring table
by jhawthorn (John Hawthorn) 17 Apr '25

17 Apr '25

Issue #21268 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Feature #21268: Implement a lock-free hash set for fstring table https://bugs.ruby-lang.org/issues/21268 * Author: jhawthorn (John Hawthorn) * Status: Open * Assignee: jhawthorn (John Hawthorn) * Target version: 3.5 ---------------------------------------- I would like to propose replacing the st_table used for fstrings (de-duplicated frozen strings) with a lock-free hash set to have improved performance in a Ractor environment. Fstrings are shared between Ractors, and can be looked up concurrently by multiple Ractors, it's desirable to be able to do this work in parallel. There should be no change visible to users (other than improved performance :) ) Previously this table used the VM lock (`RB_VM_LOCK_ENTER`, not the GVL) to serialize access. I first experimented with using a reader-writer lock, but that was a little awkward to use as it was easy to cause a deadlock between the new lock, `RB_VM_LOCK_ENTER`, and `rb_vm_barrier()`. An RW lock also can cause cache ping ponging (though striping can help). Lastly, this table is somewhat write-heavy, so it's desirable for even writes to be possible in parallel. A lock free table is not necessarily faster than a table with locking, but due to these constraints and usage I believe a lock-free table is desirable here. The design is a fairly standard open-addressing hash table with quadratic probing. Similar to our existing st_table or id_table. Deletes (which happen on GC) replace existing keys with a tombstone, which is the only type of update which can occur. Tombstones are only cleared out on rebuilds (necessary for the lock-free design I used). The set is wait-free for lookup (the thread performing lookup always makes progress) and lock-free for insert (we may sometimes retry if another thread made a concurrent insertion, we never spin on a lock) unless resizing/rebuilding. The implementation itself is at https://github.com/ruby/ruby/pull/12921 and details are within. I will include benchmarks in a comment below. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:121674] [Ruby Bug#21270] init_fast_fallback_inetsock_internal (default for TCPSocket.new/TCPSocket.open) isn't fiber aware
by drahosj (Jake Drahos) 16 Apr '25

16 Apr '25

Issue #21270 has been reported by drahosj (Jake Drahos). ---------------------------------------- Bug #21270: init_fast_fallback_inetsock_internal (default for TCPSocket.new/TCPSocket.open) isn't fiber aware https://bugs.ruby-lang.org/issues/21270 * Author: drahosj (Jake Drahos) * Status: Open * ruby -v: ruby 3.4.2 * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Sockets created via `init_fast_fallback_inetsock_internal()` don't get a chance to call the scheduler hook(s). This is the default for connections created with TCPSocket.new unless fast_fallback is specified as false. Unfortunately, this has the effect of blocking all fibers in the thread if the initial connect() call hangs. TCPSocket.open() is used in Net::HTTP, which means the issue is present there. Here's a quick proof of concept. It works as intended as written (starts both fibers even though the first fiber hangs). Switching to the default `TCPSocket.new` call causes the whole thread to block. NOTE: Uses the "toy" scheduler from test/fiber/scheduler.rb, but the Socketry async gem is also affected (should affect all schedulers since the hook isn't called). ``` ruby require 'socket' require_relative 'scheduler' Fiber.set_scheduler(Scheduler.new) puts "#{Fiber.current.object_id}: Main fiber" Fiber.schedule do puts "#{Fiber.current.object_id}: Creating socket" # Assuming that attempting to connect to example.com on port 12345 hangs # Default causes the scheduler to hang and never create second fiber # TCPSocket.new("example.com", 12345) TCPSocket.new("example.com", 12345, fast_fallback: false) puts "#{Fiber.current.object_id}: Connected" end Fiber.schedule do puts "#{Fiber.current.object_id}: Sleeping" sleep 2 puts "#{Fiber.current.object_id}: Done sleeping" end puts "#{Fiber.current.object_id}: Both fibers started" ``` Running the working PoC: ``` shell $ ruby async-connect.rb 16: Main fiber 24: Creating socket 32: Sleeping 16: Both fibers started 32: Done sleeping [Hangs here until the socket connection eventually times out] ``` However, without `fast_fallback: false`, the `TCPSocket.new` call will block the entire scheduler, never creating the second fiber: ``` shell $ ruby async-request.rb 16: Main fiber 24: Creating socket [Hangs here until the connect times out] ``` Here's a stack dump of the hung version: ``` #0 __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56 #1 0x00007fc9e4c9581c in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=0, nr=270) at cancellation.c:49 #2 0x00007fc9e4c95871 in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=0, nr=270) at cancellation.c:75 #3 0x00007fc9e4d1af07 in __GI___select (nfds=<optimized out>, readfds=<optimized out>, writefds=<optimized out>, exceptfds=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/select.c:69 #4 0x00007fc9e5233c9f in rb_fd_select (n=<optimized out>, readfds=<optimized out>, writefds=<optimized out>, exceptfds=<optimized out>, timeout=<optimized out>) at /usr/src/debug/ruby-3.4.2/thread.c:4163 #5 0x00007fc9e5236cc6 in native_fd_select (n=<optimized out>, readfds=<optimized out>, writefds=<optimized out>, exceptfds=<optimized out>, timeout=<optimized out>, th=<optimized out>) at /usr/src/debug/ruby-3.4.2/thread_pthread.c:2380 #6 do_select (p=p@entry=140504668626880) at /usr/src/debug/ruby-3.4.2/thread.c:4314 #7 0x00007fc9e50e7416 in rb_ensure (b_proc=0x7fc9e52368b0 <do_select>, data1=140504668626880, e_proc=0x7fc9e5232440 <select_set_free>, data2=140504668626880) at /usr/src/debug/ruby-3.4.2/eval.c:1074 #8 0x00007fc9e523700f in rb_thread_fd_select (max=max@entry=21, read=read@entry=0x7fc9cadce430, write=write@entry=0x7fc9cadce440, except=except@entry=0x0, timeout=timeout@entry=0x0) at /usr/src/debug/ruby-3.4.2/thread.c:4374 #9 0x00007fc9c97b5050 in init_fast_fallback_inetsock_internal (v=v@entry=140504668627872) at /usr/src/debug/ruby-3.4.2/ext/socket/ipsocket.c:894 #10 0x00007fc9e50e7416 in rb_ensure ( b_proc=b_proc@entry=0x7fc9c97b4950 <init_fast_fallback_inetsock_internal>, data1=data1@entry=140504668627872, e_proc=e_proc@entry=0x7fc9c97b7880 <fast_fallback_inetsock_cleanup>, data2=data2@entry=140504668627872) at /usr/src/debug/ruby-3.4.2/eval.c:1074 #11 0x00007fc9c97b7f3c in rsock_init_inetsock (self=self@entry=140504646018560, remote_host=remote_host@entry=140504646018640, remote_serv=remote_serv@entry=24691, local_host=local_host@entry=4, local_serv=local_serv@entry=4, type=type@entry=0, resolv_timeout=<optimized out>, connect_timeout=<optimized out>, fast_fallback=<optimized out>, test_mode_settings=<optimized out>) at /usr/src/debug/ruby-3.4.2/ext/socket/ipsocket.c:1285 #12 0x00007fc9c97b8216 in tcp_init (argc=<optimized out>, argv=<optimized out>, sock=140504646018560) at /usr/src/debug/ruby-3.4.2/ext/socket/tcpsocket.c:76 ``` `tcp_init` is the TCPSocket#initialize defined in tcpsocket.c, which just calls into the pure-C stack without ever backing out to check io_wait as is done in other socket-creation code path. It looks like the main difference is that init_fast_fallback_inetsock_internal hasn't been updated to be Fiber-aware (eg. init_inetsock_internal calls `rsock_connect` which uses the Fiber-aware `wait_connectable`), but `init_fast_fallback_inetsock_internal` is hard-coded to use thread stuff. In case anybody stumbles across this wondering why `Net::HTTP` sometimes blocks the Fiber scheduler during connect(), here's a quick workaround by hacking default `fast_fallback: false` into `TCPSocket.open`: ``` ruby orig_open = TCPSocket.method(:open) TCPSocket.define_singleton_method(:open) do |*args, **kwargs, &block| kwargs[:fast_fallback] ||= false orig_open.call(*args, **kwargs, &block) end ``` A slightly fancier workaround would be to have TCPSocket#initialize default `fast_fallback` to false if the current scheduler is non-nil. That would get rid of unexpected scheduler blocking without requiring a rewrite of the fast fallback code to be fiber-aware. I'm not sure if that's as trivial to implement in a c-defined method as it is in a native Ruby definition. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:121507] [Ruby Bug#21212] IO::Buffer can be freed while its slice is locked
by hanazuki (Kasumi Hanazuki) 16 Apr '25

16 Apr '25

Issue #21212 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #21212: IO::Buffer can be freed while its slice is locked https://bugs.ruby-lang.org/issues/21212 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * ruby -v: ruby 3.5.0dev (2025-04-01T16:11:01Z master 30e5e7c005) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- ```ruby buffer = IO::Buffer.new(100) slice = buffer.slice buffer.locked do buffer.free rescue p $! #=> IO::Buffer::LockedError (expected) end slice.locked do p slice.locked? #=> true (expected) p buffer.locked? #=> false (what should this be?) slice.free rescue p $! #=> IO::Buffer::LockedError (expected) buffer.free # Should we allow this? slice.set_value(:U8, 0, 42) # raises IO::Buffer::InvalidatedError (surprising!) end ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:121540] [Ruby Feature#21216] Implement Set as a core class
by jeremyevans0 (Jeremy Evans) 15 Apr '25

15 Apr '25

Issue #21216 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Feature #21216: Implement Set as a core class https://bugs.ruby-lang.org/issues/21216 * Author: jeremyevans0 (Jeremy Evans) * Status: Open ---------------------------------------- I propose to implement Set as a core class. Set has been an autoloaded standard library since Ruby 3.2. The standard library Set is less efficient than it could be, as it uses Hash for storage, which stores unnecessary values for each item in the set. I've submitted a pull request that implements Set as a core class: https://github.com/ruby/ruby/pull/13074 Implementation details for the pull request: * Core Set uses a modified version of `st_table`, named `set_table`. Other than `s/st_/set_/`, the main difference is that the stored records do not have values, making them 1/3 smaller. `st_table_entry` stores `hash`, `key`, and `record` (value), while `set_table_entry` only stores `hash` and `key`. This results in large sets using ~33% less memory compared to stdlib Set. For small sets, core Set uses 20% less memory (160 bytes, while stdlib set uses 40 for Set and 160 for Hash). * All methods are implemented as cfuncs, except the pretty_print methods, which were moved to `lib/pp.rb` (which is where the pretty_print methods for other core classes are defined). As is typical for core classes, internal calls call C functions and not Ruby methods. For example, to check if something is a Set, `rb_obj_is_kind_of` is used, instead of calling `is_a?(Set)` on the related object. * Almost all methods use the same algorithm that the pure-Ruby implementation used. The exception is when calling `Set#divide` with a block with 2-arity. stdlib Set used a lazy-loaded tsort to implement this. I developed an algorithm that only allocates a single intermediate hash and does not need tsort. * The `flatten_merge` protected method is no longer necessary, so it is not implemented (it could be). * Similar to Hash/Array, subclasses of Set are no longer reflected in `inspect` output. * Documentation from stdlib Set was moved to core Set, with minor updates. I developed a comprehensive benchmark suite for all public Set methods (results attached). As you would expect, core Set is faster than stdlib Set in the majority of cases (90%), and multiple times faster in many cases (47% of cases are at least 2x faster, 31% of cases are at least 4x faster). There are a few cases where it is significantly slower: * `Set.new` with no arguments (~1.6x) * `Set#compare_by_identity` for small sets (~1.3x) * `Set#clone` for small sets (~1.5x) * `Set#dup` for small sets (~1.7x) Some of these are slower as Set does not currently use the AR table optimization that Hash does, so a new set_table is initialized for each Set. I'm not sure it's worth the complexity to have an AR table-like optimization for small sets (for hashes it makes sense, as small hashes are used everywhere in Ruby). The rbs and repl_type_completor bundled gems will need updates to support core Set. The pull request marks them as allowed failures. This passes all set tests with no changes. The following specs needed modification: * Modifying frozen set error message (changed for the better) * `Set#divide` when passed a 2-arity block no longer yields the same object as both the first and second argument (this seems like an issue with the previous implementation). * Set-like objects that override `is_a?` such that `is_a?(Set)` return `true` are no longer treated as Set instances. * `Set.allocate.hash` is no longer the same as `nil.hash` * `Set#join` no longer calls `Set#to_a` (it calls the underlying C function). * `Set#flatten_merge` protected method is not implemented. Previously, `set.rb` added a `SortedSet` autoload, which loads `set/sorted_set.rb`. This replaces the `Set` autoload in `prelude.rb` with a `SortedSet` autoload, but I recommend removing it and `set/sorted_set.rb` before the release of Ruby 3.5. This moves `test/set/test_set.rb` to `test/ruby/test_set.rb`, reflecting that switch to a core class. This does not move the spec files, as I'm not sure how they should be handled currently. Eventually, the set spec files should be moved to `spec/ruby/core`, but maybe not until Ruby 3.4 is no longer supported. This does not add any functions to the C-API (all functions in `set.c` are static). I think such functions can be added later as we find other places in core where it makes sense to use a Set instead of a Hash or Array, or as we get requests from extension authors. This is partially inspired by changes in Python 2.4 (released in November 2004), which changed the set implementation from a standard library class backed by a hash (dict in Python terms) to having it as a core class. ---Files-------------------------------- set-benchmark-output.txt (74.9 KB) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:120043] [Ruby master Bug#20919] IO#seek does not clear the character buffer in some cases while transcoding
by javanthropus (Jeremy Bopp) 14 Apr '25

14 Apr '25

Issue #20919 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Bug #20919: IO#seek does not clear the character buffer in some cases while transcoding https://bugs.ruby-lang.org/issues/20919 * Author: javanthropus (Jeremy Bopp) * Status: Open * ruby -v: ruby 3.4.0dev (2024-11-28T12:38:16Z master 3af1a04741) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When transcoding characters, `IO#seek` only clears the internal character buffer if `IO#getc` is called first: ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-16le')) # Character buffer WILL NOT be cleared f.seek(2, :SET) f.getc # => 'a'.encode('utf-16le'); should be '2'.encode('utf-16le') end Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind # Added a call to #getc here f.getc f.ungetc('a'.encode('utf-16le')) # Character buffer WILL be cleared now f.seek(2, :SET) f.getc # => '2'.encode('utf-16le') end ``` -- https://bugs.ruby-lang.org/

2 6

[ruby-core:120906] [Ruby master Feature#21121] Ractor channels
by luke-gru (Luke Gruber) 14 Apr '25

14 Apr '25

Issue #21121 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Feature #21121: Ractor channels https://bugs.ruby-lang.org/issues/21121 * Author: luke-gru (Luke Gruber) * Status: Open ---------------------------------------- # Motivation: It would be nice be able to `Ractor.yield` in a non-blocking way. Right now a `Ractor.yield` blocks until another ractor calls `r.take` on the yielding ractor. This is bad in the following scenario: ```ruby main = Ractor.current rs = 10.times.map do Ractor.new(main) do |m| ret = [] loop do # do a bunch of work that takes a while, then: begin obj = m.take rescue Ractor::ClosedError end if obj ret << obj else break end end ret end end 50.times do |i| Ractor.yield(i) # this will block until some ractor calls take on us, but it could be a while if there is processing before the `take` call. end main.close_outgoing # Ideally, we could do some work in main ractor that takes some time while the other ractors do their processing. But we're blocking right now # during all the calls to `Ractor.yield`. # Finally, get the results while rs.any? r, obj = Ractor.select(*rs) $stderr.puts "Ractor #{r} got #{obj}" rs.delete(r) end ``` I'd like other ractors to be able to do work in a "fire and forget" kind of way. It isn't possible right now due to the limitations of `Ractor.yield` and `Ractor#take`. What we need is some kind of `yield` buffer so the yielder doesn't block. I propose that Ractor channels would allow this type of programming to work. ## Example using channels: ```ruby chan = Ractor::Channel.new # We could specify buffer size like Go or it could be dynamically growable rs = 10.times.map do Ractor.new(chan) do |c| ret = [] loop do obj, _closed = c.receive # returns `[nil, true]` if `c` is closed and its buffer is empty. It blocks if the buffer is empty and `c` is not closed. if obj ret << obj else break end end ret end end 50.times do |i| chan.send(i) # non-blocking, fills a buffer and only wakes a receiver if there is one end chan.close # Do some processing while ractors do their work. # Then, collect the results while rs.any? r, obj = Ractor.select(*rs) puts "Ractor #{r} got #{obj}" rs.delete(r) end ``` With an API similar to this, we could have "fire and forget" processing with ractors. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:121611] [Ruby Bug#21261] Backport fix for crash in TCPSocket.open(..., nil)
by jhawthorn (John Hawthorn) 14 Apr '25

14 Apr '25

Issue #21261 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #21261: Backport fix for crash in TCPSocket.open(..., nil) https://bugs.ruby-lang.org/issues/21261 * Author: jhawthorn (John Hawthorn) * Status: Open * Backport: 3.2: DONTNEED, 3.3: DONTNEED, 3.4: REQUIRED ---------------------------------------- https://github.com/ruby/ruby/pull/12934 https://github.com/ruby/ruby/commit/b148dfef5aa208cafa28626fd84c9762bfdfe0ac -- https://bugs.ruby-lang.org/

2 2