March 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:120710] [Ruby master Feature#21042] Add and expose Thread#memory_allocations memory allocation counters
by stanhu (Stan Hu) 08 May '25

08 May '25

Issue #21042 has been reported by stanhu (Stan Hu). ---------------------------------------- Feature #21042: Add and expose Thread#memory_allocations memory allocation counters https://bugs.ruby-lang.org/issues/21042 * Author: stanhu (Stan Hu) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- For the last 5 years, we've been patching our Ruby interpreter with https://github.com/ruby/ruby/pull/3978 in order to track memory allocations over time. This has been running in production at GitLab for a long time. I'd like to request approval for this patch to land upstream since we're getting tired of maintaining this patch, and this data seems like it would be generally useful. If this can be done via a C extension, let me know, and I can look at that. Copying from that pull request: ### Design This is designed to measure a memory allocations in a multi-threaded environments (concurrent requests processing) with an accurate information about allocated memory within a given execution context. The idea here is to provide as cheap as possible counter without an overhead of calling callbacks, and provide this information on a per-thread basis. ### Implementation This adds `Thread.current.memory_allocations`, which provides information about: * total_allocated_objects * total_malloc_bytes * total_mallocs This is based on a expectation, that allocation for a given thread always happens with a `rb_current_thread()` properly indicating a thread performing allocation. This measures total number of allocations as counters. Now, since the removal of objects is async and happening at random moment, the same cannot be said for deallocations, thus only allocations are tracked. -- https://bugs.ruby-lang.org/

5 5

[ruby-core:114025] [Ruby master Feature#19744] Namespace on read
by tagomoris (Satoshi TAGOMORI) 06 May '25

06 May '25

Issue #19744 has been reported by tagomoris (Satoshi TAGOMORI). ---------------------------------------- Feature #19744: Namespace on read https://bugs.ruby-lang.org/issues/19744 * Author: tagomoris (Satoshi TAGOMORI) * Status: Open * Priority: Normal ---------------------------------------- # What is the "Namespace on read" This proposes a new feature to define virtual top-level namespaces in Ruby. Those namespaces can require/load libraries (either .rb or native extension) separately from the global namespace. Dependencies of required/loaded libraries are also required/loaded in the namespace. ### Motivation The "namespace on read" can solve the 2 problems below, and can make a path to solve another problem: The details of those motivations are described in the below section ("Motivation details"). #### Avoiding name conflicts between libraries Applications can require two different libraries safely which use the same module name. #### Avoiding unexpected globally shared modules/objects Applications can make an independent/unshared module instance. #### (In the future) Multiple versions of gems can be required Application developers will have fewer version conflicts between gem dependencies if rubygems/bundler will support the namespace on read. ### Example code with this feature ```ruby # your_module module YourModule end # my_module.rb require 'your_module' module MyModule end # example.rb namespace1 = NameSpace.new namespace1.require('my_module') #=> true namespace1::MyModule #=> #<Module:0x00000001027ea650>::MyModule (or #<NameSpace:0x00...>::MyModule ?) namespace1::YourModule # similar to the above MyModule # NameError YourModule # NameError namespace2 = NameSpace.new # Any number of namespaces can be defined namespace2.require('my_module') # Different library "instance" from namespace1 require 'my_module' # require in the global namespace MyModule.object_id != namespace1::MyModule.object_id #=> true namespace1::MyModule.object_id != namespace2::MyModule.object_id ``` The required/loaded libraries will define different "instances" of modules/classes in those namespaces (just like the "wrapper" 2nd argument of `Kernel.load`). This doesn't introduce compatibility problems if all libraries use relative name resolution (without forced top-level reference like `::Name`). # "On read": optional, user-driven feature "On read" is a key thing of this feature. That means: * No changes are required in existing/new libraries (except for limited cases, described below) * No changes are required in applications if it doesn't need namespaces * Users can enable/use namespaces just for limited code in the whole library/application Users can start using this feature step by step (if they want it) without any big jumps. ## Motivation details This feature can solve multiple problems I have in writing/executing Ruby code. Those are from the 3 problems I mentioned above: name conflicts, globally shared modules, and library version conflicts between dependencies. I'll describe 4 scenarios about those problems. ### Running multiple applications on a Ruby process Modern computers have many CPU cores and large memory spaces. We sometimes want to have many separate applications (either micro-service architecture or modular monolith). Currently, running those applications require different processes. It requires additional computation costs (especially in developing those applications). If we have isolated namespaces and can load applications in those namespaces, we'll be able to run apps on a process, with less overhead. (I want to run many AWS Lambda applications on a process in isolated namespaces.) ### Running tests in isolated namespaces Tests that require external libraries need many hacks to: * require a library multiple times * require many different 3rd party libraries into isolated spaces (those may conflict with each other) Software with plugin systems (for example, Fluentd) will get benefit from namespaces. In addition to it, application tests can avoid unexpected side effects if tests are executed in isolated namespaces. ### Safely isolated library instances Libraries may have globally shared states. For example, [Oj](https://github.com/ohler55/oj) has a global `Obj.default_options` object to change the library behavior. Those options may be changed by any dependency libraries or applications, and it changes the behavior of `Oj` globally, unexpectedly. For such libraries, we'll be able to instantiate a safe library instance in an isolated namespace. ### Avoiding dependency hells Modern applications use many libraries, and those libraries require much more dependencies. Those dependencies will cause version conflicts very often. In such cases, application developers should resolve those by updating each libraries, or should just wait for the new release of libraries to conflict those libraries. Sometimes, library maintainers don't release updated versions, and application developers can do nothing. If namespaces can require/load a library multiple times, it also enables to require/load different versions of a library in a process. It requires the support of rubygems, but namespaces should be a good fundamental of it. ## Expected problems ### Use of top-level references In my expectation, `::Name` should refer the top-level `Name` in the global namespace. I expect that `::ENV` should contain the environment variables. But it may cause compatibility problems if library code uses `::MyLibrary` to refer themselves in their deeply nested library code. ### Additional memory consumption An extension library (dynamically linked library) may be loaded multiple times (by `dlopen` for temporarily copied dll files) to load isolated library "instances" if different namespaces require the same extension library. That consumes additional memory. In my opinion, additional memory consumption is a minimum cost to realize loading extension libraries multiple times without compatibility issues. This occurs only when programmers use namespaces. And it's only about libraries that are used in 2 or more namespaces. ### The change of `dlopen` flag about extension libraries To load an extension library multiple times without conflicting symbols, all extensions should stop sharing symbols globally. Libraries referring symbols from other extension libraries will have to change code & dependencies. (About the things about extension libraries, [Naruse also wrote an entry](https://naruse.hateblo.jp/entry/2023/05/22/193411).) # Misc The proof-of-concept branch is here: https://github.com/tagomoris/ruby/pull/1 It's still work-in-progress branch, especially for extension libraries. -- https://bugs.ruby-lang.org/

17 70

[ruby-core:120619] [Ruby master Bug#21030] Bug: #step with Range<ActiveSupport::Duration> behavior broken on Ruby 3.4.1
by johnnyshields (Johnny Shields) 05 May '25

05 May '25

Issue #21030 has been reported by johnnyshields (Johnny Shields). ---------------------------------------- Bug #21030: Bug: #step with Range<ActiveSupport::Duration> behavior broken on Ruby 3.4.1 https://bugs.ruby-lang.org/issues/21030 * Author: johnnyshields (Johnny Shields) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Issue also filed at: https://github.com/rails/rails/issues/54199 ### System configuration **Rails version**: 7.1.5.1 **Ruby version**: 3.3.4 (works) vs 3.4.1 (broken) ### What's broken Calling `#step` then `#to_a` on a `Range` whose beginning/end values are the same is expected to return an `Array` wrapping the `Range`'s singular value. In other words, `(x..x).step(n).to_a` should return `[x]` -- note this is also the same as `(x..x).to_a` The following case breaks on Ruby 3.4.1 related to `ActiveSupport::Duration` class, where it returns an empty array. It works on Ruby 3.3. Both tested with Rails 7.1.5.1. It's very easy to reproduce. I'm unsure if this is a Ruby bug or a Rails bug so I'm filing it in both places. The upgrade of Ruby is what produces the difference. ```ruby require 'active_support/all' # version 7.1.5.1 int = 100 dur = 100.seconds # ActiveSupport/Duration # The bug (dur..dur).step(10).to_a # => Ruby 3.3 returns [100 seconds] - CORRECT # => Ruby 3.4 returns [] - INCORRECT (int..dur).step(10).to_a # => Ruby 3.3 returns [100] - CORRECT # => Ruby 3.4 returns [] - INCORRECT # Note the following cases work correctly on both Ruby versions: (int..int).step(10).to_a #=> [100] (int..dur).to_a #=> [100] # Range with non-equivalent values also works fine on different versions: dur2 = 120.seconds (dur..dur2).step(10).to_a #=> [100 seconds, 110 seconds, 120 seconds] (int..dur2).step(10).to_a #=> [100, 110, 120] ``` -- https://bugs.ruby-lang.org/

6 10

[ruby-core:120855] [Ruby master Bug#21104] Net::HTTP connections failing in Ruby >= 3.4.0 on macOS with Happy Eyeballs enabled
by mjt58 (Mike Thompson) 05 May '25

05 May '25

Issue #21104 has been reported by mjt58 (Mike Thompson). ---------------------------------------- Bug #21104: Net::HTTP connections failing in Ruby >= 3.4.0 on macOS with Happy Eyeballs enabled https://bugs.ruby-lang.org/issues/21104 * Author: mjt58 (Mike Thompson) * Status: Open * ruby -v: ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- A project I work on recently upgraded Ruby to 3.4.1 from 3.3.5. Following the upgrade, and when running locally on my Mac, all attempts to connect to an external service within the project over http(s) fail. We use `mise` for managing development tool dependencies, including Ruby, and I am using macOS 15.3. For example running something as simple as: ``` require 'net/http' puts Net::HTTP.get(URI('https://bbc.co.uk')) ``` Will fail with the following stack trace: ``` /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'OpenSSL::SSL::SSLSocket#connect_nonblock': Connection reset by peer - SSL_connect (Errno::ECONNRESET) from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'Net::Protocol#ssl_socket_connect' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:1736:in 'Net::HTTP#connect' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:1636:in 'Net::HTTP#do_start' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:1625:in 'Net::HTTP#start' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:1064:in 'Net::HTTP.start' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:824:in 'Net::HTTP.get_response' from /path/to/.local/share/mise/installs/ruby/3.4.1/lib/ruby/3.4.0/net/http.rb:805:in 'Net::HTTP.get' from request.rb:2:in '<main>' ``` I tried different versions of Ruby and confirmed that the issue appears with 3.4.0. After reading the release notes for this version, I tried setting `RUBY_TCP_NO_FAST_FALLBACK=1` and this worked, allowing me to work around the problem. This issue has also been encountered by others, please see: https://github.com/rubygems/rubygems/issues/8390 -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116370] [Ruby master Bug#20203] `TestEnumerable` test failures with GCC 14
by vo.x (Vit Ondruch) 28 Apr '25

28 Apr '25

Issue #20203 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20203: `TestEnumerable` test failures with GCC 14 https://bugs.ruby-lang.org/issues/20203 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- There is ongoing mass rebuild in Fedora and that is first time GCC 14 is used and we observe test failures in `TestEnumerable`. Here are a few examples: ~~~ [ 3000/26419] TestEnumerable#test_transient_heap_sort_bymalloc_consolidate(): unaligned fastbin chunk detected ~~~ ~~~ [ 2455/26535] TestEnumerable#test_transient_heap_sort_bycorrupted size vs. prev_size in fastbins ~~~ ~~~ [ 9716/26532] TestEnumerable#test_any_with_unused_blockdouble free or corruption (fasttop) ~~~ The full logs are accessible [here](https://koji.fedoraproject.org/koji/taskinfo?taskID=112176941). Please drill through `Descendants` and `build.log` -- https://bugs.ruby-lang.org/

5 12

[ruby-core:121413] [Ruby Bug#21193] Inherited callback returns `nil` for `Object.const_source_location`
by eileencodes (Eileen Uchitelle) 20 Apr '25

20 Apr '25

Issue #21193 has been reported by eileencodes (Eileen Uchitelle). ---------------------------------------- Bug #21193: Inherited callback returns `nil` for `Object.const_source_location` https://bugs.ruby-lang.org/issues/21193 * Author: eileencodes (Eileen Uchitelle) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Since https://github.com/ruby/ruby/pull/12927 the inherited callback no longer can access the `Object.const_source_location`. A simplified reproduction is the following: ```ruby class A def self.inherited(other) super p Object.const_source_location(other.name) end end class B < A end ``` Prior to this change `Object.const_source_location(other.name)` would return `["test.rb", 8]` and now returns `nil`. Shopify's application uses the `inherited` callback to load configuration classes and since it's unable to find these classes they aren't loaded when they are called. It's not clear if this was intended or if there's a workaround. In chatting with folks at GitHub, they have a similar issue (although that case sounds less straightforward). A more accurate example of what we have (I did strip this down a bunch too): ```ruby def self.inherited(config) return super unless config.superclass == Config if (class_name = config.name) && (config_location = Object.const_source_location(class_name)) && (parent_dirname = File.dirname(config_location[0])) config_dirname = File.join(parent_dirname, "config") config.autoload(:Something, File.join(config_dirname, "something.rb")) end super end ``` Original issue: https://bugs.ruby-lang.org/issues/21143 cc/ @byroot @fxn -- https://bugs.ruby-lang.org/

6 14

[ruby-core:111526] [Ruby master Bug#19288] Ractor JSON parsing significantly slower than linear parsing
by maciej.mensfeld (Maciej Mensfeld) 18 Apr '25

18 Apr '25

Issue #19288 has been reported by maciej.mensfeld (Maciej Mensfeld). ---------------------------------------- Bug #19288: Ractor JSON parsing significantly slower than linear parsing https://bugs.ruby-lang.org/issues/19288 * Author: maciej.mensfeld (Maciej Mensfeld) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- a simple benchmark: ```ruby require 'json' require 'benchmark' CONCURRENT = 5 RACTORS = true ELEMENTS = 100_000 data = CONCURRENT.times.map do ELEMENTS.times.map do { rand => rand, rand => rand, rand => rand, rand => rand }.to_json end end ractors = CONCURRENT.times.map do Ractor.new do Ractor.receive.each { JSON.parse(_1) } end end result = Benchmark.measure do if RACTORS CONCURRENT.times do |i| ractors[i].send(data[i], move: false) end ractors.each(&:take) else # Linear without any threads data.each do |piece| piece.each { JSON.parse(_1) } end end end puts result ``` Gives following results on my 8 core machine: ```shell # without ractors: 2.731748 0.003993 2.735741 ( 2.736349) # with ractors 12.580452 5.089802 17.670254 ( 5.209755) ``` I would expect Ractors not to be two times slower on the CPU intense work. -- https://bugs.ruby-lang.org/

9 21

[ruby-core:120043] [Ruby master Bug#20919] IO#seek does not clear the character buffer in some cases while transcoding
by javanthropus (Jeremy Bopp) 14 Apr '25

14 Apr '25

Issue #20919 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Bug #20919: IO#seek does not clear the character buffer in some cases while transcoding https://bugs.ruby-lang.org/issues/20919 * Author: javanthropus (Jeremy Bopp) * Status: Open * ruby -v: ruby 3.4.0dev (2024-11-28T12:38:16Z master 3af1a04741) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When transcoding characters, `IO#seek` only clears the internal character buffer if `IO#getc` is called first: ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-16le')) # Character buffer WILL NOT be cleared f.seek(2, :SET) f.getc # => 'a'.encode('utf-16le'); should be '2'.encode('utf-16le') end Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind # Added a call to #getc here f.getc f.ungetc('a'.encode('utf-16le')) # Character buffer WILL be cleared now f.seek(2, :SET) f.getc # => '2'.encode('utf-16le') end ``` -- https://bugs.ruby-lang.org/

2 6

[ruby-core:120906] [Ruby master Feature#21121] Ractor channels
by luke-gru (Luke Gruber) 14 Apr '25

14 Apr '25

Issue #21121 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Feature #21121: Ractor channels https://bugs.ruby-lang.org/issues/21121 * Author: luke-gru (Luke Gruber) * Status: Open ---------------------------------------- # Motivation: It would be nice be able to `Ractor.yield` in a non-blocking way. Right now a `Ractor.yield` blocks until another ractor calls `r.take` on the yielding ractor. This is bad in the following scenario: ```ruby main = Ractor.current rs = 10.times.map do Ractor.new(main) do |m| ret = [] loop do # do a bunch of work that takes a while, then: begin obj = m.take rescue Ractor::ClosedError end if obj ret << obj else break end end ret end end 50.times do |i| Ractor.yield(i) # this will block until some ractor calls take on us, but it could be a while if there is processing before the `take` call. end main.close_outgoing # Ideally, we could do some work in main ractor that takes some time while the other ractors do their processing. But we're blocking right now # during all the calls to `Ractor.yield`. # Finally, get the results while rs.any? r, obj = Ractor.select(*rs) $stderr.puts "Ractor #{r} got #{obj}" rs.delete(r) end ``` I'd like other ractors to be able to do work in a "fire and forget" kind of way. It isn't possible right now due to the limitations of `Ractor.yield` and `Ractor#take`. What we need is some kind of `yield` buffer so the yielder doesn't block. I propose that Ractor channels would allow this type of programming to work. ## Example using channels: ```ruby chan = Ractor::Channel.new # We could specify buffer size like Go or it could be dynamically growable rs = 10.times.map do Ractor.new(chan) do |c| ret = [] loop do obj, _closed = c.receive # returns `[nil, true]` if `c` is closed and its buffer is empty. It blocks if the buffer is empty and `c` is not closed. if obj ret << obj else break end end ret end end 50.times do |i| chan.send(i) # non-blocking, fills a buffer and only wakes a receiver if there is one end chan.close # Do some processing while ractors do their work. # Then, collect the results while rs.any? r, obj = Ractor.select(*rs) puts "Ractor #{r} got #{obj}" rs.delete(r) end ``` With an API similar to this, we could have "fire and forget" processing with ractors. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:121389] [Ruby Bug#21187] Strings concatenated with `\` getting frozen with literal hashes (PRISM only)
by LocoDelAssembly 14 Apr '25

14 Apr '25

Issue #21187 has been reported by LocoDelAssembly (Hernán Pereira). ---------------------------------------- Bug #21187: Strings concatenated with `\` getting frozen with literal hashes (PRISM only) https://bugs.ruby-lang.org/issues/21187 * Author: LocoDelAssembly (Hernán Pereira) * Status: Open * ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- When the first elements of a literal hash are strings that are concatenated with `\`, those elements are flagged with `PM_NODE_FLAG_STATIC_LITERAL` and a special optimization that I believe was introduced in https://github.com/ruby/ruby/commit/8080de04be8e99e71309745822a9d436cc4ae37c causes the strings to be frozen. Reproduction === test.rb ``` a = { a: 'one' \ 'two', b: 'three' \ 'four', c: 'five', d: 'six' \ 'seven' } b = { a: 'one', b: 'two' \ 'three' } puts "a = #{a.map { |k,v| {k => v.frozen?} }}" puts "b = #{b.map { |k,v| {k => v.frozen?} }}" ``` With prism: ``` $ ruby test.rb a = [{a: true}, {b: true}, {c: false}, {d: false}] b = [{a: false}, {b: false}] ``` With parse.y: ``` $ ruby --parser=parse.y test.rb a = [{a: false}, {b: false}, {c: false}, {d: false}] b = [{a: false}, {b: false}] ``` (Notice `b` hash is unaffected in both parsers) Not sure if this is just part of undefined behavior or this is indeed a bug. Assigning a string concatenated with `\` to a variable doesn't make it frozen, to the best of my knowledge this seems to be hash-specific. -- https://bugs.ruby-lang.org/

2 1