December 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119621] [Ruby master Bug#20816] Potential regression in Ruby 3.3.x (compared with 3.1 and 3.2) regarding fast syscalls and multi-threading.
by adrienjarthon (Adrien Jarthon) 14 Jan '25

14 Jan '25

Issue #20816 has been reported by adrienjarthon (Adrien Jarthon). ---------------------------------------- Bug #20816: Potential regression in Ruby 3.3.x (compared with 3.1 and 3.2) regarding fast syscalls and multi-threading. https://bugs.ruby-lang.org/issues/20816 * Author: adrienjarthon (Adrien Jarthon) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ok I'm sorry in advance if this is not really a regression (maybe it's the cost to pay for other improvements) but it still showed as a 30% performance drop in my benchmark so I think it may be worth a look. For the full story about how I came down this rabbit hole I just wrote a blog post to avoid cluttering the issue: https://updown.io/blog/weird-results-comparing-ruby-3-1-3-2-3-3-with-jemall… (please read it first) Here I'm gonna put my last findings and the smallest reproducible example I was able to write. So basically I noticed that quick syscalls (at least with a good SSD) like `File.exists?` or `File.mtime`, when used in a multi-threaded environement (Sidekiq in my initial testing but I reduced it to a few Thread.new) are significantly slower in 3.3.x (tested with the same results on 3.3.0, 3.3.2 and 3.3.5 for the moment) in comparison with 3.1.2 or 3.2.5: about twice as slow. These syscalls are heavily used by the Rails Reloader which is how it had such a huge impact (30%) in my testing. Here is the small reproduction script I wrote: ``` ruby #!/usr/bin/env ruby GC.disable # just to make sure it doesn't skew results THREADS = (ARGV.first || 10).to_i # choose thread count from CLI N = 10_000_000 # to make the test longer or shorter puts "Ruby #{RUBY_VERSION} / YJIT: #{RubyVM::YJIT.enabled?} / #{THREADS} Threads" Array.new(THREADS).map do Thread.new do (N/THREADS).times { File.mtime(__FILE__) } end end.each(&:join) puts "#{N} calls completed" ``` The goal is just to run a lot of fast syscalls, in various number of threads, and here are the numbers on my machine: |Threads\Ruby|3.1.2 |3.3.5 | |--|--|--| |1 | 13.7s | ✅ 14.2s | |2 | 9.7s | ⚠️ 33.4s | |4 | 23.2s | ⚠️ 37.0s | |8 | 21.8s | ⚠️ 39.7s | |16 | 23.1s | ⚠️ 41.5s | (more numbers in the [blog post](https://updown.io/blog/weird-results-comparing-ruby-3-1-3-2-3-3-with-jemalloc-and-yjit)) I suspected the [new N:M thread scheduler introduced in 3.3](https://github.com/ruby/ruby/pull/8629), even though it's "disabled" by default on the main Ractor, thinking maybe these changes degraded a bit this (quite specific scenario). So I gave a quick try with the feature ENABLED, to see if it would at least change anything and it did ! (though not was I expected): |Threads\Ruby|3.1.2 |3.3.5 | 3.3.5 with RUBY_MN_THREADS=1 | |--|--|--|--| |1 | 13.7s | 14.2s | 14.5s ✅ | |2 | 9.7s | ⚠️ 33.4s | 12.3s 🎉 | |4 | 23.2s | ⚠️ 37.0s | 40.5s 🧐 | |8 | 21.8s | ⚠️ 39.7s | 37.9s 🧐 | |16 | 23.1s | ⚠️ 41.5s | 36.3s 🧐 | (I can make graphs for these numbers if it helps) Now I'm a bit stuck in my investigation, I'm not sure what change in 3.3 caused this (the new thread scheduler is a good lead IMO) or if it even is a regression. Maybe this downside is innevitable for other benefits? I am happy to contribute more time to this issue and perform more tests but I would need some guidance on what could be interesting to test, from people who understand a bit better than me how this works internally. Thanks ! (and sorry for the long read) -- https://bugs.ruby-lang.org/

4 5

[ruby-core:112005] [Ruby master Bug#19374] Issue with Ractor.make_shareable with curried procs
by luke-gru (Luke Gruber) 14 Jan '25

14 Jan '25

Issue #19374 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19374: Issue with Ractor.make_shareable with curried procs https://bugs.ruby-lang.org/issues/19374 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This works, but shouldn't: ```ruby class Worker def start(&blk) blk = blk.curry # bug in ruby allows sharing of non-shareable proc Ractor.make_shareable(blk) @ractor = Ractor.new(blk) do |b| main = b.call p "from ractor: #{main}" end end def work @ractor.take end end worker = Worker.new a = self # unshareable main object p "from main: #{a}" worker.start { a } worker.work ``` The curried proc has a reference to the original proc and it's not checked for shareability. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:118284] [Ruby master Misc#20575] Syntax Moratorium til 3.4
by matz (Yukihiro Matsumoto) 14 Jan '25

14 Jan '25

Issue #20575 has been reported by matz (Yukihiro Matsumoto). ---------------------------------------- Misc #20575: Syntax Moratorium til 3.4 https://bugs.ruby-lang.org/issues/20575 * Author: matz (Yukihiro Matsumoto) * Status: Open ---------------------------------------- Hi, As some may remember from the closing keynote at RubyKaigi, we are introducing Syntax Moratorium to support fair development of parsers. During the Syntax Moratorium period, new syntactical changes will be suspended (changes not includes bug fixes, of course). The moratorium period is scheduled to end with the release of 3.4, since the respective parser developers told us they did not need a very long period of time. Matz. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119944] [Ruby master Feature#20899] Reconsider adding `Array#find_map`
by toy (Ivan Kuchin) 12 Jan '25

12 Jan '25

Issue #20899 has been reported by toy (Ivan Kuchin). ---------------------------------------- Feature #20899: Reconsider adding `Array#find_map` https://bugs.ruby-lang.org/issues/20899 * Author: toy (Ivan Kuchin) * Status: Open ---------------------------------------- I would like to retry proposing method `Array#find_map` that was rejected in [8421](https://bugs.ruby-lang.org/issues/8421) which happened before introduction of `filter_map` in [15323](https://bugs.ruby-lang.org/issues/15323). It would make code nicer whenever there is a need to get the first truthy result of applying some code. Adapting examples from `filter_map` documentation, but if I need only the first value: ```rb (1..9).find_map {|i| i * 2 if i.even? } # => 4 {foo: 0, bar: 1, baz: 2}.find_map {|key, value| key if value.even? } # => :foo ``` Or an example of getting match group for first successful match: ```rb list = ['some 123', 'list 234', 'of 345', 'strings 456'] list.find_map{ |s| s[/\Aof (\d+)\z/, 1] } # => "345" ``` Currently I imagine either more code and/or inefficiency (extra calls and/or objects): ```rb # code called twice list.find{ |s| s[/\Aof (\d+)\z/, 1] }&.then{ |s| s[/\Aof (\d+)\z/, 1] } # => "345" # more logic result = nil list.each do |s| break if (result = s[/\Aof (\d+)\z/, 1]) end result # => "345" # or result = nil list.find do |s| result = s[/\Aof (\d+)\z/, 1] end result # => "345" # extra calls for items which come after item that we were looking for list.map{ |s| s[/\Aof (\d+)\z/, 1] }.find{ _1 } # => "345" # using lazy list.lazy.map{ |s| s[/\Aof (\d+)\z/, 1] }.find{ _1 } # => "345" # or as suggested by @alexbarret in https://bugs.ruby-lang.org/issues/8421?tab=history#note-7 list.lazy.filter_map{ |s| s[/\Aof (\d+)\z/, 1] }.first # => "345" # using tricks, as suggested by @zverok in https://bugs.ruby-lang.org/issues/8421?tab=history#note-4 list.find{ |s| result = s[/\Aof (\d+)\z/, 1] and break result } # => "345" ``` Implementation in ruby can be: ```rb Enumerable.class_eval do def find_map(&block) each do |element| block_result = block.call(element) return block_result if block_result end nil end end ``` An example from another language - scala method [`collect`](https://www.scala-lang.org/api/3.5.2/scala/collection/ArrayOps.html#collect-68e) works alike `filter_map` and [`collectFirst`](https://www.scala-lang.org/api/3.5.2/scala/collection/ArrayOps.html#collectFirst-25d) would be like `find_map`. -- https://bugs.ruby-lang.org/

5 5

[ruby-core:119272] [Ruby master Feature#20757] Make rb_tracearg_(parameters|eval_script|instruction_sequence) public C-API
by richardboehme 11 Jan '25

11 Jan '25

Issue #20757 has been reported by richardboehme (Richard Böhme). ---------------------------------------- Feature #20757: Make rb_tracearg_(parameters|eval_script|instruction_sequence) public C-API https://bugs.ruby-lang.org/issues/20757 * Author: richardboehme (Richard Böhme) * Status: Open ---------------------------------------- **Abstract** As a C-extension developer when using tracepoints I include "ruby/debug.h". This includes most of TracePoint's API but it seems like the C-equivalents for TracePoint#parameters, TracePoint#eval_script and TracePoint#instruction_sequence are missing/not being exported in the header. **Background** Most APIs like rb_tracearg_return_value are exported in "ruby/debug.h". If I understand correctly, the implementations for those methods are located in "ruby/vm_trace.c". The following methods implemented in "ruby/vm_trace.c" are missing in "ruby/debug.h": * rb_tracearg_parameters * rb_tracearg_eval_script * rb_tracearg_instruction_sequence **Proposal** I propose to add those methods to "ruby/debug.h". From my limiting understanding the change should be simple and not break backward compatibility, because we'd only need to add those function declarations to "ruby/debug.h". I'd be open to contribute this change if it was approved. **Use cases** I'm implementing a method call tracer for Ruby using the C-extension API. I wanted to get information about the parameters that the called method receives. When writing in Ruby this can be done using the TracePoint#parameters method, but I could not find the equivalent C-API. A workaround is to retrieve the method object (using the method_id) and check the method parameters. **See also** * Implementation of TracePoint#parameters in #14694 * Implementation of TracePoint#eval_script and TracePoint#instruction_sequence in #15287 -- https://bugs.ruby-lang.org/

3 4

[ruby-core:117826] [Ruby master Bug#20480] Complie failure for unable to configure NET_LUID using msys2 mingw64-gcc
by nekoyama32767 (Jinsong Yu) 10 Jan '25

10 Jan '25

Issue #20480 has been reported by nekoyama32767 (Jinsong Yu). ---------------------------------------- Bug #20480: Complie failure for unable to configure NET_LUID using msys2 mingw64-gcc https://bugs.ruby-lang.org/issues/20480 * Author: nekoyama32767 (Jinsong Yu) * Status: Open * ruby -v: ruby 3.4.0dev (2024-05-10T04:54:31Z master 2f915e729a) [x64-mingw32] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Master commit:`2f915e729ac8c66f4009f4b28a57773923d7e7d1` NET_LUID is not successfully checked in win32 configure. When try to compile in msys2 mingw64-gcc there will be a error message like follow. ``` win32/win32.c:4365:7: error: conflicting types for 'NET_LUID'; have 'struct <anonymous>' 4365 | } NET_LUID; | ^~~~~~~~ In file included from D:/msys64/mingw64/include/iptypes.h:17, from D:/msys64/mingw64/include/iphlpapi.h:17, from ./include/ruby/win32.h:40, from ./include/ruby/internal/dosish.h:38, from ./include/ruby/defines.h:78, from ./include/ruby/ruby.h:25, from win32/win32.c:24: D:/msys64/mingw64/include/ifdef.h:106:3: note: previous declaration of 'NET_LUID' with type 'NET_LUID' 106 | } NET_LUID, *PNET_LUID; | ^~~~~~~~ ``` Mingw64-gcc has NET_LUID, when comment out line 4357-line 4367 in `win32/win32.c` in follow, compile will be successed. ``` #ifndef HAVE_TYPE_NET_LUID typedef struct { uint64_t Value; struct { uint64_t Reserved :24; uint64_t NetLuidIndex :24; uint64_t IfType :16; } Info; } NET_LUID; #endif ``` mingw64-gcc version ``` gcc --version gcc.exe (Rev4, Built by MSYS2 project) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:120073] [Ruby master Feature#20925] Allow boolean operators at beginning of line to continue previous line
by Dan0042 (Daniel DeLorme) 09 Jan '25

09 Jan '25

Issue #20925 has been reported by Dan0042 (Daniel DeLorme). ---------------------------------------- Feature #20925: Allow boolean operators at beginning of line to continue previous line https://bugs.ruby-lang.org/issues/20925 * Author: Dan0042 (Daniel DeLorme) * Status: Open ---------------------------------------- I would like for this to become accepted syntax: condition1 || condition2 condition1 && condition2 condition1 or condition2 condition1 and condition2 This is similar to how method chaining on the second line was added in Ruby 1.9 expr .method And it has the same advantage: when you have a multi-line expression, instead of hunting for the dot or boolean operator at the end of line1, it's right there at the beginning of line2, making the structure very obvious and readable. Please contrast: request.secret_key_base.present? && request.encrypted_signed_cookie_salt.present? && request.encrypted_cookie_salt.present? && request.use_authenticated_cookie_encryption request.secret_key_base.present? && request.encrypted_signed_cookie_salt.present? && request.encrypted_cookie_salt.present? && request.use_authenticated_cookie_encryption The first expression must rely on indentation to communicate the multi-line nature of the condition, and even then it's not as immediately obvious as the second expression, where we can see easily and immediately that this is a multi-line `&&` condition. This syntax is also similar to how a trailing comma is allowed in arrays and hashes (and method calls since Ruby 1.9), with the same advantage. It makes for a cleaner diff when you add an element to the array/hash/conditional. Taking the previous example, imagine we are adding the condition `&& request.use_authenticated_cookie_encryption`. Now contrast the diff between the two styles: request.secret_key_base.present? && request.encrypted_signed_cookie_salt.present? && - request.encrypted_cookie_salt.present? + request.encrypted_cookie_salt.present? && + request.use_authenticated_cookie_encryption request.secret_key_base.present? && request.encrypted_signed_cookie_salt.present? && request.encrypted_cookie_salt.present? + && request.use_authenticated_cookie_encryption Based on the above I would say this syntax is natural and consistent with existing Ruby syntactical elements, and would greatly improve code readability. -- https://bugs.ruby-lang.org/

7 9

[ruby-core:120397] [Ruby master Bug#20980] Range#size new TypeError vs semi-open ranges
by zverok (Victor Shepelev) 09 Jan '25

09 Jan '25

Issue #20980 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #20980: Range#size new TypeError vs semi-open ranges https://bugs.ruby-lang.org/issues/20980 * Author: zverok (Victor Shepelev) * Status: Open * ruby -v: ruby 3.4.0dev (2024-12-24T14:40:12Z master 07e89bde46) +PRISM [x86_64-linux] * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONTNEED ---------------------------------------- Since #18984, `Range#size` throws `TypeError` if the `begin` is not iterable (doesn't respond to `#succ`). It produces one small inconvenience compared to 3.3: ```ruby ('a'..).size #=> nil (..'a').size # 3.3: nil # 3.4: can't iterate from NilClass (TypeError) -- well, probably makes sense # BUT: (3..).size #=> Infinity -- cool! (..3).size # 3.3: Infinity -- reasonable? # 3.4: can't iterate from NilClass (TypeError) -- worse? ``` I believe that while 3.4's behavior is explainable, 3.3's one has better DX. (Obviously too late to change before the release, but maybe might be considered for some patch version or Ruby 3.5?..) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:120350] [Ruby master Bug#20974] Required and optional anonymous parameter show differently in Proc#parameters
by alanwu (Alan Wu) 09 Jan '25

09 Jan '25

Issue #20974 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20974: Required and optional anonymous parameter show differently in Proc#parameters https://bugs.ruby-lang.org/issues/20974 * Author: alanwu (Alan Wu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following has had the same output since Ruby 1.9, when Proc#parameters was [introduced]: ```ruby p(proc { |(_a)| }.parameters) # => [[:opt, nil]] p(lambda { |(_a)| }.parameters) # => [[:req]] ``` I think they should be `[[:opt]]` and `[[:req]]` or `[[:opt, nil]]` and `[[:req, nil]]`. I prefer the nil-free option since that feels more anonymous. [added]: https://docs.ruby-lang.org/en/master/NEWS/NEWS-1_9_2.html -- https://bugs.ruby-lang.org/

2 2

[ruby-core:120428] [Ruby master Feature#20987] Add dbg - minimal debugging helper
by pawurb (Pawel Urbanek) 09 Jan '25

09 Jan '25

Issue #20987 has been reported by pawurb (Pawel Urbanek). ---------------------------------------- Feature #20987: Add dbg - minimal debugging helper https://bugs.ruby-lang.org/issues/20987 * Author: pawurb (Pawel Urbanek) * Status: Open ---------------------------------------- Hi. It's my first time contributing here, so I'm sorry in advance if I've mixed something up. I’m author of https://github.com/pawurb/dbg-rb gem. `dbg` method is inspired by Rust where it's built-in into std-lib (https://doc.rust-lang.org/std/macro.dbg.html) AFAIK in Ruby there's no simple mechanism to puts debug values together with caller info without using external dependencies. What’s more frustrating is that while `p nil` outputs `nil` to the std, `puts nil` prints a blank line, sometimes making debugging sessions confusing. I would like to propose adding a minimal `dbg` helper method to stdlib: ``` dbg("Hello world", [1, 2, 3]) # => [dir/file.rb:12] "Hello world" # => [dir/file.rb:12] [1, 2, 3] ``` `dbg` will produce verbose output together with informative file name and LOC info. I think that such built-in feature would be useful for many Ruby devs. My gem uses external dependencies, but I've came up with this barebones implementation: ``` def dbg(*msgs) loc = caller_locations.first.to_s matching_loc = loc.match(/.+(rb)\:\d+\:(in)\s/) src = if matching_loc.nil? loc else matching_loc[0][0..-5] end file, line = src.split(":") file = file.split("/").last(2).join("/") src = "[#{file}:#{line}]" msgs.each do |msg| puts "#{src} #{msg.inspect}" end nil end ``` ---Files-------------------------------- Screenshot 2024-12-27 at 00.00.23.png (81.5 KB) -- https://bugs.ruby-lang.org/

2 4