August 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118831] [Ruby master Feature#20673] Enable native SOCKS support by default
by MatzFan (Brian Cohen) 06 Sep '24

06 Sep '24

Issue #20673 has been reported by MatzFan (Brian Cohen). ---------------------------------------- Feature #20673: Enable native SOCKS support by default https://bugs.ruby-lang.org/issues/20673 * Author: MatzFan (Brian Cohen) * Status: Open ---------------------------------------- I'd like to see future Rubies have native SOCKS support enabled by default. Currently Rubies must be compiled with the `--enable-socks=true` option in order to provide access to the `SOCKSSocket` class, through which network connections may be made via the SOCKS protocol. My proposal is simply to deprecate the `--enable-socks` option IFO of SOCKS support being enabled by default. Use cases: To the best of my knowledge, the most commonly used SOCKS proxies are [Dante](https://www.inet.no/dante/) and [Tor](https://torproject.org). I know little of the former, but the latter is used as an anonymity tool. For example, the Tor Browser (based on Firefox) is built on the `tor` SOCKS proxy. Users wishing to make requests through the SOCKS protocol (through one of these proxies or otherwise) must compile their own Ruby with SOCKS enabled, or install Dante/Tor and make requests through these via another tool - e.g. a Selenium-driven browser, suitably configured. Alternatively, a lower level library like `cURL` can be utilized. Discussion: Most users do not build their own Rubies, but instead use the standard Rubies available from Ruby managers like `rvm` or `rbenv` (in both production and CI environments). Users needing to use the SOCKS protocol natively with one of these Rubies are presently reliant on the [socksify](https://rubygems.org/gems/socksify) Ruby gem (> 60M downloads). Declaration: I am the gem's current maintainer. The gem was originally built to provide native SOCKS support to users of these 'standard' Rubies via patches to the `Net/HTTP` and other Ruby core libraries. The current gem patches the private method `Net::HTTP#connect` in order to achieve this. This is a brittle and highly undesirable state of affairs and all code built using `socksify` is of course subject to breakage with each new Ruby release. It is difficult to gauge the exact nature and extent of the user need, but of the 60M Socksify downloads 20M have taken place is the last few years since I became maintainer. I consider this a significant user base. Many other languages provide native SOCKS support - e.g. Python has [PySocks](https://pypi.org/project/PySocks/). Fundamentally, is there any reason NOT to enable SOCKS support - would it break anything? I'd dearly like to retire the Socksify 'hack' in due course and see Ruby join the list of languages that offer SOCKS support by default - in all future Rubies. Many thanks for your consideration. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:118930] [Ruby master Feature#20692] Rewrite Array#bsearch in Ruby
by sebyx07 (Sebastian Buza) 06 Sep '24

06 Sep '24

Issue #20692 has been reported by sebyx07 (Sebastian Buza). ---------------------------------------- Feature #20692: Rewrite Array#bsearch in Ruby https://bugs.ruby-lang.org/issues/20692 * Author: sebyx07 (Sebastian Buza) * Status: Open ---------------------------------------- inspired by https://bugs.ruby-lang.org/issues/20182 Benchmark results (average over 10000000 iterations): user system total real Original bsearch: 7.011691 0.000038 7.011729 ( 7.013117) Native bsearch: 2.809711 0.001000 2.810711 ( 2.812067) https://github.com/sebyx07/native_ruby/blob/master/lib/native_ruby/array/bs… I also created this gem: https://github.com/sebyx07/native_ruby/ - to load patches w/o depending on the ruby/master -- https://bugs.ruby-lang.org/

4 5

[ruby-core:118745] [Ruby master Misc#20658] Dropping the "Arm Neoverse (Ubuntu) server on RubyCI
by jaruga (Jun Aruga) 06 Sep '24

06 Sep '24

Issue #20658 has been reported by jaruga (Jun Aruga). ---------------------------------------- Misc #20658: Dropping the "Arm Neoverse (Ubuntu) server on RubyCI https://bugs.ruby-lang.org/issues/20658 * Author: jaruga (Jun Aruga) * Status: Open ---------------------------------------- ## Context On July 26 2024, 4 days ago, the person at Arm sponsoring a SSH-accessing Arm server via Works on Arm program told me that they could no longer sponsor the server **after 10th August 2024**. We have been using Equinix's Arm-based virtual machine on their cloud infrastructure[2] for the server right now. The person at Arm told me that Arm is actually scaling down their servers at Equinix within the Works on Arm program. While the person at Arm suggested using their cloud offerings listed on the [1] for paid or free tier, I need to take time to investigate how to continue to use Arm-based SSH-accessing servers continuously within the Works on Arm program if we want to continue to use it. At least we need to drop the Arm-based server that is named as "Arm Neoverse (Ubuntu)" on RubyCI until 10th August 2024.[3] ## Note for Ruby project commiters using the "Arm Neoverse (Ubuntu)" server Dear commiters, Please back up your files on the "Arm Neoverse (Ubuntu)" server if you have the files. After 10th August 2024, we cannot access to the server by SSH any more. ## Steps to drop the "Arm Neoverse (Ubuntu)" I plan to do the following tasks to drop the "Arm Neoverse (Ubuntu)". * Drop the "Arm Neoverse (Ubuntu)" on RubyCI. * Remove the content about the "Ubuntu arm neoverse" on [our Wiki page](https://github.com/ruby/ruby/wiki/CI-Servers). * Drop the content about Works on Arm on www.ruby-lang.org, which I added by the pull-request <https://github.com/ruby/www.ruby-lang.org/pull/3183>. Please let me know if you find any other task about this topic. [1] Works on Arm: https://www.arm.com/markets/computing-infrastructure/works-on-arm [2] Equinix: https://www.equinix.com/ [3] RubyCI: https://rubyci.org/ -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118969] [Ruby master Feature#20702] Add `Array#fetch_values`
by toy (Ivan Kuchin) 05 Sep '24

05 Sep '24

Issue #20702 has been reported by toy (Ivan Kuchin). ---------------------------------------- Feature #20702: Add `Array#fetch_values` https://bugs.ruby-lang.org/issues/20702 * Author: toy (Ivan Kuchin) * Status: Open ---------------------------------------- `Array` and `Hash` have matching methods to fetch: * One value using `[]` (`Array` also allows fetching multiple with range or two arguments) * One value with default or exception using `fetch` method * Multiple values using `values_at` method But only `Hash` has method `fetch_values` to fetch multiple values with fallback or raising an exception (see #10017). ```ruby hash = {a: 1, b: 2, c: 3} array = [1, 2, 3] hash[:b] # => 2 hash[:d] # => nil array[1] # => 2 array[4] # => nil hash.fetch(:b) # => 2 hash.fetch(:d) # => IndexError hash.fetch(:d){ 42 } # => 42 array.fetch(1) # => 2 array.fetch(4) # => IndexError array.fetch(4){ 42 } # => 42 hash.values_at(:b, :c) # => [2, 3] hash.values_at(:b, :d) # => [2, nil] array.values_at(1, 2) # => [2, 3] array.values_at(1, 4) # => [2, nil] hash.fetch_values(:b, :c) # => [2, 3] hash.fetch_values(:b, :d) # => IndexError hash.fetch_values(:b, :d){ 42 } # => [2, 42] # missing array.fetch_values(1, 2) # => [2, 3] array.fetch_values(1, 4) # => IndexError array.fetch_values(1, 4){ 42 } # => [2, 42] ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118970] [Ruby master Feature#20703] Alias StringIO#string to StringIO#to_s
by sebyx07 (Sebastian Buza) 05 Sep '24

05 Sep '24

Issue #20703 has been reported by sebyx07 (Sebastian Buza). ---------------------------------------- Feature #20703: Alias StringIO#string to StringIO#to_s https://bugs.ruby-lang.org/issues/20703 * Author: sebyx07 (Sebastian Buza) * Status: Open ---------------------------------------- # Description Allow to use `StringIO.new('my string').to_s` to be the same as `StringIO.new('my string').string` IMO nobody really uses the current StringIO#to_s, so I don't think it will break other apps. That way you can also do interpolation easily ```ruby my_sio = StringIO.new('my string') puts "value: #{my_sio}" # value: my string ``` - issue: https://github.com/ruby/stringio/issues - pr: https://github.com/ruby/stringio/pull/103 -- https://bugs.ruby-lang.org/

4 4

[ruby-core:118994] [Ruby master Feature#20707] Move `Time#xmlschema` into core
by byroot (Jean Boussier) 05 Sep '24

05 Sep '24

Issue #20707 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20707: Move `Time#xmlschema` into core https://bugs.ruby-lang.org/issues/20707 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Performance Converting `Time` into `RFC3339 / ISO8601` representation is an significant hotspot for application that serialize data in JSON, XML or other formats. Right now this feature is currently available through the `time` default gem, so it rely on `Time#strftime`. While [`strftime` could certainly be optimized](https://github.com/ruby/ruby/pull/11508), ultimately it's a very generic API so can't make a lot of assumptions about the output. Whereas `ISO8601` is a very strict format, with very few dynamic part, so it's much easier for dedicated code to be well optimized, as the final size of the string can be computed upfront etc. [I experimented with this and was able to get a ~5x speedup with a fairly straightforward implementation](https://github.com/ruby/ruby/pull/11510). ``` compare-ruby: ruby 3.4.0dev (2024-08-29T13:11:40Z master 6b08a50a62) +YJIT [arm64-darwin23] built-ruby: ruby 3.4.0dev (2024-08-30T13:17:32Z native-xmlschema 34041ff71f) +YJIT [arm64-darwin23] warming up...... | |compare-ruby|built-ruby| |:-----------------------|-----------:|---------:| |time._xmlschema | 1.087M| 5.190M| | | -| 4.78x| |utc_time._xmlschema | 1.464M| 6.848M| | | -| 4.68x| |time._xmlschema(6) | 859.960k| 4.646M| | | -| 5.40x| |utc_time._xmlschema(6) | 1.080M| 5.917M| | | -| 5.48x| |time._xmlschema(9) | 893.909k| 4.668M| | | -| 5.22x| |utc_time._xmlschema(9) | 1.056M| 5.707M| | | -| 5.40x| ``` ### Usability Aside from the performance reason, `ISO8601` is so common that having this functionality built-in without needing to require anything would make sense to me. `xmlschema` is the most common reason why I require `time`, and in many case having it core would allow not to load it at all. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118891] [Ruby master Feature#20684] Add optimized instructions for frozen literal Hash and Array
by etienne 05 Sep '24

05 Sep '24

Issue #20684 has been reported by etienne (Étienne Barrié). ---------------------------------------- Feature #20684: Add optimized instructions for frozen literal Hash and Array https://bugs.ruby-lang.org/issues/20684 * Author: etienne (Étienne Barrié) * Status: Open ---------------------------------------- # Context Methods that take empty arrays or empty hashes as default values allocate a new object each time the method is called without the argument. Often they don't mutate the parameter. To prevent an allocation, in performance critical sections, a constant is defined that holds a frozen hash or array, and the constant is defined as the default value for the parameter. Here are some examples: Rails: https://github.com/rails/rails/blob/607d61e884237c223c24c6f47efa0b561dd8b63… Roda: https://github.com/jeremyevans/roda/blob/102926a02dcabc9a31674e3cf98f049139… dry-rb: https://github.com/dry-rb/dry-container/blob/1ee41bb109455d06bf22ebcbd94b05… and many other gems: https://gist.github.com/casperisfine/47f22243d4ad203855256ef5bfae7979 Additionally when defining a frozen literal constant, we're currently inefficient because we store the literal in the bytecode, we dup it just to freeze it again. It doesn't amount to much but would be nice to avoid. # Proposal Introduce 2 new optimized instructions `opt_ary_freeze` and `opt_hash_freeze` that behave like `opt_str_freeze` for their respective types. If the freeze method hasn't been redefined, they simply push the frozen literal value on the stack. Like for `opt_str_freeze`, these instructions are added by the peephole optimizer when applicable. In the specific case of empty array and empty hash, we use a pre-allocated global empty frozen object to avoid retaining a distinct empty object each time. This will allow code like this: https://github.com/ruby/ruby/blob/566f2eb501d94d4047a9aad4af0d74c6a96f34a9/… to be shortened and simplified like this: ```diff diff --git i/lib/rubygems/resolver/api_set/gem_parser.rb w/lib/rubygems/resolver/api_set/gem_parser.rb index 643b857107..34146fd426 100644 --- i/lib/rubygems/resolver/api_set/gem_parser.rb +++ w/lib/rubygems/resolver/api_set/gem_parser.rb @@ -1,15 +1,12 @@ # frozen_string_literal: true class Gem::Resolver::APISet::GemParser - EMPTY_ARRAY = [].freeze - private_constant :EMPTY_ARRAY - def parse(line) version_and_platform, rest = line.split(" ", 2) version, platform = version_and_platform.split("-", 2) dependencies, requirements = rest.split("|", 2).map! {|s| s.split(",") } if rest - dependencies = dependencies ? dependencies.map! {|d| parse_dependency(d) } : EMPTY_ARRAY - requirements = requirements ? requirements.map! {|d| parse_dependency(d) } : EMPTY_ARRAY + dependencies = dependencies ? dependencies.map! {|d| parse_dependency(d) } : [].freeze + requirements = requirements ? requirements.map! {|d| parse_dependency(d) } : [].freeze [version, platform, dependencies, requirements] end ``` Overall it's a minor optimization but also a very simple patch and makes code nicer. PR pending. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118542] [Ruby master Bug#20627] `require` on Ractor should run on the main Ractor
by ko1 (Koichi Sasada) 05 Sep '24

05 Sep '24

Issue #20627 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20627: `require` on Ractor should run on the main Ractor https://bugs.ruby-lang.org/issues/20627 * Author: ko1 (Koichi Sasada) * Status: Open * Assignee: ko1 (Koichi Sasada) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Now `require` on main Ractor is not allowed (raising error) but it is hard, especially for `autoload`. So let's allow `require` by running it on the main Ractor. ## Background On many libraries it is needed to run loading on the main Ractors because: 1. Setup constants with unshareable objects (such as `C = []`) are not allowed on non main Ractors. 2. Setup global variables and class variables are not allowed. `$LOADED_FEATURES` is also untouchable. 3. (maybe more reasons) So the `require` on non main Ractors is not allowed. However it is hard to program especially on `autoload`. Also dynamic `require` (`require` in methods) are not allowed too (`pp` method, for example). ## Proposal Allow `require` on non main Ractors by running `require` process on the main Ractor. ![](clipboard-202407111201-kaiem.png) (quoted on my talk at RubyKaigi 2024) ### `rb_ractor_interrupt_exec(target_ractor, func)` C-API Make a thread on `target_ractor` and run `func` (C function) on it. I think it is safe to expose on Ruby API because running `func` on a *newly created* thread (do not disturb running target threads). But now it is proposed as only (hidden) C-API. ### New Ractor methods - `Rcator.main?` returns Ractors - `Ractor.require(feature)` do `require` on the main Ractor These new methods are useful for users who override `require` method like [RubyGems](https://github.com/ruby/ruby/blob/master/lib/rubygems/core_ext/ke…. ```ruby alias orig_require require def require feature return Rator.require(feature) unless Ractor.main? # overriding require code end ``` Or we can prepend a module like: ``` Module.new do def require(feature) return Rator.require(feature) unless Ractor.main? super(feature) end Kernel.prepend self end ``` will support ractors for all overgrinding methods. But not sure it is acceptable to add additional one modules in ancestors by `prepend`. Also this technique doesn't support require overriding by prepending. ## Implementation https://github.com/ruby/ruby/pull/11142 (not matured yet) ---Files-------------------------------- clipboard-202407111201-kaiem.png (70.8 KB) -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118773] [Ruby master Misc#20660] DevMeeting-2024-09-05
by mame (Yusuke Endoh) 04 Sep '24

04 Sep '24

Issue #20660 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20660: DevMeeting-2024-09-05 https://bugs.ruby-lang.org/issues/20660 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- # The next dev meeting **Date: 2024/09/05 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/09/02. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

6 6

[ruby-core:118996] [Ruby master Bug#20708] EINTR while opening fifo isn't retried
by martin.dorey＠hds.com (Martin Dorey) 04 Sep '24

04 Sep '24

Issue #20708 has been reported by martin.dorey(a)hds.com (Martin Dorey). ---------------------------------------- Bug #20708: EINTR while opening fifo isn't retried https://bugs.ruby-lang.org/issues/20708 * Author: martin.dorey(a)hds.com (Martin Dorey) * Status: Open * ruby -v: ruby 3.4.0dev (2024-08-30T20:36:28Z master eef2121cf3) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I don't think this should fail: ``` martind@stormy:~/download/ruby$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' -e:1:in 'IO.read': Interrupted system call @ rb_sysopen - /tmp/badger (Errno::EINTR) from -e:1:in '<main>' martind@stormy:~/download/ruby$ ``` The behavior changed with 2.6.0-preview3 compared to 2.6.0-preview2: ``` martind@stormy:~/download/ruby-2.6.0-preview2$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' martind@stormy:~/download/ruby-2.6.0-preview3$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' Traceback (most recent call last): 1: from -e:1:in `<main>' -e:1:in `read': Interrupted system call @ rb_sysopen - /tmp/badger (Errno::EINTR) martind@stormy:~/download/ruby-2.6.0-preview3$ ``` I don't know what the consequences would be, so I don't mean this as a serious suggestion: ``` martind@stormy:~/download/ruby$ git diff diff --git a/io.c b/io.c index 54b60b7357..590e8d11d7 100644 --- a/io.c +++ b/io.c @@ -6956,7 +6956,10 @@ sysopen_func(void *ptr) static inline int rb_sysopen_internal(struct sysopen_struct *data) { - int fd = IO_WITHOUT_GVL_INT(sysopen_func, data); + int fd; + do { + fd = IO_WITHOUT_GVL_INT(sysopen_func, data); + } while (fd < 0 && errno == EINTR); if (0 <= fd) rb_update_max_fd(fd); return fd; martind@stormy:~/download/ruby$ ``` ... but it fixes this particular test case, for me, with today's Ruby from github. There were negligible differences between 2.6.0-preview{2,3} in io.c but the SIGCHLD handling was changed. The test case is reliable for me on at least: Linux stormy 4.19.0-26-amd64 #1 SMP Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux Linux moritz 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux I hope it's independent of the kernel, as: https://man7.org/linux/man-pages/man7/signal.7.html ... teaches: > otherwise the call fails with the error EINTR: ... > • open(2), if it can block (e.g., when opening a FIFO; see > fifo(7)). -- https://bugs.ruby-lang.org/

2 1