November 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119769] [Ruby master Feature#20875] Atomic initialization for Ractor local storage
by ko1 (Koichi Sasada) 12 Dec '24

12 Dec '24

Issue #20875 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #20875: Atomic initialization for Ractor local storage https://bugs.ruby-lang.org/issues/20875 * Author: ko1 (Koichi Sasada) * Status: Open ---------------------------------------- ## Motivation Now there is no way to initialize Ractor local storage in multi-thread. For example, if we want to introduce per-Ractor counter, which should be protected with a per-Ractor Mutex for multi-threading support. ```ruby def init Ractor[:cnt] = 0 Ractor[:mtx] = Mutex.new end def inc init unless Ractor[:cnt] Ractor[:mtx].synchronize do Ractor[:cnt] += 1 end end ``` In this code, if `inc` was called on multiple threads, `init` can be called with multiple threads and `cnt` can not be synchronized correctly. ## Proposal Let's introduce `Ractor.local_storage_init(sym){ block }` to initialize values in Ractor local storage. If there is no slot for `sym`, synchronize with per-Ractor mutex and call `block` and the slot will be filled with the evaluation with the block result. The return value of this method will be the filled value. Otherwise, returning corresponding value will be returned. The implementation is like that (in C): ```ruby class Ractor def self.local_storage_init(sym) Ractor.per_ractor_mutex.synchronize do if Ractor.local_storage_has_key?(sym) Ractor[:sym] else Ractor[:sym] = yield end end end end ``` The above examples will be rewritten with the following code: ```ruby def inc Ractor.local_storage_init(:mtx) do Ractor[:cnt] = 0 Mutex.new end.synchronize do Ractor[:cnt] += 1 end end ``` ## Discussion ### Approach There are another approach like `pthread_atfork`, maybe like `Ractor.atcreate{ init }`. A library registers a callback which will be called when a new ractor is created. However, there are many Ractors which don't use the library, so that ``atcreate` can be huge overhead for Ractor creation. ### Naming I propose `local_storage_init`, but not sure it matches. I also proposed `Ractor.local_variable_init(sym)`, but Matz said he doesn't like this naming because it should not be a "variable". (there is a `Thread#thread_variable_get` method, though). On another aspect, `lcoal_storage_init` seems it clears all of ractor local storage slots. -- https://bugs.ruby-lang.org/

3 11

[ruby-core:120035] [Ruby master Bug#20918] Prism error indicates line number of `-e` that does not exist
by mame (Yusuke Endoh) 12 Dec '24

12 Dec '24

Issue #20918 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20918: Prism error indicates line number of `-e` that does not exist https://bugs.ruby-lang.org/issues/20918 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: prism * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` $ ruby -e 'foo(' -e: -e:2: syntax error found (SyntaxError) 1 | foo( > 2 | | ^ unexpected end-of-input; expected a `)` to close the arguments ``` It says `-e:2`, but there is no line 2 in `-e 'foo('`. `eval("foo(")` reports line 1, so I guess this issue is only for `-e`. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:118750] [Ruby master Bug#20659] Speed regression of `parse.y` parser after numeric nodes were introduced
by alanwu (Alan Wu) 12 Dec '24

12 Dec '24

Issue #20659 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20659: Speed regression of `parse.y` parser after numeric nodes were introduced https://bugs.ruby-lang.org/issues/20659 * Author: alanwu (Alan Wu) * Status: Open * ruby -v: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [`mail` benchmark from yjit-bench][1] is about 20% slower on `master` compared to commit:98eeadc9 ("Development of 3.4.0 started.") as the baseline, comparing running time of the Ruby process running the benchmark for a single iteration. Much of this workload is Ruby parsing. ``` $ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby' '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb' Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 889.0 ms ± 2.0 ms [User: 776.9 ms, System: 111.7 ms] Range (min … max): 885.7 ms … 892.1 ms 10 runs Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 891.7 ms ± 1.6 ms [User: 794.5 ms, System: 96.9 ms] Range (min … max): 889.0 ms … 894.7 ms 10 runs Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 1.086 s ± 0.003 s [User: 0.951 s, System: 0.134 s] Range (min … max): 1.080 s … 1.091 s 10 runs Summary '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' ran 1.00 ± 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.22 ± 0.00 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' $ for tag in 3.3-equiv master before-numeric-nodes; do echo $tag: $(~/.rubies/$tag/bin/ruby -v); done 3.3-equiv: ruby 3.4.0dev (2023-12-25T09:13:40Z master 98eeadc932) [x86_64-linux] master: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux] before-numeric-nodes: ruby 3.4.0dev (2024-01-06T18:26:38Z master 76afbda5b5) [x86_64-linux] ``` Using Valgrind's DHAT reveals that commit:1b8d01136c3ff6c60325c7609d61e19ac42acd9f ("Introduce Numeric Node's") issues roughly 3 times more `malloc` calls compared to the baseline, most of them comping from strdup() calls in set_number_literal(). Comparison with `--parser=prism`, for interest: <details> ``` $ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby,master/bin/ruby --parser=prism' \ '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb' Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 889.7 ms ± 2.6 ms [User: 771.4 ms, System: 118.0 ms] Range (min … max): 885.8 ms … 894.6 ms 10 runs Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 890.4 ms ± 1.6 ms [User: 776.8 ms, System: 113.3 ms] Range (min … max): 888.4 ms … 892.9 ms 10 runs Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 1.087 s ± 0.004 s [User: 0.968 s, System: 0.119 s] Range (min … max): 1.083 s … 1.097 s 10 runs Benchmark 4: ~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 826.9 ms ± 2.1 ms [User: 725.8 ms, System: 100.7 ms] Range (min … max): 823.6 ms … 830.9 ms 10 runs Summary '~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb' ran 1.08 ± 0.00 times faster than '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.08 ± 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.31 ± 0.01 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' ``` </details> [1]: https://github.com/Shopify/yjit-bench/blob/b8f1bd8931d4757afdae933ef6f11e07… -- https://bugs.ruby-lang.org/

2 4

[ruby-core:119632] [Ruby master Feature#20818] Allow passing a block to Hash#store (to update current value)
by furunkel (Julian Aron Prenner) 12 Dec '24

12 Dec '24

Issue #20818 has been reported by furunkel (Julian Aron Prenner). ---------------------------------------- Feature #20818: Allow passing a block to Hash#store (to update current value) https://bugs.ruby-lang.org/issues/20818 * Author: furunkel (Julian Aron Prenner) * Status: Open ---------------------------------------- I would like to propose a block form for `Hash#store`. In addition to passing a value, it should also be allowed to pass a block. If passed a block instead of a value, the block is called with the current value or, if unset, the hash's default value; the block's return value will be the value that is stored. This is similar to e.g., `computeIfPresent`/`computeIfAbsent` in Java (I think). I can think of several situations where this would be useful, in particular for caches and counters. For instance: ```ruby counts = {} elements.each do |element| counts.store(element){ (_1 || 0) + element.weight} end ``` or even more elegant with a default value: ```ruby counts = {} counts.default = 0 elements.each do |element| counts.store(element){ _1 + element.weight} end ``` Moreover, using the block form we should be able to do operations such as `h[k] ||= x`, `h[k] -= x`, `h[k] += x`, or more generally `h[k] = f(h[k])`, with a single "hashing round-trip". If I'm not mistaken, currently these involve two separate calls to `#[]` and `#[]=` (with two calls to `#hash`?). Finally, this makes `#store` a proper dual of `#fetch` which, similarly, can be passed a block. I have an experimental implementation of this (GitHub PR) at: https://github.com/ruby/ruby/pull/11956 -- https://bugs.ruby-lang.org/

3 2

[ruby-core:120050] [Ruby master Feature#20922] Should not we omit parentheses in assert calls?
by mame (Yusuke Endoh) 12 Dec '24

12 Dec '24

Issue #20922 has been reported by mame (Yusuke Endoh). ---------------------------------------- Feature #20922: Should not we omit parentheses in assert calls? https://bugs.ruby-lang.org/issues/20922 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- I often see the style of omitting parentheses in assert calls, but it leads to annoying warnings in the following case: ``` assert_equal -1, val #=> warning: ambiguous first argument; put parentheses or a space even after `-` operator assert_match /foo/, str #=> warning: ambiguous `/`; wrap regexp in parentheses or add a space after `/` operator ``` To deal with this warning, it is necessary to add parentheses only to *some* assert calls. ``` assert_equal 1, one assert_equal 0, zero assert_equal(-1, minus_one) ``` Obviously, this is not very cool. I feel that by these warnings, Ruby says "we should not omit parentheses in assert calls". Is this what matz intended? If it is the intent, I would like to add parentheses on all assert calls in tests of Ruby and default gems. If it is not the intent, why don't we remove these warnings? Note that, as far as I recall, I encountered this problem only with `assert_equal` and `assert_match`. I don't write parentheses in `p`, `puts`, `attr_reader`, `include`, etc., but all of them rarely accept `-1` and `//` literally as the first argument in real code. As a milder approach, I came up with an idea to stop the warning only when the method name starts with "assert". It is very ad-hoc, though. -- https://bugs.ruby-lang.org/

2 3

[ruby-core:119622] [Ruby master Bug#20817] Ruby 3.4.0dev emits `warning: possibly useless use of + in void context` while Ruby 3.3.5 does not
by yahonda (Yasuo Honda) 12 Dec '24

12 Dec '24

Issue #20817 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #20817: Ruby 3.4.0dev emits `warning: possibly useless use of + in void context` while Ruby 3.3.5 does not https://bugs.ruby-lang.org/issues/20817 * Author: yahonda (Yasuo Honda) * Status: Open * ruby -v: ruby 3.4.0dev (2024-10-25T11:52:32Z master 9c8c140d73) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ruby 3.4.0dev emits `warning: possibly useless use of + in void context` while Ruby 3.3.5 does not This issue is based on Rails Action Pack test case against Ruby 3.4.0dev. Here is the minimum script to reproduce it. ### Steps to reproduce - Create a sample foo.rb file ``` class C class_eval "def throw_syntax_error; eval %( 'abc' + pluralize 'def' ); end", "lib/file.rb", 42 end c = C.new c.throw_syntax_error ``` - Run this foo.rb using Ruby 3.4.0dev ### Expected behavior It should report SyntaxError, but it shoud not emit any warnings as Ruby 3.3.5 does not. ``` $ ruby -v ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux] $ ruby -w foo.rb lib/file.rb:42:in `eval': (eval at lib/file.rb:42):2: syntax error, unexpected string literal, expecting `do' or '{' or '(' (SyntaxError) 'abc' + pluralize 'def' ^ from lib/file.rb:42:in `throw_syntax_error' from foo.rb:7:in `<main>' ``` ### Actual behavior It reports It should SyntaxError and it emits the `warning: possibly useless use of + in void context` ``` $ ruby -v ruby 3.4.0dev (2024-10-25T11:52:32Z master 9c8c140d73) +PRISM [x86_64-linux] $ ruby -w foo.rb (eval at lib/file.rb:42):2: warning: possibly useless use of + in void context lib/file.rb:42:in 'Kernel#eval': (eval at lib/file.rb:42):2: syntax error found (SyntaxError) 1 | > 2 | 'abc' + pluralize 'def' | ^ unexpected string literal, expecting end-of-input 3 | from lib/file.rb:42:in 'C#throw_syntax_error' from foo.rb:7:in '<main>' $ ``` -- https://bugs.ruby-lang.org/

5 7

[ruby-core:120027] [Ruby master Misc#20913] Proposal: Adding Jeremy Evans and Burdette Lamar to www.ruby-lang.org's English Editorial Team
by st0012 (Stan Lo) 12 Dec '24

12 Dec '24

Issue #20913 has been reported by st0012 (Stan Lo). ---------------------------------------- Misc #20913: Proposal: Adding Jeremy Evans and Burdette Lamar to www.ruby-lang.org's English Editorial Team https://bugs.ruby-lang.org/issues/20913 * Author: st0012 (Stan Lo) * Status: Open ---------------------------------------- Hi everyone, I'd like to propose that we invite @jeremyevans0 and @burdettelamar to join the editorial team for [ruby-lang.org](https://ruby-lang.org)'s English content. Specifically, this means giving them permission to merge English content pull requests into the [ruby/www.ruby-lang.org GitHub repo](https://github.com/ruby/www.ruby-lang.org). Both Jeremy and Burdette are long-time Ruby committers who have made significant contributions to Ruby and the community. During RubyConf in Chicago this month, a few members from [Ruby Central](https://rubycentral.org/) and I discussed with them the idea of improving the website, and they both expressed enthusiasm for taking on this new role. Recently, we've been seeing increased contributions to the content of [ruby-lang.org/en](https://ruby-lang.org/en), and there are plenty of opportunities for further improvements as well. Adding Jeremy and Burdette to the team would help us enhance our reviewing capacity and keep the momentum going. What do you think? -- https://bugs.ruby-lang.org/

5 5

[ruby-core:119663] [Ruby master Bug#20858] multiple parallel assignments are inconsistent
by daveola (David Stellar) 12 Dec '24

12 Dec '24

Issue #20858 has been reported by daveola (David Stellar). ---------------------------------------- Bug #20858: multiple parallel assignments are inconsistent https://bugs.ruby-lang.org/issues/20858 * Author: daveola (David Stellar) * Status: Open * ruby -v: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I may have terminology wrong, so apologies. For this bug I'm going to use "multiple assignment" to refer to using multiple assignment operators in a line, such as: ``` ruby a = b = c = 1 ``` And then parallel assignment to refer to doing multiple assignments at the same time using tuples, such as: ``` ruby a,b = 1, 2 ``` Unfortunately combining these is inconsistent. First of all, just doing this: ``` ruby a,b = c,d = 3,4 ``` Gives us: **"undefined local variable or method `c' for main (NameError)"** So if we work around that by defining all our variables, we then get unexpected results: ``` ruby a = b = c = d = 'foobar' a,b = c,d = 3,4 puts "Got: a=#{a} b=#{b} and c=#{c} d=#{d}" # Got: a=foobar b=3 and c=foobar d=3 c,d = 3,4 a,b = c,d puts "Got: a=#{a} b=#{b} and c=#{c} d=#{d}" # Got: a=3 b=4 and c=3 d=4 ``` I can imagine that if multiple parallel assignment is not supported that a,b will not get set properly, but it does not follow that c would be undefined by the expression. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119988] [Ruby master Bug#20904] 3.4.0-preview2: Building miniruby.exe fails for mswin32
by jun66j5 (Jun Omae) 11 Dec '24

11 Dec '24

Issue #20904 has been reported by jun66j5 (Jun Omae). ---------------------------------------- Bug #20904: 3.4.0-preview2: Building miniruby.exe fails for mswin32 https://bugs.ruby-lang.org/issues/20904 * Author: jun66j5 (Jun Omae) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I tried to build 3.4.0-preview2 with MSVC x86, however linking miniruby.exe failed with the following eror: ``` linking miniruby.exe Creating library miniruby.lib and object miniruby.exp win32.obj : error LNK2019: unresolved external symbol _GetSystemTimePreciseAsFileTime referenced in function _clock_gettime miniruby.exe : fatal error LNK1120: 1 unresolved externals ``` It doesn't fail with MSVC x64. Investigating it, Windows 8 is required after #20563 but `NTVER` is still `0x0600` in `win32/Makefile.sub`. I think it should be `0x0602`. Workaround is to invoke `win32\configure.bat` with `--with-ntver=0x0602`. Also, adding temporarily `-w24013` to `WARNFLAGS` in `win32/Makefile.sub`, the following warning is received. ``` compiling win32/win32.c win32.c win32/win32.c(4789): warning C4013: 'GetSystemTimePreciseAsFileTime' undefined; assuming extern returning int ``` -- https://bugs.ruby-lang.org/

3 10

[ruby-core:119442] [Ruby master Feature#20782] Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new
by shioimm (Misaki Shioi) 10 Dec '24

10 Dec '24

Issue #20782 has been reported by shioimm (Misaki Shioi). ---------------------------------------- Feature #20782: Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new https://bugs.ruby-lang.org/issues/20782 * Author: shioimm (Misaki Shioi) * Status: Open ---------------------------------------- This is an implementation of Happy Eyeballs version 2 (RFC 8305) in `TCPSocket.new`. See https://github.com/ruby/ruby/pull/11653 ### Background Prior to this implementation, I implemented Happy Eyeballs Version 2 (HEv2) for `Socket.tcp` in https://github.com/ruby/ruby/pull/9374. HEv2 is an algorithm defined in [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305), aimed at improving network connectivity. For more details on the specific cases that HEv2 helps, please refer to https://bugs.ruby-lang.org/issues/20108. ### Proposal & Outcome This proposal implements the same HEv2 algorithm in `TCPSocket.new`. Since `TCPSocket.new` is used more widely than `Socket.tcp`, this change is expected to broaden the impact of HEv2's benefits. Like `Socket.tcp`, I have also added `fast_fallback` keyword argument to `TCPSocket.new`. This option is set to true by default, enabling the HEv2 functionality. However, users can explicitly set it to false to disable HEv2 and use the previous behavior of `TCPSocket.new`. It should be noted that HEv2 is enabled only in environments where pthreads are available. This specification follows the approach taken in https://bugs.ruby-lang.org/issues/19965, where name resolution can be interrupted. (In environments where pthreads are not available, the `fast_fallback` option is ignored.) ### Performance Below is the benchmark of 100 requests to `www.ruby-lang.org` with the fast_fallback option set to true and false, respectively. While there is a slight performance degradation when HEv2 is enabled, the degradation is smaller compared to that seen in `Socket.tcp` . ```ruby require 'socket' require 'benchmark' hostname = "www.ruby-lang.org" port = 80 n = 100 Benchmark.bmbm do |x| x.report("fast_fallback: true") do n.times { TCPSocket.new(hostname, port).close } end x.report("fast_fallback: false") do n.times { TCPSocket.new(hostname, port, fast_fallback: false).close } end end ``` ``` ~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb Rehearsal -------------------------------------------------------- fast_fallback: true 0.017588 0.097045 0.114633 ( 1.460664) fast_fallback: false 0.014033 0.078984 0.093017 ( 1.413951) ----------------------------------------------- total: 0.207650sec user system total real fast_fallback: true 0.020891 0.124054 0.144945 ( 1.473816) fast_fallback: false 0.018392 0.110852 0.129244 ( 1.466014) ``` -- https://bugs.ruby-lang.org/

6 11