November 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119881] [Ruby master Feature#20884] reserve "Ruby" toplevel module for Ruby language
by Dan0042 (Daniel DeLorme) 25 Dec '24

25 Dec '24

Issue #20884 has been reported by Dan0042 (Daniel DeLorme). ---------------------------------------- Feature #20884: reserve "Ruby" toplevel module for Ruby language https://bugs.ruby-lang.org/issues/20884 * Author: Dan0042 (Daniel DeLorme) * Status: Open ---------------------------------------- `Ruby` would be a convenient namespace for many features of the Ruby language, in particular APIs related to the interpreter. All these constants: RUBY_VERSION RUBY_RELEASE_DATE RUBY_PLATFORM RUBY_PATCHLEVEL RUBY_REVISION RUBY_COPYRIGHT RUBY_ENGINE RUBY_ENGINE_VERSION RUBY_DESCRIPTION would have made a lot of sense as `Ruby::Version` etc. `Thread::Backtrace::Location` would have made a lot of sense as `Ruby::Backtrace::Location` `RubyVM` is considered specific to CRuby; so `RubyVM::AbstractSyntaxTree` should be `Ruby::AbstractSyntaxTree` if it is meant to be present in other implementations. In #6648 there's a bit of contention over where `ruby_args` should be. `RubyVM`, `RbConfig`, `Process` have all been proposed, but `Ruby` would be an excellent choice. `Process.argv0` was added in Ruby 2.1 but the `Process` namespace is really about OS-level process control (fork, signals, euid, limits) while this argv0 is not (in `ps` it's neither value of COMMAND nor CMD) so it would have made sense as `Ruby.argv0` The "ruby" gem name is reserved, so there's no conflict. https://rubygems.org/gems/ruby All in all, "Ruby" is an appropriate namespace for many Ruby things. We don't want to break compatibility over this, but we could at least start small by reserving the namespace, and see how it grows from there. module Ruby Version = ::RUBY_VERSION end -- https://bugs.ruby-lang.org/

7 7

[ruby-core:119413] [Ruby master Bug#20779] Un-rescuable thread creation limit on Mac
by hms (Hal Spitz) 23 Dec '24

23 Dec '24

Issue #20779 has been reported by hms (Hal Spitz). ---------------------------------------- Bug #20779: Un-rescuable thread creation limit on Mac https://bugs.ruby-lang.org/issues/20779 * Author: hms (Hal Spitz) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Thread.new will hang the ruby process once past the OSX limit. It does not appear that's possible to trap and rescue failed thread creation. It seems the Mac has a limit to the number of threads it supports and googling the issue suggests the limit has been known to change with hardware and OS releases. While I hit the thread creation limit due to a bug in my code and the number of threads that I was attempting to create (12,250+) is an unlikely use-case, I am hoping it would be possible to turn this into a rescuable error. While I'm reporting this on 3.3.5, I have tested the issue and the inability to rescue back to 3.3.1. (Thanks to @raels for being a great second set of eyes to make sure I wasn't doing anything goofy / wrong before making this report. Also, another callout and thanks to both @eregon and @bensheldon over at concurrent-ruby for the unbelievably quick and helpful response to my initial bug report over there). ---Files-------------------------------- hang.rb (713 Bytes) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117812] [Ruby master Bug#20478] Circular parameter syntax error rules
by kddnewton (Kevin Newton) 20 Dec '24

20 Dec '24

Issue #20478 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20478: Circular parameter syntax error rules https://bugs.ruby-lang.org/issues/20478 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I would like to revisit https://bugs.ruby-lang.org/issues/16343. These cases are syntax errors: ```ruby def foo(bar = -> { bar }) end # no lambda parameters def foo(bar = ->() { bar }) end # no lambda parameters def foo(bar = baz { bar }) end # no block parameters def foo(bar = baz { _1 + bar }) end # parameters, but no pipes def foo(bar = baz { it + bar }) end # parameters, but no pipes ``` These cases are not syntax errors: ```ruby def foo(bar = ->(baz) { bar }) end # lambda parameters def foo(bar = baz { || bar }) end # no block parameters but empty pipes def foo(bar = baz { |qux| bar }) end # block parameters ``` I don't think these rules are very intuitive, and they feel somewhat arbitrary. I would like to suggest we change them to be either: * Syntax error is raised if the parameter is ever read in its default value at any scope depth * Syntax error is raised if the parameter is ever read in its default value at depth 0 Either one is fine by me, but gating the syntax error based on the presence of pipes is really confusing. -- https://bugs.ruby-lang.org/

4 10

[ruby-core:119883] [Ruby master Feature#20885] String#gsub?
by Dan0042 (Daniel DeLorme) 18 Dec '24

18 Dec '24

Issue #20885 has been reported by Dan0042 (Daniel DeLorme). ---------------------------------------- Feature #20885: String#gsub? https://bugs.ruby-lang.org/issues/20885 * Author: Dan0042 (Daniel DeLorme) * Status: Open ---------------------------------------- I would like a variation of sub/gsub that returns a new string if there is a match, or nil otherwise. This can be currently done with `str.dup.gsub!(...)` at the mere cost of 2 unnecessary extra allocations due to String#dup TBH I'm not sure we need yet another gsub method, but what gets me here is that `gsub?` is the primitive operation from which `gsub` and `gsub!` are made. Internally, these two methods work like this: ```ruby def gsub!(...) modified = gsub?(...) and replace(modified) end def gsub(...) gsub?(...) or dup end ``` We can efficiently derive these two methods from `gsub?`, but we cannot efficiently implement `gsub?` in ruby, so it feels regrettable to me that `gsub?` is not available as a built-in. Use cases include: ```ruby #ensure gsub has modified the string newstr = str.gsub?(rx,repl) or raise "str did not contain expected value" #take an action if the string is modified if str = obj.title.gsub?(rx,repl) obj.title = str obj.save end ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119925] [Ruby master Feature#20893] random/formatter.rb: Separate from securerandom
by nobu (Nobuyoshi Nakada) 16 Dec '24

16 Dec '24

Issue #20893 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #20893: random/formatter.rb: Separate from securerandom https://bugs.ruby-lang.org/issues/20893 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- For historical reasons, random/formatter.rb is included in ruby/securerandom, but it does not contain any security-related methods as evidenced by the fact that its path is not under `securerandom/`, but only contains generic methods for PRNGs, including the standard class `Random`. So I propose to either: 1. put it back in the standard library, or 2. separate it into a new gem. Adding a new dependency may not be welcomed, as seen in the recent issues with base64. -- https://bugs.ruby-lang.org/

2 3

[ruby-core:117657] [Ruby master Bug#20447] Ruby 3.3.1 broken on i686
by vo.x (Vit Ondruch) 15 Dec '24

15 Dec '24

Issue #20447 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20447: Ruby 3.3.1 broken on i686 https://bugs.ruby-lang.org/issues/20447 * Author: vo.x (Vit Ondruch) * Status: Open * ruby -v: ruby 3.3.1 (2024-04-23 revision c56cd86388) [i686-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ~~~ gcc -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -fPIC -m32 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/i386-linux -I/builddir/build/BUILD/ruby-3.3.1/include -I/builddir/build/BUILD/ruby-3.3.1 -I/builddir/build/BUILD/ruby-3.3.1/prism -I/builddir/build/BUILD/ruby-3.3.1/enc/unicode/15.0.0 -o time.o -c /builddir/build/BUILD/ruby-3.3.1/time.c make: Leaving directory '/builddir/build/BUILD/ruby-3.3.1/redhat-linux-build' /builddir/build/BUILD/ruby-3.3.1/time.c: In function ‘zone_timelocal’: /builddir/build/BUILD/ruby-3.3.1/time.c:2345:35: error: passing argument 3 of ‘split_second’ from incompatible pointer type [-Wincompatible-pointer-types] 2345 | split_second(tobj->timew, &t, &s); | ^~ | | | wideval_t * {aka long long unsigned int *} /builddir/build/BUILD/ruby-3.3.1/time.c:616:58: note: expected ‘VALUE *’ {aka ‘long unsigned int *’} but argument is of type ‘wideval_t *’ {aka ‘long long unsigned int *’} 616 | split_second(wideval_t timew, wideval_t *timew_p, VALUE *subsecx_p) | ~~~~~~~^~~~~~~~~ make: *** [Makefile:447: time.o] Error 1 make: *** Waiting for unfinished jobs.... make: Entering directory '/builddir/build/BUILD/ruby-3.3.1/redhat-linux-build' ~~~ This is presumably due to #20327 / https://github.com/ruby/ruby/pull/10332. There seems to be medicine in master commit:git|055613fd868a8c94e43893f8c58a00cdd2a81f6d or maybe the whole https://github.com/ruby/ruby/pull/10332, not sure -- https://bugs.ruby-lang.org/

8 14

[ruby-core:119852] [Ruby master Feature#20882] Provide Boolean(...)
by getajobmike (Mike Perham) 14 Dec '24

14 Dec '24

Issue #20882 has been reported by getajobmike (Mike Perham). ---------------------------------------- Feature #20882: Provide Boolean(...) https://bugs.ruby-lang.org/issues/20882 * Author: getajobmike (Mike Perham) * Status: Open ---------------------------------------- Ruby provides Integer(...) and Float(...) global methods to coerce values. Is there a similar method for Booleans? I'd like to do something like: ``` # ENV["SOME_FEATURE"] is unset Boolean(ENV["SOME_FEATURE"]) # => false # ENV["SOME_FEATURE"] is unset, but allow a default? Boolean(ENV["SOME_FEATURE"], true) # => true # explicitly disable ENV["SOME_FEATURE"] = "0" Boolean(ENV["SOME_FEATURE"], true) # => false # explicitly enable ENV["SOME_FEATURE"] = "1" Boolean(ENV["SOME_FEATURE"]) # => true ``` -- https://bugs.ruby-lang.org/

11 11

[ruby-core:119769] [Ruby master Feature#20875] Atomic initialization for Ractor local storage
by ko1 (Koichi Sasada) 12 Dec '24

12 Dec '24

Issue #20875 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #20875: Atomic initialization for Ractor local storage https://bugs.ruby-lang.org/issues/20875 * Author: ko1 (Koichi Sasada) * Status: Open ---------------------------------------- ## Motivation Now there is no way to initialize Ractor local storage in multi-thread. For example, if we want to introduce per-Ractor counter, which should be protected with a per-Ractor Mutex for multi-threading support. ```ruby def init Ractor[:cnt] = 0 Ractor[:mtx] = Mutex.new end def inc init unless Ractor[:cnt] Ractor[:mtx].synchronize do Ractor[:cnt] += 1 end end ``` In this code, if `inc` was called on multiple threads, `init` can be called with multiple threads and `cnt` can not be synchronized correctly. ## Proposal Let's introduce `Ractor.local_storage_init(sym){ block }` to initialize values in Ractor local storage. If there is no slot for `sym`, synchronize with per-Ractor mutex and call `block` and the slot will be filled with the evaluation with the block result. The return value of this method will be the filled value. Otherwise, returning corresponding value will be returned. The implementation is like that (in C): ```ruby class Ractor def self.local_storage_init(sym) Ractor.per_ractor_mutex.synchronize do if Ractor.local_storage_has_key?(sym) Ractor[:sym] else Ractor[:sym] = yield end end end end ``` The above examples will be rewritten with the following code: ```ruby def inc Ractor.local_storage_init(:mtx) do Ractor[:cnt] = 0 Mutex.new end.synchronize do Ractor[:cnt] += 1 end end ``` ## Discussion ### Approach There are another approach like `pthread_atfork`, maybe like `Ractor.atcreate{ init }`. A library registers a callback which will be called when a new ractor is created. However, there are many Ractors which don't use the library, so that ``atcreate` can be huge overhead for Ractor creation. ### Naming I propose `local_storage_init`, but not sure it matches. I also proposed `Ractor.local_variable_init(sym)`, but Matz said he doesn't like this naming because it should not be a "variable". (there is a `Thread#thread_variable_get` method, though). On another aspect, `lcoal_storage_init` seems it clears all of ractor local storage slots. -- https://bugs.ruby-lang.org/

3 11

[ruby-core:120035] [Ruby master Bug#20918] Prism error indicates line number of `-e` that does not exist
by mame (Yusuke Endoh) 12 Dec '24

12 Dec '24

Issue #20918 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20918: Prism error indicates line number of `-e` that does not exist https://bugs.ruby-lang.org/issues/20918 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: prism * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` $ ruby -e 'foo(' -e: -e:2: syntax error found (SyntaxError) 1 | foo( > 2 | | ^ unexpected end-of-input; expected a `)` to close the arguments ``` It says `-e:2`, but there is no line 2 in `-e 'foo('`. `eval("foo(")` reports line 1, so I guess this issue is only for `-e`. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:118750] [Ruby master Bug#20659] Speed regression of `parse.y` parser after numeric nodes were introduced
by alanwu (Alan Wu) 12 Dec '24

12 Dec '24

Issue #20659 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20659: Speed regression of `parse.y` parser after numeric nodes were introduced https://bugs.ruby-lang.org/issues/20659 * Author: alanwu (Alan Wu) * Status: Open * ruby -v: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [`mail` benchmark from yjit-bench][1] is about 20% slower on `master` compared to commit:98eeadc9 ("Development of 3.4.0 started.") as the baseline, comparing running time of the Ruby process running the benchmark for a single iteration. Much of this workload is Ruby parsing. ``` $ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby' '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb' Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 889.0 ms ± 2.0 ms [User: 776.9 ms, System: 111.7 ms] Range (min … max): 885.7 ms … 892.1 ms 10 runs Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 891.7 ms ± 1.6 ms [User: 794.5 ms, System: 96.9 ms] Range (min … max): 889.0 ms … 894.7 ms 10 runs Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 1.086 s ± 0.003 s [User: 0.951 s, System: 0.134 s] Range (min … max): 1.080 s … 1.091 s 10 runs Summary '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' ran 1.00 ± 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.22 ± 0.00 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' $ for tag in 3.3-equiv master before-numeric-nodes; do echo $tag: $(~/.rubies/$tag/bin/ruby -v); done 3.3-equiv: ruby 3.4.0dev (2023-12-25T09:13:40Z master 98eeadc932) [x86_64-linux] master: ruby 3.4.0dev (2024-07-30T14:01:43Z master 1164b6a7ba) [x86_64-linux] before-numeric-nodes: ruby 3.4.0dev (2024-01-06T18:26:38Z master 76afbda5b5) [x86_64-linux] ``` Using Valgrind's DHAT reveals that commit:1b8d01136c3ff6c60325c7609d61e19ac42acd9f ("Introduce Numeric Node's") issues roughly 3 times more `malloc` calls compared to the baseline, most of them comping from strdup() calls in set_number_literal(). Comparison with `--parser=prism`, for interest: <details> ``` $ hyperfine -L ruby '3.3-equiv/bin/ruby,before-numeric-nodes/bin/ruby,master/bin/ruby,master/bin/ruby --parser=prism' \ '~/.rubies/{ruby} -Iharness-once benchmarks/mail/benchmark.rb' Benchmark 1: ~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 889.7 ms ± 2.6 ms [User: 771.4 ms, System: 118.0 ms] Range (min … max): 885.8 ms … 894.6 ms 10 runs Benchmark 2: ~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 890.4 ms ± 1.6 ms [User: 776.8 ms, System: 113.3 ms] Range (min … max): 888.4 ms … 892.9 ms 10 runs Benchmark 3: ~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 1.087 s ± 0.004 s [User: 0.968 s, System: 0.119 s] Range (min … max): 1.083 s … 1.097 s 10 runs Benchmark 4: ~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb Time (mean ± σ): 826.9 ms ± 2.1 ms [User: 725.8 ms, System: 100.7 ms] Range (min … max): 823.6 ms … 830.9 ms 10 runs Summary '~/.rubies/master/bin/ruby --parser=prism -Iharness-once benchmarks/mail/benchmark.rb' ran 1.08 ± 0.00 times faster than '~/.rubies/3.3-equiv/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.08 ± 0.00 times faster than '~/.rubies/before-numeric-nodes/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' 1.31 ± 0.01 times faster than '~/.rubies/master/bin/ruby -Iharness-once benchmarks/mail/benchmark.rb' ``` </details> [1]: https://github.com/Shopify/yjit-bench/blob/b8f1bd8931d4757afdae933ef6f11e07… -- https://bugs.ruby-lang.org/

2 4