March 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:121238] [Ruby master Misc#19122] Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack
by ioquatix (Samuel Williams) 04 Mar '25

04 Mar '25

Issue #19122 has been updated by ioquatix (Samuel Williams). On the latest Linux kernel, the constants have changed: ``` MADV_DONTNEED = 4 MADV_FREE = 8 ``` I did a quick comparison, creating 10,000 fibers x10 times in a loop: ``` samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=8 bundle exec test.rb >/dev/null <main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes. ________________________________________________________ Executed in 841.07 millis fish external usr time 373.88 millis 441.00 micros 373.44 millis sys time 463.22 millis 168.00 micros 463.05 millis samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=16 bundle exec test.rb >/dev/null <main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes. ________________________________________________________ Executed in 582.77 millis fish external usr time 357.66 millis 420.00 micros 357.24 millis sys time 220.90 millis 164.00 micros 220.74 millis samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=0 bundle exec test.rb >/dev/null ________________________________________________________ Executed in 444.61 millis fish external usr time 312.92 millis 503.00 micros 312.41 millis sys time 128.99 millis 0.00 micros 128.99 millis ``` Also memory usage as we expected: ``` samuel@aiko ~/D/s/memory-leak (main)> RUBY_SHARED_FIBER_POOL_FREE_STACKS=16 bundle exec test.rb <main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes. PID: 70235 Memory usage: 150.82 MB Clearing fibers and garbage collecting... Memory usage: 150.82 MB samuel@aiko ~/D/s/memory-leak (main)> RUBY_SHARED_FIBER_POOL_FREE_STACKS=8 bundle exec test.rb <main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes. PID: 70279 Memory usage: 151.63 MB Clearing fibers and garbage collecting... Memory usage: 73.63 MB ``` Using the following test script: ``` #!/usr/bin/env ruby require "memory/leak/system" def print_memory_usage size = Memory::Leak::System.memory_usage(Process.pid) units = %w(B KB MB GB TB) unit = 0 while size > 1024 && unit < units.size size /= 1024.0 unit += 1 end puts "Memory usage: %.2f %s" % [size, units[unit]] end puts "PID: #{Process.pid}" 10.times do |i| puts "Iteration: #{i}" # Create 10,000 fibers: fibers = 10_000.times.map do Fiber.new do Fiber.yield Fiber.current end.resume end # Print memory usage: print_memory_usage puts "Clearing fibers and garbage collecting..." # Clear fibers: fibers.clear GC.start # Print memory usage: print_memory_usage end ``` ---------------------------------------- Misc #19122: Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack https://bugs.ruby-lang.org/issues/19122#change-112186 * Author: smcgivern (Sean McGivern) * Status: Assigned * Assignee: ioquatix (Samuel Williams) ---------------------------------------- I'd like to propose that Ruby stops using MADV_FREE when freeing a Fiber's stack, and switches to using MADV_DONTNEED even when MADV_FREE is supported. MADV_FREE is used in one place in the Ruby codebase, when freeing the stack of a freed Fiber: https://git.ruby-lang.org/ruby.git/tree/cont.c#n683 The comment for `fiber_pool_stack_free` says: ```c // We advise the operating system that the stack memory pages are no longer being used. // This introduce some performance overhead but allows system to relaim memory when there is pressure. ``` Where possible (i.e. on Linux 4.5 and later), `fiber_pool_stack_free` uses `MADV_FREE` over `MADV_DONTNEED`. This has the side effect that memory statistics such as RSS will not reduce until and unless the OS actually reclaims that memory. If that doesn't happen, then the reported memory usage via RSS will be much higher than the 'real' memory usage. If this was pervasive throughtout the Ruby codebase then that would be one thing, but currently this is just for Fiber. This means that: 1. A program that doesn't use Fiber will have somewhat reliable RSS statistics on recent Linux. 2. A program that heavily uses Fiber (such as something using Async::HTTP) will see an inflated RSS statistic. Go made a similar change to the one I'm proposing here for similar reasons: https://github.com/golang/go/issues/42330 > While `MADV_FREE` is somewhat faster than `MADV_DONTNEED`, it doesn't affect many of the statistics that `MADV_DONTNEED` does until the memory is actually reclaimed. This generally leads to poor user experience, like confusing stats in `top` and other monitoring tools; and bad integration with management systems that respond to memory usage. > [...] > I propose we change the default to prefer `MADV_DONTNEED` over `MADV_FREE`, to favor user-friendliness and minimal surprise over performance. I think it's become clear that Linux's implementation of `MADV_FREE` ultimately doesn't meet our needs. As an aside, MADV_FREE was not used in Ruby 3.1 (https://bugs.ruby-lang.org/issues/19101) and I haven't found any bugs filed about this behaviour other than that one. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:121193] [Ruby master Bug#21161] Crash when locale is set to Turkish tr_TR.UTF-8
by srbaker (Steven Baker) 04 Mar '25

04 Mar '25

Issue #21161 has been reported by srbaker (Steven Baker). ---------------------------------------- Bug #21161: Crash when locale is set to Turkish tr_TR.UTF-8 https://bugs.ruby-lang.org/issues/21161 * Author: srbaker (Steven Baker) * Status: Open * ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- TL;DR this bug was reported in our tracker, and I'm pushing it upstream: https://bugzilla.opensuse.org/show_bug.cgi?id=1237861 When the locale is set to `tr_TR.UTF-8`, there is an encoding error. It has been narrowed down specifically to setting `LC_CTYPE`. To reproduce simply run `LC_CTYPE=tr_TR.UTF-8 ruby -e "puts 42"` Example from a fresh 3.4.2 install: ``` shell srbaker@geekopad:~> LC_CTYPE=tr_TR.UTF-8 ruby -e "puts 42" /home/srbaker/.local/share/mise/installs/ruby/3.4.2/lib64/ruby/3.4.0/rubygems.rb:9:in 'Kernel#require': /home/srbaker/.local/share/mise/installs/ruby/3.4.2/lib64/ruby/3.4.0/x86_64-linux/rbconfig.rb:1: unknown or invalid encoding in the magic comment (ArgumentError) > 1 | # encoding: ascii-8bit | ^~~~~~~~~~ 2 | # frozen-string-literal: false 3 | # from /home/srbaker/.local/share/mise/installs/ruby/3.4.2/lib64/ruby/3.4.0/rubygems.rb:9:in '<top (required)>' from <internal:gem_prelude>:2:in 'Kernel#require' from <internal:gem_prelude>:2:in '<internal:gem_prelude>' ``` This reproduces across multiple installs of ruby: from our packages, locally built on both GNU/Linux and macOS. It looks like it's related to some normalisation on lowercase i, which in Turkish appears to produce a lowercase i without a dot, and the string. Details in our bug linked above. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:121141] [Ruby master Bug#21153] ::Foo ||= p 1 should parse
by qnighy (Masaki Hara) 03 Mar '25

03 Mar '25

Issue #21153 has been reported by qnighy (Masaki Hara). ---------------------------------------- Bug #21153: ::Foo ||= p 1 should parse https://bugs.ruby-lang.org/issues/21153 * Author: qnighy (Masaki Hara) * Status: Open * ruby -v: ruby 3.5.0dev (2025-02-22T15:11:40Z master fd882fb681) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- There is an inconsistency between how `Foo` and `::Foo` are handled: ```console % ./miniruby --parser=parse.y -e "Foo ||= p 1" 1 % ./miniruby --parser=parse.y -e "::Foo ||= p 1" -e:1: syntax error, unexpected integer literal, expecting 'do' or '{' or '(' ::Foo ||= p 1 ./miniruby: compile error (SyntaxError) ``` Note that `Object::Foo` also parses. Prism successfully parses both, which is what I expect. ```console % ./miniruby --parser=prism -e "Foo ||= p 1" 1 % ./miniruby --parser=prism -e "::Foo ||= p 1" 1 ``` To fix this, you should probably handle all tOP_ASGNs in the op_asgn rule. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:121090] [Ruby master Bug#21145] Prism accepts newlines in-between curly unicode escape
by kddnewton (Kevin Newton) 03 Mar '25

03 Mar '25

Issue #21145 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #21145: Prism accepts newlines in-between curly unicode escape https://bugs.ruby-lang.org/issues/21145 * Author: kddnewton (Kevin Newton) * Status: Open * Assignee: prism * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED ---------------------------------------- For example: ```ruby p "\u{ 61}" ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:121217] [Ruby master Feature#19059] Introduce top level `module TimeoutError` for aggregating various timeout error classes.
by ioquatix (Samuel Williams) 02 Mar '25

02 Mar '25

Issue #19059 has been updated by ioquatix (Samuel Williams). Status changed from Assigned to Closed It is unlikely that Ruby will adopt this, so I am going to close it. ---------------------------------------- Feature #19059: Introduce top level `module TimeoutError` for aggregating various timeout error classes. https://bugs.ruby-lang.org/issues/19059#change-112158 * Author: ioquatix (Samuel Williams) * Status: Closed * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- This proposal was originally part of <https://bugs.ruby-lang.org/issues/18630> but was removed because we could not decide on the name. Introduce the following: ```ruby module TimeoutError end IO::TimeoutError.include(TimeoutError) Regexp::TimeoutError.include(TimeoutError) # Maybe? Timeout::Error.include(TimeoutError) ``` It may be easier for users. This was discussed before with the following conclusion: - Top level `TimeoutError` is available. - Using a module for a `TimeoutError` may not be consistent with other top level `class #{thing}Error`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:121207] [Ruby master Bug#21164] Performance Regression using --jit
by purbug28 (puni ru) 01 Mar '25

01 Mar '25

Issue #21164 has been reported by purbug28 (puni ru). ---------------------------------------- Bug #21164: Performance Regression using --jit https://bugs.ruby-lang.org/issues/21164 * Author: purbug28 (puni ru) * Status: Open * ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster. ```ruby # frozen_string_literal: true n = 200000 c = Array.new(n + 1, 0) (1..n).each do |i| a = [] m = 100 (1..m).each do a << i c[i] += 1 a << i / m c[i % m] += 1 end end puts c.sum ``` results of /usr/bin/time ruby 3.4.2 --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k ruby 3.3.7 --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k `ruby -v --jit` ``` ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux] ``` -- https://bugs.ruby-lang.org/

2 5