July 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118682] [Ruby master Misc#20652] Memory allocation for gsub has increased from Ruby 2.7 to 3.3
by orisano (Nao Yonashiro) 24 Feb '25

24 Feb '25

Issue #20652 has been reported by orisano (Nao Yonashiro). ---------------------------------------- Misc #20652: Memory allocation for gsub has increased from Ruby 2.7 to 3.3 https://bugs.ruby-lang.org/issues/20652 * Author: orisano (Nao Yonashiro) * Status: Open ---------------------------------------- I recently upgraded from ruby 2.7.7 to 3.3.1 and noticed that the GC load increased. When I used the allocation profiler to investigate, I found that memory allocation from gsub had increased. The problem was code like this: ```ruby s = "foo " s.gsub(/ (\s+)/) { " #{' ' * Regexp.last_match(1).length}" } ``` When I compared the results of heap-profiler between 2.7.7 and 3.3.1, I found that MatchData was increasing. https://gist.github.com/orisano/98792dee260106e9b6fcb45bbabeb1e6 https://github.com/ruby/ruby/commit/abc0304cb28cb9dcc3476993bc487884c139fd11 I discovered that the cause is this commit, which stopped reusing backref to avoid race conditions. Is there a way to reuse backref while still avoiding race conditions? -- https://bugs.ruby-lang.org/

9 27

[ruby-core:113033] [Ruby master Feature#19555] Allow passing default options to `Data.define`
by p8 (Petrik de Heus) 13 Feb '25

13 Feb '25

Issue #19555 has been reported by p8 (Petrik de Heus). ---------------------------------------- Feature #19555: Allow passing default options to `Data.define` https://bugs.ruby-lang.org/issues/19555 * Author: p8 (Petrik de Heus) * Status: Open * Priority: Normal ---------------------------------------- Defining a subclass of `Data` with default attributes can currently be implemented by overriding `intialize`: ```ruby class Point < Data.define(:x, :y, :z) def initialize(x:, y:, z: 0) = super end p Point.new(1, 2) #=> #<data Point x=1, y=2, z=0> ``` It would be nice if we could do it in `define` as well: ```ruby Point = Data.define(:x, :y, z: 0) ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:116983] [Ruby master Feature#20309] Bundled gems for Ruby 3.5
by hsbt (Hiroshi SHIBATA) 06 Feb '25

06 Feb '25

Issue #20309 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Feature #20309: Bundled gems for Ruby 3.5 https://bugs.ruby-lang.org/issues/20309 * Author: hsbt (Hiroshi SHIBATA) * Status: Assigned * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- I propose migrate the following default gems to bundled gems at Ruby 3.5. So, It means users will get warnings if users try to load them. * ostruct * irb * reline * readline (wrapper file for readline-ext and reline) * io-console * logger * fiddle * pstore * open-uri * yaml (wrapper file for psych) * win32ole I have a plan to migrate the following default gems too. But I need to more feedback from other committers about them. * rdoc * We need to change build task like download rdoc gem before document generation. * or We make document generation is optional from Ruby 3.5 * We explicitly separate `make install` and `make install-doc` * un * `ruby -run` is one of cool feature of Ruby. Should we avoid uninstalling `un` gem? * singleton * This is famous design pattern. Should we enforce users add them to their Gemfile? * forwadable * `reline` needs to add forwardable their `runtime_dependency` after migration. * weakref * I'm not sure how impact after migrating bundled gems. * fcntl * Should we integrate these constants into ruby core? I would like to migrate `ipaddr` and `uri` too. But these are used by webrick that is mock server for our test suite. We need to rewrite `webrick` with `TCPSocker` or extract `ipaddr` and `uri` dependency from `webrick` Other default gems depend on our build process or other libraries deeply. I will update this proposal if I could extract them from default gems. -- https://bugs.ruby-lang.org/

9 27

[ruby-core:117765] [Ruby master Feature#20470] Extract Ruby's Garbage Collector
by peterzhu2118 (Peter Zhu) 29 Jan '25

29 Jan '25

Issue #20470 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Feature #20470: Extract Ruby's Garbage Collector https://bugs.ruby-lang.org/issues/20470 * Author: peterzhu2118 (Peter Zhu) * Status: Open ---------------------------------------- # Extract Ruby's Garbage Collector ## Background As described in [[Feature #20351]](https://bugs.ruby-lang.org/issues/20351), we are working on the ability to plug alternative garbage collector implementations into Ruby. Our goal is to allow developers and researchers to create and experiment with new implementations of garbage collectors in Ruby in a simplified way. This will also allow experimentation with different GC implementations in production systems so users can choose the best GC implementation for their workloads. ## Implementation GitHub PR: [10721](https://github.com/ruby/ruby/pull/10721) In this patch, we have split the current `gc.c` file into two files: `gc.c` and `gc_impl.c`. `gc.c` now only contains code not specific to Ruby GC. This includes code to mark objects (which the GC implementation may choose not to use) and wrappers for internal APIs that the implementation may need to use (e.g. locking the VM). `gc_impl.c` now contains the implementation of Ruby's GC. This includes marking, sweeping, compaction, and statistics. Most importantly, `gc_impl.c` only uses public APIs in Ruby and a limited set of functions exposed in `gc.c`. This allows us to build `gc_impl.c` independently of Ruby and plug Ruby's GC into itself. ## Demonstration After [checking out the branch](https://github.com/ruby/ruby/pull/10721), we can first configure with `--with-shared-gc`: ```bash $ ./configure --with-shared-gc ... $ make -j ... ``` Let's now change the slot size of the GC to 64 bytes: ```bash $ sed -i 's/$#define BASE_SLOT_SIZE$.*/\1 64/' gc_impl.c ``` We can compile `gc_impl.c` independently using the following commands for clang or gcc (you may have to change the last `-I` to match your architecture and platform): ```bash $ clang -Iinclude -I. -I.ext/include/arm64-darwin23 -undefined dynamic_lookup -g -O3 -dynamiclib -o libgc.dylib gc_impl.c $ gcc -Iinclude -I. -I.ext/include/x86_64-linux -Wl,-undefined,dynamic_lookup -fPIC -g -O3 -shared -o libgc.so gc_impl.c ``` We can see that by default, the slot size is 40 bytes and objects are 40 bytes in size: ```bash $ ./ruby -e "puts GC.stat_heap(0, :slot_size)" 40 $ ./ruby -robjspace -e "puts ObjectSpace.dump(Object.new)" {"address":"0x1054a23f0", "type":"OBJECT", "shape_id":3, "slot_size":40, "class":"0x10528fd38", "embedded":true, "ivars":0, "memsize":40, "flags":{"wb_protected":true}} ``` We can now load our new GC using the `RUBY_GC_LIBRARY_PATH` environment variable (note that you may have to change the path to the DSO): ```bash $ RUBY_GC_LIBRARY_PATH=./libgc.dylib ./ruby -e "puts GC.stat_heap(0, :slot_size)" 64 $ RUBY_GC_LIBRARY_PATH=./libgc.dylib ./ruby -robjspace -e "puts ObjectSpace.dump(Object.new)" {"address":"0x1038de440", "type":"OBJECT", "shape_id":3, "slot_size":64, "class":"0x10355fc00", "embedded":true, "ivars":0, "memsize":64, "flags":{"wb_protected":true}} ``` ## Benchmark Benchmarks were ran on commit [c78cebb](https://github.com/ruby/ruby/commit/c78cebb469fe56b45ee5daad16ae97… on Ubuntu 22.04 using [yjit-bench](https://github.com/Shopify/yjit-bench/) on commit [cc5a76e](https://github.com/Shopify/yjit-bench/commit/cc5a76ef6240113650547…. Compiling gc_impl branch without `--with-shared-gc` (i.e. how the default Ruby is built), the benchmarks show little to no decrease in performance, with most of it being 0% to 1% slower: ``` -------------- ----------- ---------- ------------ ---------- --------------- -------------- bench master (ms) stddev (%) gc_impl (ms) stddev (%) gc_impl 1st itr master/gc_impl activerecord 73.9 0.3 74.6 0.3 1.00 0.99 chunky-png 911.1 0.2 937.2 0.2 0.97 0.97 erubi-rails 1582.4 0.1 1583.5 0.0 1.00 1.00 hexapdf 2716.2 1.1 2760.2 0.7 1.00 0.98 liquid-c 68.9 0.5 68.6 0.4 1.00 1.00 liquid-compile 67.9 0.1 68.2 0.2 0.99 1.00 liquid-render 172.8 0.1 174.9 0.1 0.99 0.99 lobsters 1033.9 0.4 1036.0 0.3 1.08 1.00 mail 135.1 0.2 136.5 0.2 0.99 0.99 psych-load 2250.8 0.1 2274.9 0.3 0.99 0.99 railsbench 2499.2 0.2 2502.9 0.1 1.00 1.00 rubocop 178.3 0.5 179.8 0.4 1.00 0.99 ruby-lsp 116.8 0.1 118.5 0.2 1.00 0.99 sequel 75.4 0.2 76.2 0.3 0.99 0.99 -------------- ----------- ---------- ------------ ---------- --------------- -------------- ``` Compiling gc_impl branch with `--with-shared-gc` and loading Ruby's current GC using `RUBY_GC_LIBRARY_PATH`, the benchmarks are still fairly good with performance decrease of only around 1% to 2%: ``` -------------- ----------- ---------- ------------ ---------- --------------- -------------- bench master (ms) stddev (%) gc_impl (ms) stddev (%) gc_impl 1st itr master/gc_impl activerecord 74.2 0.2 75.4 0.5 0.98 0.98 chunky-png 916.3 0.3 933.2 0.1 0.98 0.98 erubi-rails 1597.6 0.1 1586.3 0.2 1.01 1.01 hexapdf 2731.4 0.5 2776.8 0.7 1.00 0.98 liquid-c 68.5 0.1 68.9 0.4 0.97 0.99 liquid-compile 67.4 0.4 68.3 0.2 0.95 0.99 liquid-render 171.8 0.1 175.6 0.2 0.97 0.98 lobsters 1031.9 0.3 1041.4 0.3 0.94 0.99 mail 135.5 0.4 136.7 0.1 0.99 0.99 psych-load 2246.0 0.1 2281.3 0.1 0.99 0.98 railsbench 2490.9 0.0 2490.0 0.1 1.01 1.00 rubocop 179.8 2.3 180.0 0.4 0.94 1.00 ruby-lsp 117.3 0.1 118.5 0.1 0.99 0.99 sequel 75.8 0.5 76.3 0.2 0.99 0.99 -------------- ----------- ---------- ------------ ---------- --------------- -------------- ``` ## Limitations We recognize that our current implementation does not yet offer the flexibility required for a generic plug-in GC. Specifically, the set of APIs that the plug-in GC has to implement is relatively large, at around 70 functions. Additionally, some of these functions are specific to the current GC. We would like to emphasize that the API is NOT stable and is subject to change. We will be working on improving this API and reducing the surface area. This will be future work and we're not working on it in this phase. ## Future plans - Refactor and improve `gc_impl.c`. - Implement alternate GC implementations, such as the Epsilon GC and [MMTk](https://www.mmtk.io/) to prove that this API allows for alternate implementations of the GC. - Reduce and improve the API of the GC implementation. - Benchmark and improve performance of the DSO API. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:115117] [Ruby master Bug#19967] Already installed libruby.dylib is used for test on macOS
by nobu (Nobuyoshi Nakada) 24 Jan '25

24 Jan '25

Issue #19967 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19967: Already installed libruby.dylib is used for test on macOS https://bugs.ruby-lang.org/issues/19967 * Author: nobu (Nobuyoshi Nakada) * Status: Assigned * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- On macOS, in the case of `--enable-shared` and `--disable-load-relative`, already install libruby.dylib is used during tests if exists. This is because since [Bug #14992], `DYLD_FALLBACK_LIBRARY_PATH` is used instead of `DYLD_LIBRARY_PATH`. The latter environment variable is used preferentially, whereas the former is used as a fallback, as the name implies. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116640] [Ruby master Feature#20249] Print only backtraces in rb_bug(), by default
by osyoyu (Daisuke Aritomo) 23 Jan '25

23 Jan '25

Issue #20249 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Feature #20249: Print only backtraces in rb_bug(), by default https://bugs.ruby-lang.org/issues/20249 * Author: osyoyu (Daisuke Aritomo) * Status: Open * Priority: Normal ---------------------------------------- ## Background When a segfault or some unexpected situation occurs, `rb_bug()` is called and prints some few hundred to thousands of lines. The most helpful parts are (arguably) "Ruby level backtrace information" and "C-level backtrace information", but those parts are buried in the very lengthy report. In particular, the "Other runtime information" which contains the list of loaded features (scripts?) and the process memory map could be extremely long despite it does not come very useful, at least when developing C extensions. Even a minimal report from a simple script would consist of 250 lines and require 7 PgUps on my MacBook Air (13 inch) to reach the backtrace part, which contains all the information I need. ## Proposal My proposal is to default to a "minimal report" mode with a limited set of sections, perhaps only "Ruby level backtrace information" and "C level backtrace information" only When a full report is desired (i.e. for bug reports), the user could re-run the script with an special environment variable, such as `RUBY_FULL_CRASH_REPORT=1`. Rust implmements a similar pattern. It doesn't print the full backtrace on panics by default; instead, it guides the user to re-run the program with `RUST_BACKTRACE=1`. It might be hard to reproduce some crashes and segfaults, especially in long-running daemons. It might be nice to default to the "full" mode when stdout is not a tty, since daemons tend to run in non-tty environments. ## Appendix A typical crash report would look like this: ``` ../../example.rb: [BUG] ruby 3.4.0dev (2024-01-20T15:27:19Z master 366b14c0cd) [arm64-darwin23] -- Crash Report log information -------------------------------------------- (5 lines) -- Control frame information ----------------------------------------------- (~50 lines) -- Ruby level backtrace information ---------------------------------------- (depends on program; typically ~50 lines in Rails) -- C level backtrace information ------------------------------------------- (50+-ish lines, depends on program) -- Machine register context ------------------------------------------------ (~10 lines) -- Threading information --------------------------------------------------- (2 lines) -- Other runtime information ----------------------------------------------- * Loaded script (1 line) * Loaded features (depends on program; 800+ lines in Rails) * Process memory map (depends on environment; around 200 lines?) -- https://bugs.ruby-lang.org/

5 10

[ruby-core:111121] [Ruby master Bug#19165] Method (with no param) delegation with *, **, and ... is slow
by matsuda (Akira Matsuda) 16 Jan '25

16 Jan '25

Issue #19165 has been reported by matsuda (Akira Matsuda). ---------------------------------------- Bug #19165: Method (with no param) delegation with *, **, and ... is slow https://bugs.ruby-lang.org/issues/19165 * Author: matsuda (Akira Matsuda) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0dev (2022-12-01T08:05:41Z master 4e68b59431) +YJIT [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- I found that method delegation via Forwardable is much slower than normal method call when delegating a method that does not take parameters. Here's a benchmark that explains what I mean. ``` require 'forwardable' require 'pp' require 'benchmark/ips' class Obj extend Forwardable attr_accessor :other def initialize @other = Other.new end def foo_without_splat @other.foo end def foo_with_splat(*) @other.foo(*) end def foo_with_splat_with_name(*args) @other.foo(*args) end def foo_with_splat_and_double_splat(*, **) @other.foo(*, **) end def foo_with_triple_dots(...) @other.foo(...) end delegate :foo => :@other end class Other def foo() end end o = Obj.new Benchmark.ips do |x| x.report 'simple call' do o.other.foo end x.report 'delegate without splat' do o.foo_without_splat end x.report 'delegate with splat' do o.foo_with_splat end x.report 'delegate with splat with name' do o.foo_with_splat_with_name end x.report 'delegate with splat and double splat' do o.foo_with_splat_and_double_splat end x.report 'delegate with triple dots' do o.foo_with_triple_dots end x.report 'delegate via forwardable' do o.foo end end (result) simple call 38.918M (± 0.9%) i/s - 194.884M delegate without splat 31.933M (± 1.6%) i/s - 159.611M delegate with splat 10.269M (± 1.6%) i/s - 51.631M delegate with splat with name 9.888M (± 1.0%) i/s - 49.588M delegate with splat and double splat 4.117M (± 0.9%) i/s - 20.696M delegate with triple dots 4.169M (± 0.9%) i/s - 20.857M delegate via forwardable 9.204M (± 2.1%) i/s - 46.295M ``` It shows that Method delegation with a splat is 3-4 times slower (regardless of whether the parameter is named or not), and delegation with a triple-dot literal is 9-10 times slower than a method delegation without an argument. This may be because calling a method taking a splat always assigns an Array object even when no actual argument was given, and calling a method taking triple-dots assigns five Array objects and two Hash objects (this is equivalent to `*, **`). Are there any chance reducing these object assignments and making them faster? My concern is that the Rails framework heavily uses this kind of method delegations, and presumably it causes unignorable performance overhead. -- https://bugs.ruby-lang.org/

6 8

[ruby-core:112005] [Ruby master Bug#19374] Issue with Ractor.make_shareable with curried procs
by luke-gru (Luke Gruber) 13 Jan '25

13 Jan '25

Issue #19374 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19374: Issue with Ractor.make_shareable with curried procs https://bugs.ruby-lang.org/issues/19374 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This works, but shouldn't: ```ruby class Worker def start(&blk) blk = blk.curry # bug in ruby allows sharing of non-shareable proc Ractor.make_shareable(blk) @ractor = Ractor.new(blk) do |b| main = b.call p "from ractor: #{main}" end end def work @ractor.take end end worker = Worker.new a = self # unshareable main object p "from main: #{a}" worker.start { a } worker.work ``` The curried proc has a reference to the original proc and it's not checked for shareability. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:118284] [Ruby master Misc#20575] Syntax Moratorium til 3.4
by matz (Yukihiro Matsumoto) 13 Jan '25

13 Jan '25

Issue #20575 has been reported by matz (Yukihiro Matsumoto). ---------------------------------------- Misc #20575: Syntax Moratorium til 3.4 https://bugs.ruby-lang.org/issues/20575 * Author: matz (Yukihiro Matsumoto) * Status: Open ---------------------------------------- Hi, As some may remember from the closing keynote at RubyKaigi, we are introducing Syntax Moratorium to support fair development of parsers. During the Syntax Moratorium period, new syntactical changes will be suspended (changes not includes bug fixes, of course). The moratorium period is scheduled to end with the release of 3.4, since the respective parser developers told us they did not need a very long period of time. Matz. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117826] [Ruby master Bug#20480] Complie failure for unable to configure NET_LUID using msys2 mingw64-gcc
by nekoyama32767 (Jinsong Yu) 09 Jan '25

09 Jan '25

Issue #20480 has been reported by nekoyama32767 (Jinsong Yu). ---------------------------------------- Bug #20480: Complie failure for unable to configure NET_LUID using msys2 mingw64-gcc https://bugs.ruby-lang.org/issues/20480 * Author: nekoyama32767 (Jinsong Yu) * Status: Open * ruby -v: ruby 3.4.0dev (2024-05-10T04:54:31Z master 2f915e729a) [x64-mingw32] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Master commit:`2f915e729ac8c66f4009f4b28a57773923d7e7d1` NET_LUID is not successfully checked in win32 configure. When try to compile in msys2 mingw64-gcc there will be a error message like follow. ``` win32/win32.c:4365:7: error: conflicting types for 'NET_LUID'; have 'struct <anonymous>' 4365 | } NET_LUID; | ^~~~~~~~ In file included from D:/msys64/mingw64/include/iptypes.h:17, from D:/msys64/mingw64/include/iphlpapi.h:17, from ./include/ruby/win32.h:40, from ./include/ruby/internal/dosish.h:38, from ./include/ruby/defines.h:78, from ./include/ruby/ruby.h:25, from win32/win32.c:24: D:/msys64/mingw64/include/ifdef.h:106:3: note: previous declaration of 'NET_LUID' with type 'NET_LUID' 106 | } NET_LUID, *PNET_LUID; | ^~~~~~~~ ``` Mingw64-gcc has NET_LUID, when comment out line 4357-line 4367 in `win32/win32.c` in follow, compile will be successed. ``` #ifndef HAVE_TYPE_NET_LUID typedef struct { uint64_t Value; struct { uint64_t Reserved :24; uint64_t NetLuidIndex :24; uint64_t IfType :16; } Info; } NET_LUID; #endif ``` mingw64-gcc version ``` gcc --version gcc.exe (Rev4, Built by MSYS2 project) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ``` -- https://bugs.ruby-lang.org/

2 3