February 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:120940] [Ruby master Bug#21128] dir.c defines O_CLOEXEC before it is defined by the system.
by collinfunk (Collin Funk) 12 Feb '25

12 Feb '25

Issue #21128 has been reported by collinfunk (Collin Funk). ---------------------------------------- Bug #21128: dir.c defines O_CLOEXEC before it is defined by the system. https://bugs.ruby-lang.org/issues/21128 * Author: collinfunk (Collin Funk) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Building from master commit hash bfcf36fe2243d34a01b5707754be9164849150c9 on GNU/Linux, I see: ``` compiling dir.c In file included from /usr/include/bits/fcntl.h:61, from /usr/include/fcntl.h:35, from dir.c:39: /usr/include/bits/fcntl-linux.h:144:10: warning: "O_CLOEXEC" redefined 144 | # define O_CLOEXEC __O_CLOEXEC /* Set close_on_exec. */ | ^~~~~~~~~ dir.c:26:11: note: this is the location of the previous definition 26 | # define O_CLOEXEC 0 | ^~~~~~~~~ ``` This is because the file includes unistd.h before falling back to 0 for O_CLOEXEC. But glibc requires inclusion of fcntl.h for O_CLOEXEC. Since that file is included after, it is redefined. The C standard leaves the behavior of redefining a variable undefined and it is a simple fix, so I will create a pull request. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:120939] [Ruby master Bug#18605] Fails to run on (newer) 32bit Windows with ucrt
by luke-gru (Luke Gruber) 11 Feb '25

11 Feb '25

Issue #18605 has been updated by luke-gru (Luke Gruber). I'm trying to get ARM64 build of Windows to work so I can debug an issue I was having with a branch I'm working on that's failing in CI with Visual Studio 2022. It would be great if ARM64 Windows was working. Any news on this? ---------------------------------------- Bug #18605: Fails to run on (newer) 32bit Windows with ucrt https://bugs.ruby-lang.org/issues/18605#change-111830 * Author: lazka (Christoph Reiter) * Status: Open * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- 32bit ruby using ucrt has started to fail on newer Windows with "unexpected ucrtbase.dll" -> https://github.com/ruby/ruby/blob/3fb7d2cadc18472ec107b14234933b017a33c14d/… The problem is that ruby depends on ucrt internals and those have apparently changed with newer versions. See https://github.com/msys2/MINGW-packages/pull/10878 and https://github.com/msys2/MINGW-packages/issues/10896 for some background and a potential fix. But ideally ruby wouldn't depend on Windows internals like this. thanks! -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120938] [Ruby master Bug#16145] regexp match error if mixing /i, character classes, and utf8
by mjrzasa 11 Feb '25

11 Feb '25

Issue #16145 has been updated by mjrzasa (Maciek Rząsa). I've tested it for Polish letters, the bug appears only for `ó`, all other work OK: ``` pry(main)> ['ą', 'ę', 'ó', 'ś', 'ł', 'ć', 'ź', 'ż', 'ń'].map { [_1, _1.bytes, /[x#{_1}]/i.match?("qwer#{_1.capitalize}")] } => [["ą", [196, 133], true], ["ę", [196, 153], true], ["ó", [195, 179], false], ["ś", [197, 155], true], ["ł", [197, 130], true], ["ć", [196, 135], true], ["ź", [197, 186], true], ["ż", [197, 188], true], ["ń", [197, 132], true]] ``` `ó`, like `é` starts with a byte of `195` ``` pry(main)> 'é'.bytes => [195, 169] ``` ---------------------------------------- Bug #16145: regexp match error if mixing /i, character classes, and utf8 https://bugs.ruby-lang.org/issues/16145#change-111829 * Author: zenspider (Ryan Davis) * Status: Open * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- (reported on behalf of mage(a)mage.gold -- there appears to be an error in registration or login): See: ruby-talk @ X-Mail-Count: 440336 2.6.3 :049 > 'SHOP' =~ /[xo]/i => 2 2.6.3 :050 > 'CAFÉ' =~ /[é]/i => 3 2.6.3 :051 > 'CAFÉ' =~ /[xé]/i => nil 2.6.3 :052 > 'CAFÉ' =~ /[xÉ]/i => 3 Expected result: 2.6.3 :051 > 'CAFÉ' =~ /[xé]/i => 3 I tested it on random regex online pages. It does not match on https://regex101.com/ It matches on: https://regexr.com/ https://www.regextester.com/ https://www.freeformatter.com/regex-tester.html (Ignore case turned on). The reason I suppose it’s more like a bug than a feature is the fact that /[é]/i matches 'CAFÉ'. If the //i didn’t work for UTF-8 characters then the /[é]/i wouldn’t match it either. For example, [é] does not match 'CAFÉ' on https://regex101.com/ I could not find a page or a system that behaves the same way as Ruby does. For example, it matches in PostgreSQL 10 (under FreeBSD 12) too: # select 'CAFÉ'~ '[xé]'; ?column? ---------- f (1 row) # select 'CAFÉ' ~* '[xé]'; ?column? ---------- t (1 row) Tested it in IRB on macOS and FreeBSD. $ uname -a && ruby -v && locale Darwin xxx 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64 ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin18] LANG="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_ALL="en_US.UTF-8" $ uname -a && ruby -v && locale FreeBSD xxx 12.0-RELEASE-p9 FreeBSD 12.0-RELEASE-p9 GENERIC amd64 ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-freebsd12.0] LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_ALL=en_US.UTF-8 I installed Ruby with RVM. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120914] [Ruby master Bug#21124] Enumerable#find called without block returns Enumerator without size
by andrykonchin (Andrew Konchin) 11 Feb '25

11 Feb '25

Issue #21124 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #21124: Enumerable#find called without block returns Enumerator without size https://bugs.ruby-lang.org/issues/21124 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.4.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- When a collection's size is known then enumerator that iterates it usually has size (that's `#size` returns a collection size). But an enumerator returned by `Enumerable#find` called without a block doesn't have size: ```ruby [1, 2, 3].find.size # => nil ``` Expected result here is `3`. Usually similar methods in the Enumerable module return enumerators with size (when these enumerators for instance iterate the collection itself): ```ruby [1, 2, 3].each_entry.size # => 3 ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:120922] [Ruby master Feature#21126] Drop default_proc when Hash#freeze is called for better Ractor support
by osyoyu (Daisuke Aritomo) 10 Feb '25

10 Feb '25

Issue #21126 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Feature #21126: Drop default_proc when Hash#freeze is called for better Ractor support https://bugs.ruby-lang.org/issues/21126 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- Hash instances with default_proc set cannot be sent/moved across Ractors, even if they are frozen. Consider the following code. Using a default proc to set an empty Array is a very common pattern, even introduced in the docs. ```ruby h = Hash.new {|h, k| h[k] = [] } h[:foo] << 1 h.freeze Ractor.new(h) {|h| p h }.take # <internal:ractor>:282:in 'Ractor.new': allocator undefined for Proc (TypeError) ``` https://docs.ruby-lang.org/en/3.4/Hash.html#class-Hash-label-Default+Proc One must explicitly call `h.default_proc = nil` before sending the hash to another Ractor. This isn't the most friendly way for the programmer since (1) it is not easy to spot that the default hash is rendering the Hash unsendable, and (2) `Hash#default_proc=` isn't a widely known API anyway (at least from my perspective). ## Proposal Automatically drop the `default` `default_proc` when Hash#freeze is called. They have little use after the Hash gets frozen. Nevertheless, it should be pointed out that this is an incompatibility for `Hash#default` `Hash#default_proc` -- they currently return the original value, but they will return `nil`. Patch: https://github.com/ruby/ruby/pull/12717 Dropping the default_proc on `Hash#freeze` will also be nicer for `Ractor.make_shareable` users, since it does not require users to find the particular Hash with the default_proc buried somewhere. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:120930] [Ruby master Bug#21127] Thread deadlock does not display backtraces
by pocke (Masataka Kuwabara) 10 Feb '25

10 Feb '25

Issue #21127 has been reported by pocke (Masataka Kuwabara). ---------------------------------------- Bug #21127: Thread deadlock does not display backtraces https://bugs.ruby-lang.org/issues/21127 * Author: pocke (Masataka Kuwabara) * Status: Open * ruby -v: ruby 3.5.0dev (2025-02-10T00:27:18Z master 135479a58d) +PRISM [arm64-darwin24] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Previously, Ruby displayed backtraces for each thread on deadlock. However, it has not been shown since Ruby 3.0. It should display the backtrace for debugging. (related: https://bugs.ruby-lang.org/issues/8214) ## Step to reproduce ```ruby def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join ``` Ruby 2.7.8: It displays the trace. ``` ruby-2.7.8 ruby 2.7.8p225 (2023-03-30 revision 1f4d455848) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000004000653e80 main thread:0x000000400035da50 * #<Thread:0x000000400038ad70 sleep_forever> rb_thread_t:0x000000400035da50 native:0x0000004002fb7740 int:0 -e:9:in `join' -e:9:in `<main>' * #<Thread:0x00000040004fe760 -e:6 sleep_forever> rb_thread_t:0x00000040004518a0 native:0x00000040067ae700 int:0 depended by: tb_thread_id:0x000000400035da50 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:6:in `block in <main>' * #<Thread:0x00000040004fe620 -e:7 sleep_forever> rb_thread_t:0x0000004000653e80 native:0x000000400e9af700 int:0 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:7:in `block in <main>' from -e:9:in `<main>' ``` Ruby 3.0.0: There is no trace. ``` ruby-3.0.0 ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00000040006e88c0 main thread:0x00000040003ffa10 * #<Thread:0x000000400046f448 sleep_forever> rb_thread_t:0x00000040003ffa10 native:0x0000004003059740 int:0 * #<Thread:0x000000400073c2e8 -e:6 sleep_forever> rb_thread_t:0x00000040006e15a0 native:0x0000004006850700 int:0 depended by: tb_thread_id:0x00000040003ffa10 * #<Thread:0x000000400073c1f8 -e:7 sleep_forever> rb_thread_t:0x00000040006e88c0 native:0x000000400ea51700 int:0 from -e:9:in `<main>' ``` Ruby 3.5.0.dev (latest): There is no trace. ``` ruby 3.5.0dev (2025-02-10T00:27:18Z master 135479a58d) +PRISM [arm64-darwin24] -e:9:in 'Thread#join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x000000012ae3b530 main thread:0x000000012af04080 * #<Thread:0x0000000104aaaf78 sleep_forever> rb_thread_t:0x000000012af04080 native:0x00000001e9d93240 int:0 * #<Thread:0x0000000104e5e048 -e:6 sleep_forever> rb_thread_t:0x000000012ae23a90 native:0x000000016b5f3000 int:0 depended by: tb_thread_id:0x000000012af04080 * #<Thread:0x0000000104e5deb8 -e:7 sleep_forever> rb_thread_t:0x000000012ae3b530 native:0x000000016b6ff000 int:0 from -e:9:in '<main>' ``` Note: The results of 2.7 and 3.0 are indented because I executed them with all-ruby. ## Cause This commit has introduced this change. https://github.com/ruby/ruby/commit/3b24b7914c16930bfadc89d6aff6326a51c54295 The ticket is https://bugs.ruby-lang.org/issues/17031 It changes `rb_ec_backtrace_str_ary` function in `vm_backtrace.c`. Previously, `rb_ec_backtrace_str_ary(ec, 0, 0)` returns all traces, but it returns an empty array with this change. It should pass `ALL_BACKTRACE_LINES` (`-1`) to get all traces. I used rubyfarm also to confirm that this commit was the cause. With the commit, it does not display the traces. ``` docker run --rm -ti rubylang/rubyfarm:3b24b7914c16930bfadc89d6aff6326a51c54295 ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 2.8.0dev (2020-08-27T22:17:36Z :detached: 3b24b7914c) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000004000566a80 main thread:0x0000004000202a60 * #<Thread:0x000000400026faf8 sleep_forever> rb_thread_t:0x0000004000202a60 native:0x000000400440d740 int:0 * #<Thread:0x00000040004f1768 -e:6 sleep_forever> rb_thread_t:0x0000004000566710 native:0x00000040081fe700 int:0 depended by: tb_thread_id:0x0000004000202a60 * #<Thread:0x00000040004f1600 -e:7 sleep_forever> rb_thread_t:0x0000004000566a80 native:0x00000040103ff700 int:0 from -e:9:in `<main>' ``` With the parent commit, it displays the traces. ``` docker run --rm -ti rubylang/rubyfarm:8095114f1715070fcdc2b29303dcf55a7fcc32a3 ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 2.8.0dev (2020-08-27T19:54:36Z :detached: 8095114f17) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x000000400055b3a0 main thread:0x0000004000202a60 * #<Thread:0x000000400026faf8 sleep_forever> rb_thread_t:0x0000004000202a60 native:0x000000400440d740 int:0 -e:9:in `join' -e:9:in `<main>' * #<Thread:0x00000040004f1768 -e:6 sleep_forever> rb_thread_t:0x000000400055afe0 native:0x00000040081fe700 int:0 depended by: tb_thread_id:0x0000004000202a60 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:6:in `block in <main>' * #<Thread:0x00000040004f1600 -e:7 sleep_forever> rb_thread_t:0x000000400055b3a0 native:0x00000040103ff700 int:0 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:7:in `block in <main>' from -e:9:in `<main>' ``` I think it's not an intentional change. The ticket and NEWS do not mention the backtrace on a deadlock. (And the backtrace is really helpful!) ## Solution The following patch fixes the problem. I'll make a PR on GitHub soon. ```diff diff --git a/eval_intern.h b/eval_intern.h index 9a551120ff..f783c93aff 100644 --- a/eval_intern.h +++ b/eval_intern.h @@ -300,6 +300,8 @@ void rb_vm_set_progname(VALUE filename); VALUE rb_vm_cbase(void); /* vm_backtrace.c */ +#define BACKTRACE_START 0 +#define ALL_BACKTRACE_LINES -1 VALUE rb_ec_backtrace_object(const rb_execution_context_t *ec); VALUE rb_ec_backtrace_str_ary(const rb_execution_context_t *ec, long lev, long n); VALUE rb_ec_backtrace_location_ary(const rb_execution_context_t *ec, long lev, long n, bool skip_internal); diff --git a/thread.c b/thread.c index d40d5140b7..5280c667af 100644 --- a/thread.c +++ b/thread.c @@ -5596,7 +5596,7 @@ debug_deadlock_check(rb_ractor_t *r, VALUE msg) } } rb_str_catf(msg, "\n "); - rb_str_concat(msg, rb_ary_join(rb_ec_backtrace_str_ary(th->ec, 0, 0), sep)); + rb_str_concat(msg, rb_ary_join(rb_ec_backtrace_str_ary(th->ec, BACKTRACE_START, ALL_BACKTRACE_LINES), sep)); rb_str_catf(msg, "\n"); } } diff --git a/vm_backtrace.c b/vm_backtrace.c index 56a0d861d6..6f2cf9e169 100644 --- a/vm_backtrace.c +++ b/vm_backtrace.c @@ -31,9 +31,6 @@ id2str(ID id) } #define rb_id2str(id) id2str(id) -#define BACKTRACE_START 0 -#define ALL_BACKTRACE_LINES -1 - inline static int calc_pos(const rb_iseq_t *iseq, const VALUE *pc, int *lineno, int *node_id) { ``` The result is here: ``` $ ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 3.5.0dev (2025-02-10T00:27:18Z origin/master 135479a58d) +PRISM [arm64-darwin24] -e:9:in 'Thread#join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000000157a053a0 main thread:0x0000000156704080 * #<Thread:0x000000011bcfaf78 sleep_forever> rb_thread_t:0x0000000156704080 native:0x00000001e9d93240 int:0 -e:9:in 'Thread#join' -e:9:in '<main>' * #<Thread:0x000000011d2d0bb8 -e:6 sleep_forever> rb_thread_t:0x0000000157a04d40 native:0x000000016dcb7000 int:0 depended by: tb_thread_id:0x0000000156704080 <internal:thread_sync>:18:in 'Thread::Queue#pop' -e:3:in 'Object#h' -e:2:in 'Object#g' -e:1:in 'Object#f' -e:6:in 'block in <main>' * #<Thread:0x000000011d2d07a8 -e:7 sleep_forever> rb_thread_t:0x0000000157a053a0 native:0x000000016ddc3000 int:0 <internal:thread_sync>:18:in 'Thread::Queue#pop' -e:3:in 'Object#h' -e:2:in 'Object#g' -e:1:in 'Object#f' -e:7:in 'block in <main>' from -e:9:in '<main>' ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120908] [Ruby master Bug#21122] library/socket/basicsocket/recv_nonblock_spec.rb fails with IO::EAGAINWaitReadable sometimes.
by hsbt (Hiroshi SHIBATA) 10 Feb '25

10 Feb '25

Issue #21122 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #21122: library/socket/basicsocket/recv_nonblock_spec.rb fails with IO::EAGAINWaitReadable sometimes. https://bugs.ruby-lang.org/issues/21122 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- After https://github.com/ruby/ruby/commit/d7a5ad2a21f7d2c45e3fea674ff077bb0e2cadae, I got test failure with `library/socket/basicsocket/recv_nonblock_spec.rb` sometimes. * https://github.com/ruby/ruby/actions/runs/13108171270/job/36566403804#step:… * https://github.com/ruby/ruby/actions/runs/13195573860/job/36836170533?pr=12… * my local macOS. @andrykonchin @Eregon Can you look this? -- https://bugs.ruby-lang.org/

2 2

[ruby-core:120869] [Ruby master Misc#21110] Should Marshal.dump always use object links for repeated Float values?
by andrykonchin (Andrew Konchin) 09 Feb '25

09 Feb '25

Issue #21110 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Misc #21110: Should Marshal.dump always use object links for repeated Float values? https://bugs.ruby-lang.org/issues/21110 * Author: andrykonchin (Andrew Konchin) * Status: Open ---------------------------------------- I've noticed (during the downstreaming ruby/spec in https://github.com/ruby/ruby/pull/12679) that a repeated Float value may be dumped "directly"/"immediately" without an objct link on x86 architecture, and with a link in other cases. It seems it depends on whether Float is `Flonum` or not. amd64: ```ruby Marshal.dump([0.0, 0.0]) # => "\x04\b[\af\x060@\x06", where @\x06 is an object link ``` i686: ```ruby Marshal.dump([0.0, 0.0]) # => "\x04\b[\af\x060f\x060", so f\x060 is dumped twice ``` Both dumps are loaded correctly, so it seems there is no any compatibility issue. Should dumping be consistent and don't depend on architecture? -- https://bugs.ruby-lang.org/

4 5

[ruby-core:120573] [Ruby master Misc#21019] DevMeeting-2025-02-13
by mame (Yusuke Endoh) 08 Feb '25

08 Feb '25

Issue #21019 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #21019: DevMeeting-2025-02-13 https://bugs.ruby-lang.org/issues/21019 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- # The next dev meeting **Date: 2025/02/13 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2025/02/10. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

5 4

[ruby-core:120749] [Ruby master Feature#21084] Declare objects have weak references
by peterzhu2118 (Peter Zhu) 06 Feb '25

06 Feb '25

Issue #21084 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Feature #21084: Declare objects have weak references https://bugs.ruby-lang.org/issues/21084 * Author: peterzhu2118 (Peter Zhu) * Status: Open ---------------------------------------- # Summary The current way of marking weak references uses `rb_gc_mark_weak(VALUE *ptr)`. This presents challenges because Ruby's GC is incremental, meaning that if the `ptr` changes (e.g. realloc'd or free'd), then we could have an invalid memory access. This also overwrites `*ptr = Qundef` if `*ptr` is dead, which prevents any cleanup to be run (e.g. freeing memory or deleting entries from hash tables). This ticket proposes `rb_gc_declare_weak_references` which declares that an object has weak references and calls a cleanup function after marking, allowing the object to clean up any memory for dead objects. # Introduction In [[Feature #19783]](https://bugs.ruby-lang.org/issues/19783), I introduced an API allowing objects to mark weak references, the function signature looks like this: ```c void rb_gc_mark_weak(VALUE *ptr); ``` `rb_gc_mark_weak` is called during the marking phase of the GC to specify that the memory at `ptr` holds a pointer to a Ruby object that is weakly referenced. `rb_gc_mark_weak` appends this pointer to a list that is processed after the marking phase of the GC. If the object at `*ptr` is no longer alive, then it overwrites the object reference with a special value (`*ptr = Qundef`). However, this API resulted in two challenges: 1. Ruby's default GC is incremental, which means that the GC is not ran in one phase, but rather split into chunks of work that interleaves with Ruby execution. The `ptr` passed into `rb_gc_mark_weak` could be on the malloc heap, and that memory could be realloc'd or even free'd. We had to use workarounds such as `rb_gc_remove_weak` to ensure that there were no illegal memory accesses. This made `rb_gc_mark_weak` difficult to use, impacted runtime performance, and increased memory usage. 2. When an object dies, `rb_gc_mark_weak` only overwites the reference with `Qundef`. This means that if we want to do any cleanup (e.g. free a piece of memory or delete a hash table entry), we could not do that and had to defer this process elsewhere (e.g. during marking or runtime). # Declarative weak references In this ticket, I'm proposing a new API for weak references. Instead of an object marking its weak references during the marking phase, the object declares that it has weak references using the `rb_gc_declare_weak_references` function. This declaration occurs during runtime (e.g. after the object has been created) rather than during GC. After an object declares that it has weak references, it will have its callback function called after marking as long as that object is alive. This callback function can then call a special function `rb_gc_handle_weak_references_alive_p` to determine whether its references are alive. This will allow the callback function to do whatever it wants on the object, allowing it to perform any cleanup work it needs. This significantly simplifies the code for `ObjectSpace::WeakMap` and `ObjectSpace::WeakKeyMap` because it no longer needs to have the workarounds for the limitations of `rb_gc_mark_weak`. # Performance The performance results below demonstrate that `ObjectSpace::WeakMap#[]=` is now about 60% faster because the implementation has been simplified and the number of allocations has been reduced. We can see that there is not a significant impact on the performance of `ObjectSpace::WeakMap#[]`. Base: ``` ObjectSpace::WeakMap#[]= 4.620M (± 6.4%) i/s (216.44 ns/i) - 23.342M in 5.072149s ObjectSpace::WeakMap#[] 30.967M (± 1.9%) i/s (32.29 ns/i) - 154.998M in 5.007157s ``` Branch: ``` ObjectSpace::WeakMap#[]= 7.336M (± 2.8%) i/s (136.31 ns/i) - 36.755M in 5.013983s ObjectSpace::WeakMap#[] 30.902M (± 5.4%) i/s (32.36 ns/i) - 155.901M in 5.064060s ``` Code: ``` require "bundler/inline" gemfile do source "https://rubygems.org" gem "benchmark-ips" end wmap = ObjectSpace::WeakMap.new key = Object.new val = Object.new wmap[key] = val Benchmark.ips do |x| x.report("ObjectSpace::WeakMap#[]=") do |times| i = 0 while i < times wmap[Object.new] = Object.new i += 1 end end x.report("ObjectSpace::WeakMap#[]") do |times| i = 0 while i < times wmap[key] wmap[val] # does not exist i += 1 end end end ``` # Alternative designs Currently, `rb_gc_declare_weak_references` is designed to be an internal-only API. This allows us to assume the object types that call `rb_gc_declare_weak_references`. In the future, if we want to open up this API to third parties, we may want to change this function to something like: ```c void rb_gc_add_cleaner(VALUE obj, void (*callback)(VALUE obj)); ``` This will allow the third party to implement a custom `callback` that gets called after the marking phase of GC to clean up any dead references. I chose not to implement this design because it is less efficient as we would need to store a mapping from `obj` to `callback`, which requires extra memory. -- https://bugs.ruby-lang.org/

2 2