May 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112906] [Ruby master Bug#19531] ObjectSpace::WeakMap: replaced values still clear the key they were assigned to
by byroot (Jean Boussier) 22 Jul '23

22 Jul '23

Issue #19531 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #19531: ObjectSpace::WeakMap: replaced values still clear the key they were assigned to https://bugs.ruby-lang.org/issues/19531 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 2.7: WONTFIX, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- ### Reproduction script ```ruby wmap = ObjectSpace::WeakMap.new a = "A" b = "B" wmap[1] = a wmap[1] = b # the table entry with 1 is still in the list of entries to clear when `a` is GCed a = nil GC.start p wmap[1] # Should be `"B"`, but is `nil` ``` ### Explanation What happens is that when we set `wmap[1] = "A"`, WeakMap internally keeps a list of keys to clear when `"A"` is GCed, e.g. pseudo code: ```ruby class WeakMap def []=(key, value) @hash[key] = value @reverse[value] << key end end ``` But it doesn't clear previously kept mapping when a key is overwritten. I'll work on a fix. ### References https://github.com/protocolbuffers/protobuf/pull/12216 -- https://bugs.ruby-lang.org/

2 2

[ruby-core:111873] [Ruby master Bug#19351] Promote bundled gems at Ruby 3.3
by hsbt (Hiroshi SHIBATA) 19 Jul '23

19 Jul '23

Issue #19351 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19351: Promote bundled gems at Ruby 3.3 https://bugs.ruby-lang.org/issues/19351 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- In Ruby 3.2, the default gems and bundled gems are changed only adding `syntax_suggest`. I and some committers are considering promote default gems to bundled gems again for Ruby 3.3+. We hope to keep the current developer experience with dependency resolution and ignore the additional work like "Put gem "xxx" into your Gemfile" for developers. ### Proposal We propose the following libraries will promote default gems to bundled gems at Ruby 3.3. They are not the dependencies of Rails and RubyGems/Bundler. ``` abbrev getoptlong optparse observable resolv resolv-replace rinda un fcntl nkf syslog win32ole ``` ### Additional works I also propose to poromote rails dependencies: ``` ostruct base64 irb rdoc tsort singleton delegate ``` and gems maintained by @kou ``` csv strscan fiddle stringio ``` But if we promote them to bundled gems, many of users need to add `gem "csv"` into their Gemfile. I'm considering to avoid this situation. Can we the specific feature of bundled gems to RubyGems or Bundler? Example, bundler have allowed list for bundled gems. So, listed gems could be require without Gemfile under the bundle exec. -- https://bugs.ruby-lang.org/

6 25

[ruby-core:113517] [Ruby master Misc#19679] Migrate Wiki from bugs.ruby-lang.org to ruby/ruby GitHub repository
by jemmai (Jemma Issroff) 19 Jul '23

19 Jul '23

Issue #19679 has been reported by jemmai (Jemma Issroff). ---------------------------------------- Misc #19679: Migrate Wiki from bugs.ruby-lang.org to ruby/ruby GitHub repository https://bugs.ruby-lang.org/issues/19679 * Author: jemmai (Jemma Issroff) * Status: Open * Priority: Normal ---------------------------------------- # Background There is currently a Wiki at https://bugs.ruby-lang.org/projects/ruby/wiki which contains documentation of processes, documentation of code, developers' meeting notes, and various other notes. There are three main problems with the current Wiki setup: * **Out of date:** Much of the documentation is out of date, or no longer relevant (for example: [How to Release Ruby 1.9](https://bugs.ruby-lang.org/projects/ruby/wiki/HowToReleaseRuby19)) * **Duplication:** Some of this documentation exists both in the Wiki and in the docs in ruby/ruby (for example: [DTrace Probes in the docs](https://docs.ruby-lang.org/en/master/dtrace_probes_rdoc.html) and [DTrace Probes in the Wiki](https://bugs.ruby-lang.org/projects/ruby/wiki/DTraceProbes) * **Editing docs is limited:** Only Core contributors can edit the Wiki, and version control is more limited (for example: I couldn't edit any mistakes I found in the Wiki while preparing this issue). # Proposal This proposal is to migrate any still relevant docs within the Wiki from the bugs.ruby-lang.org Wiki to a new GitHub Wiki. I have categorized all of the Wiki pages in [this google sheet](https://docs.google.com/spreadsheets/d/1Ld83ZKxknYgECXxNSh28fjFw82pS… and given my analysis of which should be migrated. If there are any pages that should be migrated which aren't currently included, please indicate so in the comments and we will migrate them. I also propose we migrate the old Developers' Meeting notes to the [developer's meeting repository](https://github.com/ruby/dev-meeting-log). I spoke about this idea with @hsbt at Ruby Kaigi, and he and @naruse suggested that I write this proposal. # Implementation 0. Before I begin the migration, someone who has the appropriate permissions on GitHub will need [to create the Wiki](https://docs.github.com/en/communities/documenting-your-project-with-… 1. I will migrate all pages listed in the Google sheet with "migrate?" checked to the new GitHub wiki 2. I will migrate all of the Developers' Meeting Notes to the developers' meeting repo 3. Someone who has the appropriate permissions will need to remove the Wiki from bugs.ruby-lang.org 4. I will update all GitHub wiki pages with "update?" checked in the Google Sheet to reflect the latest state -- https://bugs.ruby-lang.org/

5 10

[ruby-core:113109] [Ruby master Bug#19577] Backport commit 537183cd2ac0163851277b46a2f21ea5914c11c0
by peterzhu2118 (Peter Zhu) 18 Jul '23

18 Jul '23

Issue #19577 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19577: Backport commit 537183cd2ac0163851277b46a2f21ea5914c11c0 https://bugs.ruby-lang.org/issues/19577 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- Please backport commit [537183cd2ac0163851277b46a2f21ea5914c11c0](https://github.com/ruby/ruby/comm…. We've seen this crash in our production systems. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113129] [Ruby master Bug#19580] Ensure ruby_xfree won't segfault if called after ruby_vm_destruct
by mdalessio (Mike Dalessio) 18 Jul '23

18 Jul '23

Issue #19580 has been reported by mdalessio (Mike Dalessio). ---------------------------------------- Bug #19580: Ensure ruby_xfree won't segfault if called after ruby_vm_destruct https://bugs.ruby-lang.org/issues/19580 * Author: mdalessio (Mike Dalessio) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Github PR: https://github.com/ruby/ruby/pull/7663 The POSIX Threads API provides the ability to define a destructor to clean up thread-local storage at thread exit by using `pthread_key_create` to bind a storage key and an associated destructor function, and `pthread_setspecific` to associate a value with that key. If a C extension is using `ruby_xmalloc` and `ruby_xfree` to manage memory, then the destructor function will likely call `ruby_xfree`. There is a small window of time as a process is exiting -- after `ruby_vm_destruct` but before the process exits -- in which a thread may exit and the destructor function called, leading to a segfault. ### Real-world use case The real-world scenario motivating this change is libxml2's pthread code which uses `pthread_key_create` to set up a destructor that is called at thread exit to clean up thread-local storage associated with the thread by `pthread_setspecific`. Nokogiri has, since 2009, configured libxml2 to use `ruby_xmalloc` and `ruby_xfree` so that Ruby's memory management is aware of the allocated memory. As a result, `ruby_xfree` is being called by the pthread destructor, which may lead to a segfault that looks like: [BUG] Segmentation fault at 0x0000000000000420 ruby 3.3.0dev (2023-03-19T06:13:25Z master ca9355e173) [x86_64-linux] -- Machine register context ------------------------------------------------ RIP: 0x00007f9b6311124e RBP: 0x00007f9b58001530 RSP: 0x00007f9b5d8eed60 RAX: 0x0000000000000000 RBX: 0x00007f9b58001888 RCX: 0x00000000000003ff RDX: 0x00007f9b5dad62b0 RDI: 0x00007f9b58007090 RSI: 0x0000000000000000 R8: 0x00007f9b5d8eed64 R9: 0x00000000000000ca R10: 0x0000000000000000 R11: 0x0000000000000246 R12: 0x00007f9b58007090 R13: 0x00007f9b62e1bae8 R14: 0x0000000000000004 R15: 0x00007f9b5d8efb58 EFL: 0x0000000000010206 -- C level backtrace information ------------------------------------------- .../3.3.0-dev/lib/libruby.so.3.3(rb_print_backtrace+0xd) [0x7f9b632e6f1f] /home/flavorjones/code/oss/ruby/vm_dump.c:785 .../3.3.0-dev/lib/libruby.so.3.3(rb_vm_bugreport) /home/flavorjones/code/oss/ruby/vm_dump.c:1101 .../3.3.0-dev/lib/libruby.so.3.3(rb_bug_for_fatal_signal+0xf4) [0x7f9b630eead4] /home/flavorjones/code/oss/ruby/error.c:813 .../3.3.0-dev/lib/libruby.so.3.3(sigsegv+0x4d) [0x7f9b63233bbd] /home/flavorjones/code/oss/ruby/signal.c:964 /lib/x86_64-linux-gnu/libc.so.6(0x7f9b62c42520) [0x7f9b62c42520] .../3.3.0-dev/lib/libruby.so.3.3(ruby_sized_xfree+0x3) [0x7f9b6311124e] /home/flavorjones/code/oss/ruby/gc.c:12666 .../3.3.0-dev/lib/libruby.so.3.3(ruby_sized_xfree) /home/flavorjones/code/oss/ruby/gc.c:12663 .../lib/nokogiri/nokogiri.so(xmlResetError+0x0) [0x7f9b5da6ab4a] .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/error.c:880 .../lib/nokogiri/nokogiri.so(xmlResetError) .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/error.c:873 .../lib/nokogiri/nokogiri.so(xmlResetError) (null):0 .../lib/nokogiri/nokogiri.so(xmlFreeGlobalState+0x14) [0x7f9b5dad62c4] .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/threads.c:548 /lib/x86_64-linux-gnu/libc.so.6(__nptl_deallocate_tsd+0x77) [0x7f9b62c91711] ./nptl/nptl_deallocate_tsd.c:73 /lib/x86_64-linux-gnu/libc.so.6(__nptl_deallocate_tsd) ./nptl/nptl_deallocate_tsd.c:22 /lib/x86_64-linux-gnu/libc.so.6(start_thread+0x17a) [0x7f9b62c949ca] ./nptl/pthread_create.c:453 [0x7f9b62d26a00] I've reproduced this in Ruby 3.0, 3.1, 3.2, and master by using the following ruby code: # must have an error in it to cause pthread_setspecific to be called html = "<div foo='asdf>asdf</div>" Thread.new { Nokogiri::HTML4::Document.parse(html) } sleep 3 # THREAD_CACHE_TIME exit 0 The timing is admittedly hard to reproduce, but [Gitlab has seen this in their CI pipelines a few dozen times](https://gitlab.com/gitlab-org/gitlab/-/issues/390313) and it's made easier if the process lifetime is extended by an `atexit`-registered function. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:113198] [Ruby master Bug#19593] Crash due to throw data set as cause
by peterzhu2118 (Peter Zhu) 18 Jul '23

18 Jul '23

Issue #19593 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19593: Crash due to throw data set as cause https://bugs.ruby-lang.org/issues/19593 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/7696 rb_ec_setup_exception did not check if errinfo is a throw_data. This can cause crashes in code since it is assumed that id_cause is an object. We saw a crash in show_cause due to id_cause of errinfo being a throw_data. It crashes on rb_obj_is_kind_of since it cannot be called on T_IMEMO objects. Unfortunately, we couldn't find a reproduction script, however we debugged the core dump and rb_ec_setup_exception is the only place where id_cause is assigned from errinfo without checking if it is a throw_data. ``` 0x0000556c5708e6dd in sigsegv (sig=11, info=0x7f301befa3f0, ctx=0x7f301befa2c0) at signal.c:964 0x00007f301d046420 in <signal handler called> () at /lib/x86_64-linux-gnu/libpthread.so.0 class_search_class_ancestor (c=139844586301760, cl=<optimized out>) at object.c:810 rb_obj_is_kind_of (obj=obj@entry=139839221734880, c=139844586301760) at object.c:861 0x0000556c56f2f00f in show_cause (errinfo=errinfo@entry=139838840645160, str=str@entry=139839221730520, opt=139839221730480, highlight=0, reverse=reverse@entry=0, backtrace_limit=backtrace_limit@entry=-1, shown_causes=0x7ffe9d1a2d68) at ./include/ruby/internal/special_consts.h:175 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113260] [Ruby master Bug#19601] YJIT `try to mark T_NONE object` stemming from object shape transition on `self`
by alanwu (Alan Wu) 17 Jul '23

17 Jul '23

Issue #19601 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #19601: YJIT `try to mark T_NONE object` stemming from object shape transition on `self` https://bugs.ruby-lang.org/issues/19601 * Author: alanwu (Alan Wu) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: REQUIRED ---------------------------------------- We've identified a false collection bug with YJIT. Symptoms can range from `[BUG] try to mark T_NONE object` to SEGVs. Due to the bug requiring specific transient heap state to reproduce, it may be hard to identify by looking at the crash-site stack trace. `ruby --yjit-call-threshold=1` reproducer: ```ruby class RegressionTest def initialize @a = @b = @fourth_ivar_does_shape_transition = nil end def extender @first_extended_ivar = [:ok] end end test = RegressionTest.new # Fill up the transient heap, so rb_ensure_iv_list_size() # listens to GC.stress and yields to the GC. fill = Array.new(0x400000) GC.stress = true # Used to crash due to GC run in rb_ensure_iv_list_size() # not marking the newly allocated [:ok]. test.extender GC.start ``` I will post a patch shortly. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:112533] [Ruby master Bug#19455] Ruby 3.2: wrong Regexp encoding with non-ASCII comments
by janosch-x 17 Jul '23

17 Jul '23

Issue #19455 has been reported by janosch-x (Janosch Müller). ---------------------------------------- Bug #19455: Ruby 3.2: wrong Regexp encoding with non-ASCII comments https://bugs.ruby-lang.org/issues/19455 * Author: janosch-x (Janosch Müller) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- comments and comment groups don't trigger the correct `Regexp#encoding` on Ruby 3.2 anymore: ```ruby # ruby 3.1 /#a/x.encoding # => #<Encoding:US-ASCII> # OK /(?#a)/.encoding # => #<Encoding:US-ASCII> # OK /#ü/x.encoding # => #<Encoding:UTF-8> # OK /(?#ü)/.encoding # => #<Encoding:UTF-8> # OK # ruby 3.2 /#a/x.encoding # => #<Encoding:US-ASCII> # OK /(?#a)/.encoding # => #<Encoding:US-ASCII> # OK /#ü/x.encoding # => #<Encoding:US-ASCII> # BUG /(?#ü)/.encoding # => #<Encoding:US-ASCII> # BUG /#ü/x.inspect # => "/#\\xC3\\xBC/x" /(?#ü)/.inspect # => "/(?#\\xC3\\xBC)/" # bug is hidden if there are non-ascii chars outside comments /ü#ü/x.encoding # => #<Encoding:UTF-8> /ü(?#ü)/.encoding # => #<Encoding:UTF-8> ``` i think these changes might be the cause: https://github.com/ruby/ruby/commit/ec3542229b29ec93062e9d90e877ea29d3c1947… @jeremyevans0 JFYI -- https://bugs.ruby-lang.org/

6 6

[ruby-core:112922] [Ruby master Bug#19533] Behavior of ===/include? on a beginless/endless range (nil..nil) changed in ruby 3.2
by knu (Akinori MUSHA) 17 Jul '23

17 Jul '23

Issue #19533 has been reported by knu (Akinori MUSHA). ---------------------------------------- Bug #19533: Behavior of ===/include? on a beginless/endless range (nil..nil) changed in ruby 3.2 https://bugs.ruby-lang.org/issues/19533 * Author: knu (Akinori MUSHA) * Status: Open * Priority: Normal * Target version: 3.2 * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Starting from Ruby 2.7.0 a range `nil..nil` used to match any value until 3.2.0-preview3, but 3.2.0-rc1 started to reject it. ``` % docker run -it --rm rubylang/all-ruby ./all-ruby -e 'p( (nil..nil) === 1 )' (snip) ruby-2.6.10 false ruby-2.7.0-preview1 true ... ruby-3.2.0-preview3 true ruby-3.2.0-rc1 -e:1:in `===': cannot determine inclusion in beginless/endless ranges (TypeError) p( (nil..nil) === 1 ) ^ from -e:1:in `<main>' exit 1 ... ruby-3.2.1 -e:1:in `===': cannot determine inclusion in beginless/endless ranges (TypeError) p( (nil..nil) === 1 ) ^ from -e:1:in `<main>' ``` The previous behavior was so useful because when you have optional lower and upper bounds `lbound..rbound` would always work regardless of each end being nil or not. There is no mention of this in doc/NEWS/NEWS-3.2.0.md, so I'm afraid it was caused unintentionally while fixing other problems. ``` % docker run -it --rm rubylang/all-ruby ./all-ruby -e 'p( (nil..nil).cover?(1) )' (snip) ruby-2.6.10 false ruby-2.7.0-preview1 true ... ruby-3.2.1 true ``` This was pointed out by the following blog article (written in Japanese) as a "pitfall" that you need to work around when upgrading Ruby from 3.0/3.1 to 3.2. https://blog.studysapuri.jp/entry/2023/03/16/ujihisa-ruby32#endless-range%E… -- https://bugs.ruby-lang.org/

5 5

[ruby-core:113182] [Ruby master Bug#19589] Expecting stack overflow but crashing
by alpaca-tc (Hiroyuki Ishii) 17 Jul '23

17 Jul '23

Issue #19589 has been reported by alpaca-tc (Hiroyuki Ishii). ---------------------------------------- Bug #19589: Expecting stack overflow but crashing https://bugs.ruby-lang.org/issues/19589 * Author: alpaca-tc (Hiroyuki Ishii) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-04-11T00:54:20Z master 65e276096f) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following code expects stack overflow but crashes. The version it occurs in is newer than 3.2.0. ``` def expect_system_stack_error(h) h.each_key {} h.each_value { expect_system_stack_error(h) } end expect_system_stack_error(a: nil) ``` -- https://bugs.ruby-lang.org/

4 3