- ruby-core - ml.ruby-lang.org

[ruby-core:113109] [Ruby master Bug#19577] Backport commit 537183cd2ac0163851277b46a2f21ea5914c11c0
by peterzhu2118 (Peter Zhu) 18 Jul '23

18 Jul '23

Issue #19577 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19577: Backport commit 537183cd2ac0163851277b46a2f21ea5914c11c0 https://bugs.ruby-lang.org/issues/19577 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- Please backport commit [537183cd2ac0163851277b46a2f21ea5914c11c0](https://github.com/ruby/ruby/comm…. We've seen this crash in our production systems. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113129] [Ruby master Bug#19580] Ensure ruby_xfree won't segfault if called after ruby_vm_destruct
by mdalessio (Mike Dalessio) 18 Jul '23

18 Jul '23

Issue #19580 has been reported by mdalessio (Mike Dalessio). ---------------------------------------- Bug #19580: Ensure ruby_xfree won't segfault if called after ruby_vm_destruct https://bugs.ruby-lang.org/issues/19580 * Author: mdalessio (Mike Dalessio) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Github PR: https://github.com/ruby/ruby/pull/7663 The POSIX Threads API provides the ability to define a destructor to clean up thread-local storage at thread exit by using `pthread_key_create` to bind a storage key and an associated destructor function, and `pthread_setspecific` to associate a value with that key. If a C extension is using `ruby_xmalloc` and `ruby_xfree` to manage memory, then the destructor function will likely call `ruby_xfree`. There is a small window of time as a process is exiting -- after `ruby_vm_destruct` but before the process exits -- in which a thread may exit and the destructor function called, leading to a segfault. ### Real-world use case The real-world scenario motivating this change is libxml2's pthread code which uses `pthread_key_create` to set up a destructor that is called at thread exit to clean up thread-local storage associated with the thread by `pthread_setspecific`. Nokogiri has, since 2009, configured libxml2 to use `ruby_xmalloc` and `ruby_xfree` so that Ruby's memory management is aware of the allocated memory. As a result, `ruby_xfree` is being called by the pthread destructor, which may lead to a segfault that looks like: [BUG] Segmentation fault at 0x0000000000000420 ruby 3.3.0dev (2023-03-19T06:13:25Z master ca9355e173) [x86_64-linux] -- Machine register context ------------------------------------------------ RIP: 0x00007f9b6311124e RBP: 0x00007f9b58001530 RSP: 0x00007f9b5d8eed60 RAX: 0x0000000000000000 RBX: 0x00007f9b58001888 RCX: 0x00000000000003ff RDX: 0x00007f9b5dad62b0 RDI: 0x00007f9b58007090 RSI: 0x0000000000000000 R8: 0x00007f9b5d8eed64 R9: 0x00000000000000ca R10: 0x0000000000000000 R11: 0x0000000000000246 R12: 0x00007f9b58007090 R13: 0x00007f9b62e1bae8 R14: 0x0000000000000004 R15: 0x00007f9b5d8efb58 EFL: 0x0000000000010206 -- C level backtrace information ------------------------------------------- .../3.3.0-dev/lib/libruby.so.3.3(rb_print_backtrace+0xd) [0x7f9b632e6f1f] /home/flavorjones/code/oss/ruby/vm_dump.c:785 .../3.3.0-dev/lib/libruby.so.3.3(rb_vm_bugreport) /home/flavorjones/code/oss/ruby/vm_dump.c:1101 .../3.3.0-dev/lib/libruby.so.3.3(rb_bug_for_fatal_signal+0xf4) [0x7f9b630eead4] /home/flavorjones/code/oss/ruby/error.c:813 .../3.3.0-dev/lib/libruby.so.3.3(sigsegv+0x4d) [0x7f9b63233bbd] /home/flavorjones/code/oss/ruby/signal.c:964 /lib/x86_64-linux-gnu/libc.so.6(0x7f9b62c42520) [0x7f9b62c42520] .../3.3.0-dev/lib/libruby.so.3.3(ruby_sized_xfree+0x3) [0x7f9b6311124e] /home/flavorjones/code/oss/ruby/gc.c:12666 .../3.3.0-dev/lib/libruby.so.3.3(ruby_sized_xfree) /home/flavorjones/code/oss/ruby/gc.c:12663 .../lib/nokogiri/nokogiri.so(xmlResetError+0x0) [0x7f9b5da6ab4a] .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/error.c:880 .../lib/nokogiri/nokogiri.so(xmlResetError) .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/error.c:873 .../lib/nokogiri/nokogiri.so(xmlResetError) (null):0 .../lib/nokogiri/nokogiri.so(xmlFreeGlobalState+0x14) [0x7f9b5dad62c4] .../tmp/x86_64-linux/nokogiri/3.0.5/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3/libxml2-2.10.3/threads.c:548 /lib/x86_64-linux-gnu/libc.so.6(__nptl_deallocate_tsd+0x77) [0x7f9b62c91711] ./nptl/nptl_deallocate_tsd.c:73 /lib/x86_64-linux-gnu/libc.so.6(__nptl_deallocate_tsd) ./nptl/nptl_deallocate_tsd.c:22 /lib/x86_64-linux-gnu/libc.so.6(start_thread+0x17a) [0x7f9b62c949ca] ./nptl/pthread_create.c:453 [0x7f9b62d26a00] I've reproduced this in Ruby 3.0, 3.1, 3.2, and master by using the following ruby code: # must have an error in it to cause pthread_setspecific to be called html = "<div foo='asdf>asdf</div>" Thread.new { Nokogiri::HTML4::Document.parse(html) } sleep 3 # THREAD_CACHE_TIME exit 0 The timing is admittedly hard to reproduce, but [Gitlab has seen this in their CI pipelines a few dozen times](https://gitlab.com/gitlab-org/gitlab/-/issues/390313) and it's made easier if the process lifetime is extended by an `atexit`-registered function. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:113198] [Ruby master Bug#19593] Crash due to throw data set as cause
by peterzhu2118 (Peter Zhu) 18 Jul '23

18 Jul '23

Issue #19593 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19593: Crash due to throw data set as cause https://bugs.ruby-lang.org/issues/19593 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/7696 rb_ec_setup_exception did not check if errinfo is a throw_data. This can cause crashes in code since it is assumed that id_cause is an object. We saw a crash in show_cause due to id_cause of errinfo being a throw_data. It crashes on rb_obj_is_kind_of since it cannot be called on T_IMEMO objects. Unfortunately, we couldn't find a reproduction script, however we debugged the core dump and rb_ec_setup_exception is the only place where id_cause is assigned from errinfo without checking if it is a throw_data. ``` 0x0000556c5708e6dd in sigsegv (sig=11, info=0x7f301befa3f0, ctx=0x7f301befa2c0) at signal.c:964 0x00007f301d046420 in <signal handler called> () at /lib/x86_64-linux-gnu/libpthread.so.0 class_search_class_ancestor (c=139844586301760, cl=<optimized out>) at object.c:810 rb_obj_is_kind_of (obj=obj@entry=139839221734880, c=139844586301760) at object.c:861 0x0000556c56f2f00f in show_cause (errinfo=errinfo@entry=139838840645160, str=str@entry=139839221730520, opt=139839221730480, highlight=0, reverse=reverse@entry=0, backtrace_limit=backtrace_limit@entry=-1, shown_causes=0x7ffe9d1a2d68) at ./include/ruby/internal/special_consts.h:175 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114224] [Ruby master Bug#19773] Intermittent SIGSEGV during marking of Ripper parser
by eightbitraptor (Matthew Valentine-House) 18 Jul '23

18 Jul '23

Issue #19773 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Bug #19773: Intermittent SIGSEGV during marking of Ripper parser https://bugs.ruby-lang.org/issues/19773 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Intermittent SIGSEGV during marking of the Ripper parser. A failing build [can be seen on ci.rvm.jp (build 4643684)](http://ci.rvm.jp/results/trunk-random2@ruby-sp2-docker/4643684). Inside `ripper_s_allocate` first we use `TypedData_Make_Struct` to create a new `T_DATA` object in the Ruby heap. This allocates an empty struct (`struct ripper`). Further down the function we allocate another struct (`struct parser_params` during `rb_ruby_ripper_parser_allcoate`) and assign it to a pointer from the first struct, which is attached to the `T_DATA` object. `rb_ruby_ripper_parser_allocate` can trigger GC, as it uses `ruby_xcalloc`. But if GC is triggered at this point then the `parser_params` have not been allocated yet, and so the `struct ripper` contains a `NULL` reference. This causes a SEGV when we try and mark the `T_DATA` wrapper around the `struct ripper`. This can be replicated locally using the following test script ``` require "ripper" GC.stress = true Ripper.new ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:114137] [Ruby master Feature#19764] Introduce defp keyword for defining overloadable, pattern matched methods
by zeke (Zeke Gabrielse) 18 Jul '23

18 Jul '23

Issue #19764 has been reported by zeke (Zeke Gabrielse). ---------------------------------------- Feature #19764: Introduce defp keyword for defining overloadable, pattern matched methods https://bugs.ruby-lang.org/issues/19764 * Author: zeke (Zeke Gabrielse) * Status: Open * Priority: Normal ---------------------------------------- Pattern matching has become one of my favorite features of Ruby, if not my favorite. It changed the way I write and express my thoughts through clean, maintainable code. And I'd like to use it *more*. I propose a new keyword, `defp`, for defining a method which applies pattern matching to its arguments. ```ruby defp call(String => s unless s in /^[a-z]/) puts "string: #{s.inspect} (capitalized)" end defp call(String => s) puts "string: #{s.inspect}" end defp call(Hash(foo:, bar:) => h) puts "hash: #{h.inspect}" end defp call(**nil) puts "no keyword args" end call("Example") # => string: "Example" (capitalized) call("test") # => string: "test" call(foo: 1, bar: 2) # => hash: { :foo => 1, :bar => 2 } ``` Internally, this could be represented as the following `case..in` pseudocode: ```ruby def call(...) case ... in String => s unless s in /foo/ puts "string: #{s.inspect} (not foo)" in String => s puts "string: #{s.inspect}" in Hash(foo:, bar:) => h puts "hash: #{h.inspect}" in **nil puts "no keyword args" else raise NoMatchingMethod end end ``` As you could imagine, this could be used to refactor a lot of code, making the developer's intent much clearer. From [complex methods that use `case` statements](https://github.com/rails/rails/blob/593893c901f87b4ed205751f72d… for taking varied arguments (I'm sure all our code bases contain such `case` statements), to defining smaller, simpler methods that handle particular argument patterns. In addition, not only can this improve code quality, but it brings in method overloads, and it also adds a way to define more typing to the language -- something that RBS has tried to do, to mixed reactions -- but in a more Ruby-like way that Rubyists are already learning *and loving*. Thoughts? Original idea by Victor Shepelev: https://zverok.space/blog/2023-05-05-ruby-types.html Further discussion: https://news.ycombinator.com/item?id=35834351 -- https://bugs.ruby-lang.org/

5 4

[ruby-core:114049] [Ruby master Bug#19750] Ruby hangs when parsing Regexp with unterminated unicode "/\u{"
by tompng (tomoya ishida) 17 Jul '23

17 Jul '23

Issue #19750 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19750: Ruby hangs when parsing Regexp with unterminated unicode "/\u{" https://bugs.ruby-lang.org/issues/19750 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Ruby hangs up for a while and gets `failed to allocate memory (NoMemoryError)` ~~~shell $ ruby -e "/\\u{" ruby: failed to allocate memory (NoMemoryError) ~~~ ~~~ruby RubyVM::AbstractSyntaxTree.parse "/\\u{" # => SyntaxError RubyVM::AbstractSyntaxTree.parse "/\\u{\n" # => failed to allocate memory (NoMemoryError) ~~~ Found in https://github.com/ruby/irb/issues/620 -- https://bugs.ruby-lang.org/

4 4

[ruby-core:113260] [Ruby master Bug#19601] YJIT `try to mark T_NONE object` stemming from object shape transition on `self`
by alanwu (Alan Wu) 17 Jul '23

17 Jul '23

Issue #19601 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #19601: YJIT `try to mark T_NONE object` stemming from object shape transition on `self` https://bugs.ruby-lang.org/issues/19601 * Author: alanwu (Alan Wu) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: REQUIRED ---------------------------------------- We've identified a false collection bug with YJIT. Symptoms can range from `[BUG] try to mark T_NONE object` to SEGVs. Due to the bug requiring specific transient heap state to reproduce, it may be hard to identify by looking at the crash-site stack trace. `ruby --yjit-call-threshold=1` reproducer: ```ruby class RegressionTest def initialize @a = @b = @fourth_ivar_does_shape_transition = nil end def extender @first_extended_ivar = [:ok] end end test = RegressionTest.new # Fill up the transient heap, so rb_ensure_iv_list_size() # listens to GC.stress and yields to the GC. fill = Array.new(0x400000) GC.stress = true # Used to crash due to GC run in rb_ensure_iv_list_size() # not marking the newly allocated [:ok]. test.extender GC.start ``` I will post a patch shortly. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:112533] [Ruby master Bug#19455] Ruby 3.2: wrong Regexp encoding with non-ASCII comments
by janosch-x 17 Jul '23

17 Jul '23

Issue #19455 has been reported by janosch-x (Janosch Müller). ---------------------------------------- Bug #19455: Ruby 3.2: wrong Regexp encoding with non-ASCII comments https://bugs.ruby-lang.org/issues/19455 * Author: janosch-x (Janosch Müller) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- comments and comment groups don't trigger the correct `Regexp#encoding` on Ruby 3.2 anymore: ```ruby # ruby 3.1 /#a/x.encoding # => #<Encoding:US-ASCII> # OK /(?#a)/.encoding # => #<Encoding:US-ASCII> # OK /#ü/x.encoding # => #<Encoding:UTF-8> # OK /(?#ü)/.encoding # => #<Encoding:UTF-8> # OK # ruby 3.2 /#a/x.encoding # => #<Encoding:US-ASCII> # OK /(?#a)/.encoding # => #<Encoding:US-ASCII> # OK /#ü/x.encoding # => #<Encoding:US-ASCII> # BUG /(?#ü)/.encoding # => #<Encoding:US-ASCII> # BUG /#ü/x.inspect # => "/#\\xC3\\xBC/x" /(?#ü)/.inspect # => "/(?#\\xC3\\xBC)/" # bug is hidden if there are non-ascii chars outside comments /ü#ü/x.encoding # => #<Encoding:UTF-8> /ü(?#ü)/.encoding # => #<Encoding:UTF-8> ``` i think these changes might be the cause: https://github.com/ruby/ruby/commit/ec3542229b29ec93062e9d90e877ea29d3c1947… @jeremyevans0 JFYI -- https://bugs.ruby-lang.org/

6 6

[ruby-core:112922] [Ruby master Bug#19533] Behavior of ===/include? on a beginless/endless range (nil..nil) changed in ruby 3.2
by knu (Akinori MUSHA) 17 Jul '23

17 Jul '23

Issue #19533 has been reported by knu (Akinori MUSHA). ---------------------------------------- Bug #19533: Behavior of ===/include? on a beginless/endless range (nil..nil) changed in ruby 3.2 https://bugs.ruby-lang.org/issues/19533 * Author: knu (Akinori MUSHA) * Status: Open * Priority: Normal * Target version: 3.2 * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Starting from Ruby 2.7.0 a range `nil..nil` used to match any value until 3.2.0-preview3, but 3.2.0-rc1 started to reject it. ``` % docker run -it --rm rubylang/all-ruby ./all-ruby -e 'p( (nil..nil) === 1 )' (snip) ruby-2.6.10 false ruby-2.7.0-preview1 true ... ruby-3.2.0-preview3 true ruby-3.2.0-rc1 -e:1:in `===': cannot determine inclusion in beginless/endless ranges (TypeError) p( (nil..nil) === 1 ) ^ from -e:1:in `<main>' exit 1 ... ruby-3.2.1 -e:1:in `===': cannot determine inclusion in beginless/endless ranges (TypeError) p( (nil..nil) === 1 ) ^ from -e:1:in `<main>' ``` The previous behavior was so useful because when you have optional lower and upper bounds `lbound..rbound` would always work regardless of each end being nil or not. There is no mention of this in doc/NEWS/NEWS-3.2.0.md, so I'm afraid it was caused unintentionally while fixing other problems. ``` % docker run -it --rm rubylang/all-ruby ./all-ruby -e 'p( (nil..nil).cover?(1) )' (snip) ruby-2.6.10 false ruby-2.7.0-preview1 true ... ruby-3.2.1 true ``` This was pointed out by the following blog article (written in Japanese) as a "pitfall" that you need to work around when upgrading Ruby from 3.0/3.1 to 3.2. https://blog.studysapuri.jp/entry/2023/03/16/ujihisa-ruby32#endless-range%E… -- https://bugs.ruby-lang.org/

5 5

[ruby-core:113182] [Ruby master Bug#19589] Expecting stack overflow but crashing
by alpaca-tc (Hiroyuki Ishii) 17 Jul '23

17 Jul '23

Issue #19589 has been reported by alpaca-tc (Hiroyuki Ishii). ---------------------------------------- Bug #19589: Expecting stack overflow but crashing https://bugs.ruby-lang.org/issues/19589 * Author: alpaca-tc (Hiroyuki Ishii) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-04-11T00:54:20Z master 65e276096f) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following code expects stack overflow but crashes. The version it occurs in is newer than 3.2.0. ``` def expect_system_stack_error(h) h.each_key {} h.each_value { expect_system_stack_error(h) } end expect_system_stack_error(a: nil) ``` -- https://bugs.ruby-lang.org/

4 3