[ruby-core:126480] [Ruby Bug#22259] Arrays sharding a buffer segfault when concatenating with each other
Issue #22259 has been reported by chucke (Tiago Cardoso). ---------------------------------------- Bug #22259: Arrays sharding a buffer segfault when concatenating with each other https://bugs.ruby-lang.org/issues/22259 * Author: chucke (Tiago Cardoso) * Status: Open * ruby -v: 4.0.6 * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- A succession of crashes was observed on a particular workload from our app, involving recursive set operations, after upgrading it to ruby 4.0.6 . LLM was used to build a reproduction (shared in the description of the PR linked below), after which we used it as a harness to get to the actual culprit and fix it. The crash is due to a particular behaviour of how (btw, TIL, so correct my explanation if necessary) ruby arrays share internal buffers, for optimal memory usage. `Array#dup` creates a separate array object, but may share (under certain conditions, i.e. does not embed) the internal C buffer, where objects are stored, with it. Certain modification operations, like `Array#dup`, don't forcefully "copy on write", and instead bump/dec an index on the shared buffer, as so to determine the slice of the original shared buffer it points to. The issue happens with how other operations work, such as `Array#concat`, which is, internally, under certain conditions, copying the full shared buffer to the destination array buffer, without regards of which slice it is about. This is easily demonstrable by the first reproduction in the PR linked below, where the last `concat` operation adds a "phantom" element to the destination array. that element is, at that point, gibberish, malloc-uninitialized data, which causes the VM to crash, depending on what it does with the address found there. This bug has been there for a while, and can be reproduced (with different faulty outcomes) since ruby 3.3 at least (didn't try further back). PR with fix here: https://github.com/ruby/ruby/pull/18472 -- https://bugs.ruby-lang.org/
Issue #22259 has been updated by mame (Yusuke Endoh). Confirmed and reproduced. Thanks for the report. While looking into it I found that the same line causes another bug. ```ruby a = [:A, :B, :C, :D, :E] b = a[1, 4] #=> [:B, :C, :D, :E] a[0, 0] = b p a # expected: [:B, :C, :D, :E, :A, :B, :C, :D, :E] # actual: [:B, :C, :D, :A, :A, :B, :C, :D, :E] ``` It silently returns a wrong element. It happens whenever the source starts after `beg`, and dates back to 2329d8b0de4. I think the simple fix is to stop guessing from the pointer and let the callers say whether the source and the destination are the same array. I opened a PR: https://github.com/ruby/ruby/pull/18481 ---------------------------------------- Bug #22259: Arrays sharding a buffer segfault when concatenating with each other https://bugs.ruby-lang.org/issues/22259#change-118643 * Author: chucke (Tiago Cardoso) * Status: Open * ruby -v: 4.0.6 * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- A succession of crashes was observed on a particular workload from our app, involving recursive set operations, after upgrading it to ruby 4.0.6 . LLM was used to build a reproduction (shared in the description of the PR linked below), after which we used it as a harness to get to the actual culprit and fix it. The crash is due to a particular behaviour of how (btw, TIL, so correct my explanation if necessary) ruby arrays share internal buffers, for optimal memory usage. `Array#dup` creates a separate array object, but may share (under certain conditions, i.e. does not embed) the internal C buffer, where objects are stored, with it. Certain modification operations, like `Array#dup`, don't forcefully "copy on write", and instead bump/dec an index on the shared buffer, as so to determine the slice of the original shared buffer it points to. The issue happens with how other operations work, such as `Array#concat`, which is, internally, under certain conditions, copying the full shared buffer to the destination array buffer, without regards of which slice it is about. This is easily demonstrable by the first reproduction in the PR linked below, where the last `concat` operation adds a "phantom" element to the destination array. that element is, at that point, gibberish, malloc-uninitialized data, which causes the VM to crash, depending on what it does with the address found there. This bug has been there for a while, and can be reproduced (with different faulty outcomes) since ruby 3.3 at least (didn't try further back). PR with fix here: https://github.com/ruby/ruby/pull/18472 -- https://bugs.ruby-lang.org/
Issue #22259 has been updated by chucke (Tiago Cardoso). Thx for the patch, much cleaner than mine. Indeed I thought there must have been a cleaner fix, but I admit I was out of my depth. ---------------------------------------- Bug #22259: Arrays sharding a buffer segfault when concatenating with each other https://bugs.ruby-lang.org/issues/22259#change-118645 * Author: chucke (Tiago Cardoso) * Status: Open * ruby -v: 4.0.6 * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- A succession of crashes was observed on a particular workload from our app, involving recursive set operations, after upgrading it to ruby 4.0.6 . LLM was used to build a reproduction (shared in the description of the PR linked below), after which we used it as a harness to get to the actual culprit and fix it. The crash is due to a particular behaviour of how (btw, TIL, so correct my explanation if necessary) ruby arrays share internal buffers, for optimal memory usage. `Array#dup` creates a separate array object, but may share (under certain conditions, i.e. does not embed) the internal C buffer, where objects are stored, with it. Certain modification operations, like `Array#dup`, don't forcefully "copy on write", and instead bump/dec an index on the shared buffer, as so to determine the slice of the original shared buffer it points to. The issue happens with how other operations work, such as `Array#concat`, which is, internally, under certain conditions, copying the full shared buffer to the destination array buffer, without regards of which slice it is about. This is easily demonstrable by the first reproduction in the PR linked below, where the last `concat` operation adds a "phantom" element to the destination array. that element is, at that point, gibberish, malloc-uninitialized data, which causes the VM to crash, depending on what it does with the address found there. This bug has been there for a while, and can be reproduced (with different faulty outcomes) since ruby 3.3 at least (didn't try further back). PR with fix here: https://github.com/ruby/ruby/pull/18472 -- https://bugs.ruby-lang.org/
participants (2)
-
chucke (Tiago Cardoso) -
mame (Yusuke Endoh)