[ruby-core:121239] [Ruby master Bug#21170] Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash

Issue #21170 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170 * Author: jhawthorn (John Hawthorn) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/

Issue #21170 has been updated by alanwu (Alan Wu). Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN to 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED Started with the 3.2 series. ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170#change-112187 * Author: jhawthorn (John Hawthorn) * Status: Open * Backport: 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/

Issue #21170 has been updated by jhawthorn (John Hawthorn). Fixed in bf10e99bf9d9676b12ab470b8970e4f12c9b4392 Backports: 3.2: https://github.com/ruby/ruby/pull/12860 3.3: https://github.com/ruby/ruby/pull/12861 3.4: https://github.com/ruby/ruby/pull/12862 ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170#change-112194 * Author: jhawthorn (John Hawthorn) * Status: Open * Backport: 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/

Issue #21170 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED to 3.1: DONTNEED, 3.2: REQUIRED, 3.3: DONE, 3.4: REQUIRED Merged into ruby_3_3 at ecb9f7ef372c70c3e4fa81a5002533814a94aa86 ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170#change-112218 * Author: jhawthorn (John Hawthorn) * Status: Closed * Backport: 3.1: DONTNEED, 3.2: REQUIRED, 3.3: DONE, 3.4: REQUIRED ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/

Issue #21170 has been updated by hsbt (Hiroshi SHIBATA). Backport changed from 3.1: DONTNEED, 3.2: REQUIRED, 3.3: DONE, 3.4: REQUIRED to 3.1: DONTNEED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED Merged `ruby_3_2` at 5e69c1a96264f827eaa611d317c3dab0497f084a ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170#change-112228 * Author: jhawthorn (John Hawthorn) * Status: Closed * Backport: 3.1: DONTNEED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/

Issue #21170 has been updated by tenderlovemaking (Aaron Patterson). Backport changed from 3.1: DONTNEED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED to 3.1: DONTNEED, 3.2: DONE, 3.3: DONE, 3.4: DONE Backported in 698741d73a0e6c998fb47297ce7b129366b4835b ---------------------------------------- Bug #21170: Corrupted Hash (bad VALUE and missing entry) when -1 returned from .hash https://bugs.ruby-lang.org/issues/21170#change-112388 * Author: jhawthorn (John Hawthorn) * Status: Closed * Backport: 3.1: DONTNEED, 3.2: DONE, 3.3: DONE, 3.4: DONE ---------------------------------------- ``` ruby class MyObj attr_reader :hash def initialize(val) = @hash = val end values = 0.downto(-16).to_a hash = {} values.each do |val| hash[MyObj.new(val)] = val end p hash.values # => [0, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15, false, -16] ``` --- st_table reserves -1 as a special hash value to indicate that an entry has been deleted. So that that's a valid value to be returned from the hash function, do_hash replaces -1 with 0 so that it is not mistaken for the sentinel. Previously, when upgrading an AR table to an ST table, rb_st_add_direct_with_hash was used which did not perform the same conversion, this could lead to a hash in a broken state where one if its entries which was supposed to exist being marked as a tombstone. The hash could then become further corrupted when the ST table required resizing as the falsely tombstoned entry would be skipped but it would be counted in num entries, leading to an uninitialized entry at index 15. https://github.com/ruby/ruby/pull/12852 -- https://bugs.ruby-lang.org/
participants (5)
-
alanwu (Alan Wu)
-
hsbt (Hiroshi SHIBATA)
-
jhawthorn (John Hawthorn)
-
nagachika (Tomoyuki Chikanaga)
-
tenderlovemaking (Aaron Patterson)