[ruby-core:118686] [Ruby master Bug#20653] Memory leak in String#start_with? when regexp times out
Issue #20653 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20653: Memory leak in String#start_with? when regexp times out https://bugs.ruby-lang.org/issues/20653 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11247 This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error. This fixes a memory leak in String#start_with when the regexp times out. For example: ```ruby regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do str.start_with?(regex) rescue end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 33216 51936 71152 81728 97152 103248 120384 133392 133520 133616 ``` After: ``` 14912 15376 15824 15824 16128 16128 16144 16144 16160 16160 ``` -- https://bugs.ruby-lang.org/
Issue #20653 has been updated by peterzhu2118 (Peter Zhu). Ruby 3.3 backport PR: https://github.com/ruby/ruby/pull/11255 ---------------------------------------- Bug #20653: Memory leak in String#start_with? when regexp times out https://bugs.ruby-lang.org/issues/20653#change-109233 * Author: peterzhu2118 (Peter Zhu) * Status: Closed * Backport: 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11247 This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error. This fixes a memory leak in String#start_with when the regexp times out. For example: ```ruby regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do str.start_with?(regex) rescue end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 33216 51936 71152 81728 97152 103248 120384 133392 133520 133616 ``` After: ``` 14912 15376 15824 15824 16128 16128 16144 16144 16160 16160 ``` -- https://bugs.ruby-lang.org/
Issue #20653 has been updated by k0kubun (Takashi Kokubun). Backport changed from 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: REQUIRED to 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: DONE ruby_3_3 commit:ce565cd4b851977bf37a470bee54e441bb60486d. ---------------------------------------- Bug #20653: Memory leak in String#start_with? when regexp times out https://bugs.ruby-lang.org/issues/20653#change-109578 * Author: peterzhu2118 (Peter Zhu) * Status: Closed * Backport: 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: DONE ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11247 This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error. This fixes a memory leak in String#start_with when the regexp times out. For example: ```ruby regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do str.start_with?(regex) rescue end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 33216 51936 71152 81728 97152 103248 120384 133392 133520 133616 ``` After: ``` 14912 15376 15824 15824 16128 16128 16144 16144 16160 16160 ``` -- https://bugs.ruby-lang.org/
participants (2)
-
k0kubun (Takashi Kokubun) -
peterzhu2118 (Peter Zhu)