- ruby-core - ml.ruby-lang.org

[ruby-core:114826] [Ruby master Bug#19894] Memory leak in complemented method entries
by peterzhu2118 (Peter Zhu) 17 Oct '23

17 Oct '23

Issue #19894 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19894: Memory leak in complemented method entries https://bugs.ruby-lang.org/issues/19894 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: REQUIRED ---------------------------------------- GitHub pull request: https://github.com/ruby/ruby/pull/8481 When a copy of a complemented method entry is created, there are two issues: 1. IMEMO_FL_USER3 is not copied, so the complemented status is not copied over. 2. In rb_method_entry_clone we increment both alias_count and complemented_count. However, when we free the method entry in rb_method_definition_release, we only decrement one of the two counters, resulting in the rb_method_definition_t being leaked. The following script reproduces this issue: ```ruby 10.times do 20_000.times do $c = Class.new do def foo; end end $m = Module.new do refine $c do def foo; end end end Class.new do using $m def initialize o = $c.new o.method(:foo).unbind end end.new end puts `ps -o rss= -p #{$$}` end ``` Before this fix: ``` 17328 19248 21408 23296 25600 27408 29424 31520 33728 35664 ``` After this fix: ``` 16240 17680 18800 19744 20640 21840 22896 24336 25280 26096 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114992] [Ruby master Bug#19917] Segmentation fault or lost objects when using Ractor.select with moved exceptions
by toy (Ivan Kuchin) 15 Oct '23

15 Oct '23

Issue #19917 has been reported by toy (Ivan Kuchin). ---------------------------------------- Bug #19917: Segmentation fault or lost objects when using Ractor.select with moved exceptions https://bugs.ruby-lang.org/issues/19917 * Author: toy (Ivan Kuchin) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I stumbled upon loss of messages with exceptions or even Segmentation fault during transfer of exception from Ractor when moving the exception. In versions 3.0 and 3.1 I saw only loss of messages, in 3.2 also Segmentation fault. Loss of messages happens only when using `Ractor.select` (not `Ractor#take`), but segmentation fault happens also using `Ractor#take`. ```ruby 100.times do ractor_count = 10 ractors = Array.new(ractor_count) do |i| Ractor.new do begin raise 'foo' rescue => e # It is not possible to move exception without duplicating, but also # without it showing bactrace errors in 3.0 and is empty in 3.1+ e = Marshal.load(Marshal.dump(e)) Ractor.yield e, move: true end 'message got lost' end end ractor_count.times do ractor, result = Ractor.select(*ractors) p result ractors.delete(ractor) end end ``` In 3.0 and 3.1 will contain mix of `#<RuntimeError: foo>` and `"message got lost"`, in 3.2 may contain both and cause Segmentation fault. Excerpt of output (full output attached) from container running ruby:3.2 (`ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]`): ``` … #<RuntimeError: RuntimeError> "message got lost" #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> #<RuntimeError: RuntimeError> <internal:marshal>:34: [BUG] Segmentation fault at 0x0000000000000000 ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0004 p:0006 s:0019 e:000018 METHOD <internal:marshal>:34 c:0003 p:0022 s:0011 e:000010 RESCUE (irb):11 c:0002 p:0007 s:0007 e:000006 BLOCK (irb):6 [FINISH] c:0001 p:---- s:0003 e:000002 DUMMY [FINISH] -- Ruby level backtrace information ---------------------------------------- (irb):6:in `block (3 levels) in <top (required)>' (irb):11:in `rescue in block (3 levels) in <top (required)>' <internal:marshal>:34:in `load' -- Machine register context ------------------------------------------------ RIP: 0x00007fabc1d64490 RBP: 0x0000000000000000 RSP: 0x00007fabbc71b0c0 RAX: 0x0000000000000003 RBX: 0x00007fabbcebee88 RCX: 0x000000000000015f RDX: 0x0000556127db90f0 RDI: 0x000000000000015f RSI: 0x00007fabc2068628 R8: 0x0000000000000001 R9: 0x0000000000000020 R10: 0x00005561285633d0 R11: 0x0000000000000001 R12: 0x0000556127dbb710 R13: 0x00007fabc0601be0 R14: 0x0000000000000018 R15: 0x0000000000000000 EFL: 0x0000000000010206 … ``` ---Files-------------------------------- segmentation-fault.txt (27.8 KB) -- https://bugs.ruby-lang.org/

2 2

[ruby-core:115054] [Ruby master Feature#18822] Ruby lack a proper method to percent-encode strings for URIs (RFC 3986)
by noraj (Alexandre ZANNI) 14 Oct '23

14 Oct '23

Issue #18822 has been updated by noraj (Alexandre ZANNI). I just want to complete what was said before. URI.escape and URI.unescape were deprecated but they were replaced by URI::Parser.new.escape and URI::Parser.new.unescape that implements RFC 2396. In fact this is calling URI::RFC2396_Parser.escape and URI::RFC2396_Parser.unescape. But it's not useless since RFC 2396 was a Draft Standard and was obsoleted and updated by RFC 3986 which is an Internet Standard as CGI.escapeURIComponent and CGI.unescapeURIComponent implements RFC 3986. ---------------------------------------- Feature #18822: Ruby lack a proper method to percent-encode strings for URIs (RFC 3986) https://bugs.ruby-lang.org/issues/18822#change-104929 * Author: byroot (Jean Boussier) * Status: Closed * Priority: Normal ---------------------------------------- ### Context There are two fairly similar encoding methods that are often confused. `application/x-www-form-urlencoded` which is how form data is encoded, and "percent-encoding" as defined by [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986). AFAIK, the only way they differ is that "form encoding" escape space characters as `+`, and RFC 3986 escape them as `%20`. Most of the time it doesn't matter, but sometimes it does. ### Ruby form and URL escape methods - `URI.escape(" ") # => "%20"` but it was deprecated and removed (in 3.0 ?). - `ERB::Util.url_encode(" ") # => "%20"` but it's implemented with a `gsub` and isn't very performant. It's also awkward to have to reach for `ERB` - `CGI.escape(" ") # => "+"` - `URI.encode_www_form_component(" ") # => "+"` ### Unescape methods For unescaping, it's even more of a clear cut since `URI.unescape` was removed. So there's no available method that won't treat an unescaped `+` as simply `+`. e.g. in Javascript: `decodeURIComponent("foo+bar") #=> "foo+bar"`. If one were to use `CGI.unescape`, the string might be improperly decoded: `GI.unescape("foo+bar") #=> "foo bar"`. ### Other languages - Javascript `encodeURI` and `encodeURIComponent` use `%20`. - PHP has `urlencode` using `+` and `rawurlencode` using `%20`. - Python has `urllib.parse.quote` using `%20` and `urllib.parse.quote_plus` using `+`. ### Proposal Since `CGI` already have a very performant encoder for `application/x-www-form-urlencoded`, I think it would make sense that it would provide another method for RFC3986. I propose: - `CGI.url_encode(" ") # => "%20"` - Or `CGI.encode_url`. - Alias `CGI.escape` as `GCI.encode_www_form_component` - Clarify the documentation of `CGI.escape`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115053] [Ruby master Feature#16294] Make MatchData frozen and forbid MatchData.allocate
by Eregon (Benoit Daloze) 14 Oct '23

14 Oct '23

Issue #16294 has been updated by Eregon (Benoit Daloze). nobu (Nobuyoshi Nakada) wrote in #note-8: > At the latest dev-meeting, there was a suggestion that whereas `MatchData` cannot be dumped, `Regexp.allocate` might be used by serializer libraries. That does not seem used in practice. TruffleRuby already does not define `Regexp.allocate`, and there seems to be no compatibility issue with that except some specs. `Regexp.new` (or `eval`) is good enough for deserializing regexps. ---------------------------------------- Feature #16294: Make MatchData frozen and forbid MatchData.allocate https://bugs.ruby-lang.org/issues/16294#change-104928 * Author: Eregon (Benoit Daloze) * Status: Closed * Priority: Normal ---------------------------------------- Currently, `MatchData.allocate` is allowed, but almost every MatchData method called on it `raise TypeError, 'uninitialized Match'`. I think MatchData should be frozen, none of its internal fields are mutable and I don't see any use case for storing instance variables on it. Once frozen, we can implement MatchData#dup and #clone as just `return self`, and we don't need to check for the uninitialized case. And Marshal can have special treatment to create an initialized MatchData directly. My main motivation is looking at the code in TruffleRuby required to implement `MatchData.allocate` and check if it's initialized in so many places: https://github.com/oracle/truffleruby/pull/1792/files Thoughts? Anyone against? cc @alanwu -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115052] [Ruby master Misc#18984] Doc for Range#size for Float/Rational does not make sense
by kyanagi (Kouhei Yanagita) 14 Oct '23

14 Oct '23

Issue #18984 has been updated by kyanagi (Kouhei Yanagita). I created a PR: https://github.com/ruby/ruby/pull/8663 What should we do about `Range#count`? ``` (..0).count # => Infinity (nil..nil).count # => Infinity ``` ---------------------------------------- Misc #18984: Doc for Range#size for Float/Rational does not make sense https://bugs.ruby-lang.org/issues/18984#change-104927 * Author: masasakano (Masa Sakano) * Status: Open * Priority: Normal ---------------------------------------- When `Range` consists of any Numeric, according to [Official docs for Ruby-3.1.2](https://ruby-doc.org/core-3.1.2/Range.html#method-i-size), `Range#size` should, > Returns the count of elements in self if both begin and end values are numeric; Indeed, when `Range` consists of only `Integer`, this makes sense. However, when the begin value of `Range` is a `Float` or `Rational`, "*the count of elements*" does not make sense. Such Ranges are not iteratable, suggesting there are no such things as "elements": ```ruby (0.51..5.quo(2)).each{} # => TypeError ``` Yet, `Range#size` of such Ranges returns an Integer ```ruby (0.51..5.quo(2)).size # => 2 ``` It seems both begin and end values of a Range are rounded (`Numeric#round`) to the nearest Integer before `Range#size` is calculated. If this is the specification, I suggest it should be clearly stated in the [Official docs](https://ruby-doc.org/core-3.1.2/Range.html#method-i-size), avoiding the confusing expression "the count of elements", because "elements" are not unambiguously defined for Range of Float/Rational. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115051] [Ruby master Feature#16294] Make MatchData frozen and forbid MatchData.allocate
by nobu (Nobuyoshi Nakada) 13 Oct '23

13 Oct '23

Issue #16294 has been updated by nobu (Nobuyoshi Nakada). At the latest dev-meeting, there was a suggestion that whereas `MatchData` cannot be dumped, `Regexp.allocate` might be used by serializer libraries. ---------------------------------------- Feature #16294: Make MatchData frozen and forbid MatchData.allocate https://bugs.ruby-lang.org/issues/16294#change-104925 * Author: Eregon (Benoit Daloze) * Status: Closed * Priority: Normal ---------------------------------------- Currently, `MatchData.allocate` is allowed, but almost every MatchData method called on it `raise TypeError, 'uninitialized Match'`. I think MatchData should be frozen, none of its internal fields are mutable and I don't see any use case for storing instance variables on it. Once frozen, we can implement MatchData#dup and #clone as just `return self`, and we don't need to check for the uninitialized case. And Marshal can have special treatment to create an initialized MatchData directly. My main motivation is looking at the code in TruffleRuby required to implement `MatchData.allocate` and check if it's initialized in so many places: https://github.com/oracle/truffleruby/pull/1792/files Thoughts? Anyone against? cc @alanwu -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115035] [Ruby master Bug#19926] Range#size returns an incorrect result for ranges with a Rational endpoint
by kyanagi (Kouhei Yanagita) 13 Oct '23

13 Oct '23

Issue #19926 has been reported by kyanagi (Kouhei Yanagita). ---------------------------------------- Bug #19926: Range#size returns an incorrect result for ranges with a Rational endpoint https://bugs.ruby-lang.org/issues/19926 * Author: kyanagi (Kouhei Yanagita) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-12T17:32:45Z master 81399a5c46) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ``` % ~/tmp/ruby-master/bin/ruby -e '(1...3.1r).each { p _1 }' 1 2 3 ``` ``` % ~/tmp/ruby-master/bin/ruby -e 'p (1...3.1r).size' 2 ``` ``` % ~/tmp/ruby-master/bin/ruby -e 'p (1...3.1).size' 3 ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:115049] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by byroot (Jean Boussier) 13 Oct '23

13 Oct '23

Issue #18915 has been updated by byroot (Jean Boussier). > I wouldn't expect a test framework to rescue OutOfMemoryError for example, Well Minitest does rescue Exception: https://github.com/minitest/minitest/blob/6719ad8d8d49779669083f5029ea9a042… Pretty sure RSpec does as well. It's one of the rare justifiable cases for doing so. ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-104923 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (1.57 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115048] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by tenderlovemaking (Aaron Patterson) 13 Oct '23

13 Oct '23

Issue #18915 has been updated by tenderlovemaking (Aaron Patterson). byroot (Jean Boussier) wrote in #note-10: > > since NotImplementedError doesn't inherit from StandardError I kind of wish there was a new exception class > > Could you elaborate? For this use case I think not inheriting from `StandardError` is better, as it's not something you'd want to be rescued broadly. It is something I would like rescued broadly. For example if I'm test driving a concrete implementation, the test framework needs to specifically rescue `NotImplementedError` in order to report the error. I wouldn't expect a test framework to rescue OutOfMemoryError for example, but definitely rescue NotImplementedError. ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-104922 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (1.57 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115046] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by byroot (Jean Boussier) 13 Oct '23

13 Oct '23

Issue #18915 has been updated by byroot (Jean Boussier). > since NotImplementedError doesn't inherit from StandardError I kind of wish there was a new exception class Could you elaborate? For this use case I think not inheriting from `StandardError` is better, as it's not something you'd want to be rescued broadly. ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-104920 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (1.57 KB) -- https://bugs.ruby-lang.org/

1 0