April 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:115196] [Ruby master Feature#19979] Allow methods to declare that they don't accept a block via `&nil`
by ufuk (Ufuk Kayserilioglu) 24 Jul '24

24 Jul '24

Issue #19979 has been reported by ufuk (Ufuk Kayserilioglu). ---------------------------------------- Feature #19979: Allow methods to declare that they don't accept a block via `&nil` https://bugs.ruby-lang.org/issues/19979 * Author: ufuk (Ufuk Kayserilioglu) * Status: Open * Priority: Normal ---------------------------------------- ## Abstract This feature proposes new syntax to allow methods to explicitly declare that they don't accept blocks, and makes passing of a block to such methods an error. ## Background In #15554, it was proposed to automatically detect methods that do not use the block passed to them, and to error if a block was passed to such methods. As far as I can tell, it was later on closed since #10499 solved a large part of the problem. That proposal has, as part of the dev meeting discussion, a proposal from @matz to allow methods to use `&nil` to explicitly declare that they don't accept a block. At the time, the proposal was trying to solve a bigger problem, so this sub-proposal was never considered seriously. However, notes in the proposal say: > It is explicit, but it is tough to add this `&nil` parameter declaration to all of methods (do you want to add it to `def []=(i, e, &nil)`?). (I agree `&nil` is valuable on some situations) This proposal extracts that sub-proposal to make this a new language feature. ## Proposal In Ruby, it is always valid for the caller to pass a block to a method call, even if the callee is not expecting a block to be passed. This leads to subtle user errors, where the author of some code assumes a method call uses a block, but the block passed to the method call is silently ignored. The proposal is to introduce `&nil` at method declaration sites to mean "This method does not accept a block". This is symmetric to the ability to pass `&nil` at call sites to mean "I am not passing a block to this method call", which is sometimes useful when making `super` calls (since blocks are always implicitly passed). Explicitly, the proposal is to make the following behaviour be a part of Ruby: ```ruby def find(item = nil, &nil) # some implementation that doesn't call `yield` or `block_given?` end find { |i| i == 42 } # => ArgumentError: passing block to the method `find' that does not accept a block. ``` ## Implementation I assume the implementation would be a grammar change to make `&nil` valid at method declaration sites, as well as raising an `ArgumentError` for methods that are called with a block but are declared with `&nil`. ## Evaluation Since I don't have an implementation, I can't make a proper evaluation of the feature proposal. However, I would expect the language changes to be minimal with no runtime costs for methods that don't use the `&nil` syntax. ## Discussion This proposal has much smaller scope than #15554 so that the Ruby language can start giving library authors the ability to explicitly mark their methods as not accepting a block. This is fully backward compatible, since it is an opt-in behaviour and not an opt-out one. Future directions after this feature proposal could be a way to signal to the VM that any method in a file that doesn't explicitly use `yield`/`block_given?` or explicitly declared a block parameter should be treated as not accepting a block. This can be done via some kind of pragma similar to `frozen_string_literal`, or through other means. However, such future directions are beyond the scope of this proposal. ## Summary Adding the ability for methods to declare that they don't accept a block will make writing code against such methods safer and more resilient, and will prevent silently ignored behaviour that is often hard to catch or troubleshoot. -- https://bugs.ruby-lang.org/

9 20

[ruby-core:111448] [Ruby master Bug#19266] URI::Generic should use URI::RFC3986_PARSER instead of URI::DEFAULT_PARSER
by gareth (Gareth Adams) 21 Jul '24

21 Jul '24

Issue #19266 has been reported by gareth (Gareth Adams). ---------------------------------------- Bug #19266: URI::Generic should use URI::RFC3986_PARSER instead of URI::DEFAULT_PARSER https://bugs.ruby-lang.org/issues/19266 * Author: gareth (Gareth Adams) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- In June 2014, [`uri/common` was updated][1] to introduce a RFC3986-compliant parser (`URI::RFC3986_PARSER`) as an alternative to the previous RFC2396 parser, and common methods like `URI()` were updated to use that new parser by default. The only methods in `common` not updated were [`URI.extract` and `URI.regexp`][2] which are marked as obsolete. (The old parser was kept in the `DEFAULT_PARSER` constant despite it not being the default for those methods, presumably for backward compatibility.) However, similar [methods called on `URI::Generic`][3] were never updated to use this new parser. This means that methods like `URI::Generic.build` fail when given input that succeeds normally, and this also affects subclasses like URI::HTTP: ``` $ pry -r uri -r uri/common -r uri/generic [1] pry(main)> URI::Generic.build(host: "underscore_host.example") URI::InvalidComponentError: bad component(expected host component): underscore_host.example from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host' [2] pry(main)> URI::HTTP.build(host: "underscore_host.example") URI::InvalidComponentError: bad component(expected host component): underscore_host.example from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host' [3] pry(main)> URI("http://underscore_host.example") => #<URI::HTTP http://underscore_host.example> ``` `URI::Generic.new` allows a configurable `parser` positional argument to override the class' default parser, but other factory methods like `.build` don't allow this override. Arguably this doesn't cause problems because at least in the case above, the URI can be built with the polymorphic constructor, but having the option to build URIs from explicit named parts is useful, and leaving the outdated functionality in the `Generic` class is ambiguous. It's possible that the whole Generic class and its subclasses aren't intended to be used directly how I'm intending here, but there's nothing I could see that suggested this is the case. I'm not aware of the entire list of differences between RFC2396 and RFC3986. The relevant difference here is that in RFC2396 an individual segment of a host ([`domainlabel`s][4]) could only be `alphanum | alphanum *( alphanum | "-" ) alphanum`, whereas RFC3986 allows [hostnames][5] to include any of `ALPHA / DIGIT / "-" / "." / "_" / "~"`. It's possible that other differences might cause issues for developers, but since this has gone over 8 years without anyone else caring about this, this is definitely not especially urgent. [1]: https://github.com/ruby/ruby/commit/bb83f32dc3e0424d25fa4e55d8ff32b061320e41 [2]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/… [3]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/… [4]: https://www.rfc-editor.org/rfc/rfc2396#section-3.2.2 [5]: https://www.rfc-editor.org/rfc/rfc3986#page-13 -- https://bugs.ruby-lang.org/

4 8

[ruby-core:113352] [Ruby master Bug#19619] Endless method definition with parameters for an instance given as numbered parameters doesn't work
by okuramasafumi (Masafumi OKURA) 20 Jul '24

20 Jul '24

Issue #19619 has been reported by okuramasafumi (Masafumi OKURA). ---------------------------------------- Bug #19619: Endless method definition with parameters for an instance given as numbered parameters doesn't work https://bugs.ruby-lang.org/issues/19619 * Author: okuramasafumi (Masafumi OKURA) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This works: ```ruby class Foo def bar(baz) = 'foo' end p Foo.new.bar('baz') ``` This also works: ```ruby o = Object.new o.tap { |obj| def obj.bar(baz) = 'foo' } p o.bar('baz') ``` Even this works: ```ruby o = Object.new o.tap { def _1.bar = 'foo' } p o.bar ``` But this DOESN'T work: ```ruby o = Object.new o.tap { def _1.bar(baz) = 'foo' } p o.bar('baz') ``` So, when we define a method with parameters on an instance given as numbered parameters, it doesn't work. If we change one condition (definition way, parameter existence, and so on), it works. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:114792] [Ruby master Bug#19887] RUBYOPT doesn't work unless starting with `-`
by nobu (Nobuyoshi Nakada) 20 Jul '24

20 Jul '24

Issue #19887 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19887: RUBYOPT doesn't work unless starting with `-` https://bugs.ruby-lang.org/issues/19887 * Author: nobu (Nobuyoshi Nakada) * Status: Assigned * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- It was working till 1.8. ```sh-session $ for ruby in ruby{1.{8..9},2.{0..7},3.{0..3}}; do RUBYOPT=w $ruby -e 'puts "'$ruby': #{$VERBOSE}"'; done ruby1.8: true ruby1.9: false ruby2.0: false ruby2.1: false ruby2.2: false ruby2.3: false ruby2.4: false ruby2.5: false ruby2.6: false ruby2.7: false ruby3.0: false ruby3.1: false ruby3.2: false ruby3.3: false ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:115728] [Ruby master Bug#20062] Numbered parameters are broken in Ruby 3.3-dev
by osyo (manga osyo) 20 Jul '24

20 Jul '24

Issue #20062 has been reported by osyo (manga osyo). ---------------------------------------- Bug #20062: Numbered parameters are broken in Ruby 3.3-dev https://bugs.ruby-lang.org/issues/20062 * Author: osyo (manga osyo) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-13T13:16:11Z master f390c51b15) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Define Numbered parameters and a method in one block and then reference Numbered parameters in another block, you will get a SyntaxError in Ruby 3.3. This is not a problem in Ruby 3.2. ```ruby 'a'.tap do p _1 def f1; end end 'a'.tap do p _1 end __END__ output: Ruby 3.2.2 => "a" "a" Ruby 3.3.0dev(2023-12-13T13:16:11Z master f390c51b15) => /path/to/test.rb: /path/to/test.rb:8: numbered parameter is already used in (SyntaxError) /path/to/test.rb:2: outer block here ``` I noticed this while checking the operation of https://bugs.ruby-lang.org/issues/18966. I haven't checked, but https://bugs.ruby-lang.org/issues/19619 may be related. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:114258] [Ruby master Bug#19781] Stack consistency error with tailcall and YJIT enabled
by k0kubun (Takashi Kokubun) 20 Jul '24

20 Jul '24

Issue #19781 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #19781: Stack consistency error with tailcall and YJIT enabled https://bugs.ruby-lang.org/issues/19781 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal * Assignee: yjit * Backport: 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- ```rb src = <<-EOS def apply_one_and_two(&block) [1, p(1)] end def add_one_and_two apply_one_and_two(&:+) end EOS RubyVM::InstructionSequence.new( "proc {|_|_.class_eval {#{src}}}", __FILE__, __FILE__, 1, tailcall_optimization: true, trace_instruction: false, ).eval[self.singleton_class] def entry add_one_and_two end entry ``` This script causes "Stack consistency error" with `ruby --yjit-call-threshold=1` on any YJIT-enabled build (3.1, 3.2, and master). -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115549] [Ruby master Bug#20030] `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string.
by tompng (tomoya ishida) 15 Jul '24

15 Jul '24

Issue #20030 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20030: `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string. https://bugs.ruby-lang.org/issues/20030 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-11-30T16:23:25Z master d048bae96b) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ~~~ruby Ripper.tokenize '"\\C-あ"' # or Ripper.tokenize "\"\\C-\u3042\"" # => ["\"", "\x81", "\x82", "\""] ~~~ I expect all tokens to be valid_encoding if the source string is valid_encoding. This is causing IRB crash when typing `"\C-あ"`. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117200] [Ruby master Bug#20342] Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load
by nobu (Nobuyoshi Nakada) 15 Jul '24

15 Jul '24

Issue #20342 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20342: Top level `public`, `private` and `ruby2_keywords` do not work in wrapped load https://bugs.ruby-lang.org/issues/20342 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- With this file: ```ruby # load.rb public def f = :ok ``` It is OK when `require`d. ```sh-session $ ruby -r ./load.rb -e 'p f' :ok ``` Simple `load` is OK too. ```sh-session $ ruby -e 'load ARGV[0]; p f' load.rb :ok ``` Wrapped `load` fails. ```sh-session $ ruby -e 'load ARGV[0], true' load.rb load.rb:1:in 'public': undefined method 'f' for class 'Object' (NameError) public def f = :ok ^^^^^^ from load.rb:1:in '<top (required)>' from -e:1:in 'Kernel#load' from -e:1:in '<main>' ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:117697] [Ruby master Bug#20453] Pointer being freed was not allocated in Regexp timeout
by dodecadaniel (Daniel Colson) 15 Jul '24

15 Jul '24

Issue #20453 has been reported by dodecadaniel (Daniel Colson). ---------------------------------------- Bug #20453: Pointer being freed was not allocated in Regexp timeout https://bugs.ruby-lang.org/issues/20453 * Author: dodecadaniel (Daniel Colson) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- https://bugs.ruby-lang.org/issues/20228 frees `stk_base` to avoid a memory leak, but `stk_base` is sometimes stack allocated ([see `xalloca`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1177-L1181)). So the free only works if the regex stack grows enough that it needs to double ([see `xmalloc` and `xrealloc` in `stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1210-L1249). Reproduction: ```ruby Regexp.timeout = 0.001 /^(a*)x$/ =~ "a" * 1000000 + "x"' ``` I'll open a PR shortly. https://bugs.ruby-lang.org/issues/20228 was backported to 3.3.1, so this bug affects that version as well. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116950] [Ruby master Bug#20304] Memory leak when setting Encoding.default_internal
by MaxLap (Maxime Lapointe) 14 Jul '24

14 Jul '24

Issue #20304 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20304: Memory leak when setting Encoding.default_internal https://bugs.ruby-lang.org/issues/20304 * Author: MaxLap (Maxime Lapointe) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Problem is present in Ruby 3.2.2, 3.2.3, 3.3.0. Didn't check before. Put this in a file: ``` 10.times do 100000.times do Encoding.default_internal = nil end puts `ps -o rss= -p #{$$}`.to_i end ``` Result: ``` $ ruby local.rb 27044 30212 33116 36284 39452 42620 45788 48956 51860 55028 ``` -- https://bugs.ruby-lang.org/

4 4