November 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:115197] [Ruby master Misc#19980] Is the Ruby 3.3 ABI frozen?
by mdalessio (Mike Dalessio) 13 Dec '23

13 Dec '23

Issue #19980 has been reported by mdalessio (Mike Dalessio). ---------------------------------------- Misc #19980: Is the Ruby 3.3 ABI frozen? https://bugs.ruby-lang.org/issues/19980 * Author: mdalessio (Mike Dalessio) * Status: Open * Priority: Normal ---------------------------------------- I'm a co-maintainer of [rake-compiler-dock](https://github.com/rake-compiler/rake-compiler-dock) which is used to build precompiled native packages for some gems, including nokogiri, sqlite3, grpc, re2, and ruby-magic. In the past, precompiled native gems generally released support for a new version of Ruby days or weeks after Ruby's release (for example, [Nokogiri v1.14.0 released on 2023-01-12](https://nokogiri.org/CHANGELOG.html)) In some cases, the lack of native gem support has slowed user adoption of the new version of Ruby. This year, I'd like to see native gems release Ruby 3.3 support before the final version is released. This will encourage testing of Ruby 3.3 previews and allow users to immediately upgrade to Ruby 3.3 final when it is released. To do that, I would like to cut a release of the rake-compiler-dock gem (and its [build containers](https://github.com/rake-compiler/rake-compiler-dock/pkgs/contai…) that supports Ruby 3.3 as soon as possible (I already have a feature branch mostly working based on 3.3.0_preview1 at https://github.com/rake-compiler/rake-compiler-dock/pull/105) Is the Ruby 3.3 ABI frozen now? If I build a native gem against Ruby 3.3.0_preview2, is there any reason to believe that it wouldn't work with Ruby 3.3 final when it is released? Thank you for any guidance you can provide. -- https://bugs.ruby-lang.org/

3 6

[ruby-core:112119] [Ruby master Bug#19392] Endless method vs and/or
by zverok (Victor Shepelev) 08 Dec '23

08 Dec '23

Issue #19392 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #19392: Endless method vs and/or https://bugs.ruby-lang.org/issues/19392 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- [Discovered](https://twitter.com/lucianghinda/status/1617783952353406977) by Lucian Ghinda: ```ruby def test = puts("foo") and puts("bar") # prints "bar" immediately test # prints "foo" ``` It seems that it is a parser error, right?.. ```ruby RubyVM::AbstractSyntaxTree.parse('def test = puts("foo") and puts("bar")') # => # (SCOPE@1:0-1:38 # tbl: [] # args: nil # body: # (AND@1:0-1:38 # (DEFN@1:0-1:22 # mid: :test # body: # (SCOPE@1:0-1:22 # tbl: [] # args: # (ARGS@1:0-1:8 pre_num: 0 pre_init: nil opt: nil first_post: nil post_num: 0 post_init: nil rest: nil kw: nil kwrest: nil block: nil) # body: (FCALL@1:11-1:22 :puts (LIST@1:16-1:21 (STR@1:16-1:21 "foo") nil)))) # (FCALL@1:27-1:38 :puts (LIST@1:32-1:37 (STR@1:32-1:37 "bar") nil)))) ``` E.g. it is parsed as ```ruby (def test = puts("foo")) and (puts("bar")) ``` ...which is hardly intentional or have any practical use. The rightly parsed code in this case _can_ have practical use, like ```ruby def write(data) = File.write(@filename, data) == data.size or raise "Something went wrong" ``` -- https://bugs.ruby-lang.org/

12 19

[ruby-core:115430] [Ruby master Bug#20012] Fix keyword splat passing as regular argument
by jeremyevans0 (Jeremy Evans) 07 Dec '23

07 Dec '23

Issue #20012 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20012: Fix keyword splat passing as regular argument https://bugs.ruby-lang.org/issues/20012 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-11-12 master 60e19a0b5f) [x86_64-openbsd] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Since Ruby 3.0, Ruby has passed a keyword splat as a regular argument in the case of a call to a Ruby method where the method does not accept keyword arguments, if the method call does not contain an argument splat: ```ruby def self.f(obj) obj end def self.fs(*obj) obj[0] end h = {a: 1} f(**h).equal?(h) # Before: true; After: false fs(**h).equal?(h) # Before: true; After: false a = [] f(*a, **h).equal?(h) # Before and After: false fs(*a, **h).equal?(h) # Before and After: false ``` The fact that the behavior differs when passing an empty argument splat makes it obvious that something is not working the way it is intended. The fact that the hash is copied for C methods also makes it obvious. Ruby 2 always copied the keyword splat hash, and I think that is the expected behavior in Ruby 3. This bug is because of a missed check in setup_parameters_complex. If the keyword splat passed is not mutable, then it points to an existing object and not a new object, and therefore it must be copied. I did not bisect to find the commit that introduced the problem, but this seems likely to be something I introduced during keyword argument separation. I'm surprised that this has not been filed as a bug sooner. Maybe it just rarely comes up in practice, and I only discovered it recently. Possibly, other developers that may have discovered this may have decided against submitting this as an issue, as we have had specs for two years that give the impression that the broken behavior was expected. I've submitted a pull request to fix this: https://github.com/ruby/ruby/pull/8970 -- https://bugs.ruby-lang.org/

4 5

[ruby-core:114722] [Ruby master Bug#19877] Non intuitive behavior of syntax only applied to literal value
by tompng (tomoya ishida) 07 Dec '23

07 Dec '23

Issue #19877 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19877: Non intuitive behavior of syntax only applied to literal value https://bugs.ruby-lang.org/issues/19877 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-09-08T23:08:32Z master b635a66e95) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Non intuitive behavior of syntax only applied to literal value Some ruby syntax is only applied to literal value. ~~~ruby def 1.foo; end # receiver is a literal, it is Syntax Error /(?<a>)/ =~ s # receiver is regexp literal, it will assign to local variable if cond1..cond2; end # range-like syntax appears in condition, it is flipflop ~~~ If it is wrapped with parenthesis, the behavior seems not intuitive for me, and YARP parses it differently. ~~~ruby def (1).foo; end # Syntax Error def ((1;1)).foo; end # Syntax Error def ((;1)).foo; end # Syntax OK def ((1+1;1)).foo; end # Syntax OK def ((%s();1)).foo; end # Syntax Error def ((%w();1)).foo; end # Syntax OK def ("#{42}").foo; end # Syntax Error def (:"#{42}").foo; end # Syntax OK (/(?<a>)/) =~ s # assigns to a (;/(?<a>)/) =~ s # does not assigns (%s();/(?<a>)/) =~ s # assigns to a (%w();/(?<a>)/) =~ s # does not assigns (1; (2; 3; (4; /(?<a>)/))) =~ s # assigns to a (1+1; /(?<a>)/) =~ s # does not assign if ((cond1..cond2)); end # flipflop if (; cond1..cond2); end # range if (1; cond1..cond2); end # flipflop if (%s(); cond1..cond2); end # flipflop if (%w(); cond1..cond2); end # range if (1; (2; (3; 4; cond1..cond2))); end # flipflop if (1+1; cond1..cond2); end # range ~~~ I expect YARP and parse.y parses same. I expect all parenthesis-wrapped result same. I think it is simple and intuitive if parenthesis-wrapped code always behaves different from non-wrapped code because there are more complex variation like this ~~~ruby def (class A; 1; end).foo; end (break; /?<a>/) =~ s class A; /?<a>/; end =~ s ~~~ -- https://bugs.ruby-lang.org/

4 5

[ruby-core:114251] [Ruby master Bug#19779] `eval "return"` at top level raises `LocalJumpError`
by andrykonchin (Andrew Konchin) 07 Dec '23

07 Dec '23

Issue #19779 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19779: `eval "return"` at top level raises `LocalJumpError` https://bugs.ruby-lang.org/issues/19779 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Wondering whether it's intentional behaviour. It seems it's the only difference I've noticed between calling `return` with and without `eval`: ``` ruby -e 'return' ruby -e 'eval "return"' (eval):1:in `<main>': unexpected return (LocalJumpError) from -e:1:in `eval' from -e:1:in `<main>' ``` -- https://bugs.ruby-lang.org/

4 3

[ruby-core:115462] [Ruby master Feature#20018] Get the error codes as Socket::EAI_XXX when getaddrinfo and getnameinfo fail
by shioimm (Misaki Shioi) 07 Dec '23

07 Dec '23

Issue #20018 has been reported by shioimm (Misaki Shioi). ---------------------------------------- Feature #20018: Get the error codes as Socket::EAI_XXX when getaddrinfo and getnameinfo fail https://bugs.ruby-lang.org/issues/20018 * Author: shioimm (Misaki Shioi) * Status: Open * Priority: Normal ---------------------------------------- Feature for implementing Happy Eyeballs version 2 (RFC 8305) in `Socket.tcp`. ### Probrem I would like to implement Happy Eyeballs version 2 (RFC 8305) in `Socket.tcp`. In HEv2, `EAI_ADDRFAMILY` and `EAI_AGAIN` should be ignored when `getaddrinfoo` returns them in hostname resolution. However, there is currently no way to ask `SocketError` for errors returned by `getaddrinfo` ### Proposal I opened a PR to get errors when `getaddrinfo` and `getnameinfo` returns them. https://github.com/ruby/ruby/pull/9018 Todo this, - I added `Socket::ResolutionError` as a subclass of `SocketError` - I added an attribute of `Socket::ResolutionError` to get error code via `#error_code - I replaced `SocketError` (`rb_eSocket`) with `Socket::ResolutionError (`rb_eResolutionError`) in `rsock_raise_socket_error` because this function is only called when `getaddrinfo` and `getnameinfo` fail - I renamed `rsock_raise_socket_error` to `rock_raise_resolution_error` for the same reason In addition, `Socket::ResolutionError` is a subclass of `SocketError`, so it does not affect `rescue SocketError` in source codes. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115428] [Ruby master Feature#20011] Reduce implicit array allocations on caller side of method calling
by jeremyevans0 (Jeremy Evans) 07 Dec '23

07 Dec '23

Issue #20011 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Feature #20011: Reduce implicit array allocations on caller side of method calling https://bugs.ruby-lang.org/issues/20011 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal ---------------------------------------- I would like to use the peephole optimizer to eliminate caller-side array allocations for the following cases, by switching `splatarray true` to `splatarray false`: ```ruby f(1, *a) f(1, *a, &lvar) f(1, *a, &@ivar) f(*a, **lvar) f(*a, **@ivar) f(*a, **lvar, &lvar) f(*a, **@ivar, &@ivar) f(*a, kw: 1) f(*a, kw:1, &lvar) f(*a, kw:1, &@ivar) ``` In terms of safety, currently, `f(*a, &lvar)` and `f(*a, &@ivar)` both avoid array allocations (`splatarray false`), and all of the above are as safe as those in terms of safety. Note that since at least Ruby 1.8, in pathlogical cases, `lvar.to_proc` can modify `a`, which results in behavior contrary to expected evaluation order: ```ruby ary = [1,2] kwd = Object.new kwd.define_singleton_method(:to_proc) {ary << 4; lambda{}} p(*ary, &kwd) # 4 included in output ``` I think that for both the current `f(*a, &lvar)` and `f(*a, &@ivar)` cases and all of the above cases, the benefit of avoiding the allocation is higher than the costs, considering that only pathologic cases fail. If we do not want the above optimization, for consistency, we should remove the optimization of `f(*a, &lvar)` and `f(*a, &@ivar)` (or update the optimization so that it is only used if `lvar` or `@ivar` is already a proc), which will slow Ruby down. I have submitted a pull request for these changes: https://github.com/ruby/ruby/pull/8853 To make sure these cases are worth optimizing, I did some analysis: For ruby -e '' (just loads rubygems): ``` 5 : f(1, *a) 2 : f(1, *a, &lvar) | f(1, *a, &@ivar) ``` Current stdlib: ``` 139 : f(1, *a) 3 : f(1, *a, &lvar) | f(1, *a, &@ivar) 4 : f(*a, **lvar) | f(*a, **@ivar) ``` Rails master: ``` 77 : f(1, *a) 5 : f(1, *a, &lvar) | f(1, *a, &@ivar) 26 : f(*a, **lvar) | f(*a, **@ivar) 5 : f(*a, kw: 1) ``` minitest 5.20 (bundled gem): ``` 4 : f(1, *a) 1 : f(1, *a, &lvar) | f(1, *a, &@ivar) 2 : f(*a, **lvar) | f(*a, **@ivar) 2 : f(*a, **lvar, &lvar) | f(*a, **@ivar, &@ivar) ``` capybara 3.39.2: ``` 15 : f(1, *a) 2 : f(1, *a, &lvar) | f(1, *a, &@ivar) 19 : f(*a, **lvar) | f(*a, **@ivar) 4 : f(*a, **lvar, &lvar) | f(*a, **@ivar, &@ivar) ``` This shows that all cases optimized except `f(*a, kw:1, &lvar)` and `f(*a, kw:1, &@ivar)` are used in common gems. Those cases could be removed if people think they are not worth optimizing. Code backing the above analysis can be found at https://github.com/ruby/ruby/pull/8853#issuecomment-1817656139 -- https://bugs.ruby-lang.org/

3 7

[ruby-core:115388] [Ruby master Feature#20005] Add C API to return symbols of native extensions resolved from features
by tagomoris (Satoshi Tagomori) 07 Dec '23

07 Dec '23

Issue #20005 has been reported by tagomoris (Satoshi Tagomori). ---------------------------------------- Feature #20005: Add C API to return symbols of native extensions resolved from features https://bugs.ruby-lang.org/issues/20005 * Author: tagomoris (Satoshi Tagomori) * Status: Open * Priority: Normal ---------------------------------------- I want an API to resolve symbols of other native extensions by a feature name and a symbol name (just like `dlsym`). (`rb_dln_resolve_symbol` is an example function name of this feature in the example below) ```c // "a_client_open" is a function defined in "/..../a_client.so", that should be loaded in Ruby beforehand. // in the extension to be "a.so" VALUE (*a_client_open)(VALUE); void Init_a(void) { a_client_open = (VALUE(*)(VALUE))rb_dln_resolve_symbol("a_client", "a_client_open"); // the return value may be NULL if the symbols is not found } ``` This API is to replace direct reference of external symbols. Currently, native extensions have to just call the function of other extensions directly on the assumption of: * The dependency native extension is already loaded * The symbol is defined correctly in the dependency (the version of dependency is correct&expected) Otherwise, it crashes. This API provides a way for developers to raise exceptions if the resolved symbol is `NULL`, and these exceptions can display meaningful messages and instructions to the users of extensions. -- https://bugs.ruby-lang.org/

4 13

[ruby-core:115304] [Ruby master Feature#19993] Optionally Free all memory at exit
by HParker (Adam Hess) 06 Dec '23

06 Dec '23

Issue #19993 has been reported by HParker (Adam Hess). ---------------------------------------- Feature #19993: Optionally Free all memory at exit https://bugs.ruby-lang.org/issues/19993 * Author: HParker (Adam Hess) * Status: Open * Priority: Normal ---------------------------------------- Add a runtime option allowing Ruby to optionally free all memory at shutdown. # why Today, memory sanitizers are difficult to use with Ruby, since not all memory is freed at shutdown. it is difficult to detect memory leaks or errors in Ruby or in Ruby C extensions when these tools are not available. While implementing this feature, we were able to identify and fix a number of memory leaks and errors. https://github.com/ruby/ruby/pull/8556 https://github.com/ruby/ruby/pull/8512 https://github.com/ruby/ruby/pull/8503 https://github.com/ruby/ruby/pull/8487 https://github.com/ruby/ruby/pull/8452 https://github.com/ruby/ruby/pull/8501 https://github.com/ruby/ruby/pull/8481 https://github.com/ruby/ruby/pull/8555 This shows that having access to tools like this can make finding and fixing memory bugs easier. # current progress Today we can allow ruby developers to enable freeing memory at shutdown via the `free-at-shutdown` runtime option. PR: https://github.com/ruby/ruby/pull/8868 running, without `--free-on-shutdown`: ``` valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes -- ./miniruby basictest/test.rb ``` ``` ==573270== LEAK SUMMARY: ==573270== definitely lost: 658,324 bytes in 5,387 blocks ==573270== indirectly lost: 955,708 bytes in 11,957 blocks ==573270== possibly lost: 2,071,096 bytes in 12 blocks ==573270== still reachable: 161,881 bytes in 275 blocks ==573270== suppressed: 0 bytes in 0 blocks ==573270== Reachable blocks (those to which a pointer was found) are not shown. ==573270== To see them, rerun with: --leak-check=full --show-leak-kinds=all ``` with `--free-on-shutdown` ``` valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes -- ./miniruby --free-on-shutdown basictest/test.rb ``` ``` ==573306== HEAP SUMMARY: ==573306== in use at exit: 0 bytes in 0 blocks ==573306== total heap usage: 43,643 allocs, 43,643 frees, 29,222,534 bytes allocated ==573306== ==573306== All heap blocks were freed -- no leaks are possible ``` # future plans * Continue improving memory safety * add `valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes -- ./miniruby --free-on-shutdown basictest/test.rb` to CI * Allow C extensions to do a "optional destruct for memory safety" so they can leverage the same memory sanitizer tools. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:115333] [Ruby master Misc#19997] DevMeeting-2023-11-30
by mame (Yusuke Endoh) 06 Dec '23

06 Dec '23

Issue #19997 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #19997: DevMeeting-2023-11-30 https://bugs.ruby-lang.org/issues/19997 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2023/11/30 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2023/11/27. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

10 14