June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118128] [Ruby master Bug#20516] The version of rexml in ruby 3.3.2 has not been updated since 3.2.6.
by naitoh (Jun NAITOH) 21 Jun '24

21 Jun '24

Issue #20516 has been reported by naitoh (Jun NAITOH). ---------------------------------------- Bug #20516: The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. https://bugs.ruby-lang.org/issues/20516 * Author: naitoh (Jun NAITOH) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. This is still a DoS vulnerable version. https://www.ruby-lang.org/en/news/2024/05/16/dos-rexml-cve-2024-35176/ ``` $ ruby -v ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] $ gem list rexml *** LOCAL GEMS *** rexml (3.2.6) ``` -- https://bugs.ruby-lang.org/

5 12

[ruby-core:117746] [Ruby master Bug#20462] Native threads are no longer reused
by tenderlovemaking (Aaron Patterson) 20 Jun '24

20 Jun '24

Issue #20462 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20462: Native threads are no longer reused https://bugs.ruby-lang.org/issues/20462 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ruby used to reuse native threads in order to amortize the cost of making a pthread. For example this program: ```ruby ntids = 1000.times.map { Thread.new { Thread.current.native_thread_id }.value } p ntids.uniq.length ``` With Ruby 3.2.0, this would return 1. With Ruby 3.3.x, it returns 1000. It means we cannot amortize the cost of a pthread for short lived threads. I was able to bisect this to commit be1bbd5b7d40ad863ab35097765d3754726bbd54. But the change is big so I don't know how to fix it. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:118343] [Ruby master Bug#20584] Multiple irbrc files loaded
by aosho235 (Aoyama Shotaro) 18 Jun '24

18 Jun '24

Issue #20584 has been reported by aosho235 (Aoyama Shotaro). ---------------------------------------- Bug #20584: Multiple irbrc files loaded https://bugs.ruby-lang.org/issues/20584 * Author: aosho235 (Aoyama Shotaro) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Both ~/.irbrc and ./.irbrc are loaded on startup of irb. Only the first found one should be loaded. 3.2.2 is so. > The path to the configuration file is the first found among: https://docs.ruby-lang.org/en/3.3/IRB.html ``` $ ruby --version ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117498] [Ruby master Feature#20425] Optimize forwarding callers and callees
by tenderlovemaking (Aaron Patterson) 18 Jun '24

18 Jun '24

Issue #20425 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #20425: Optimize forwarding callers and callees https://bugs.ruby-lang.org/issues/20425 * Author: tenderlovemaking (Aaron Patterson) * Status: Open ---------------------------------------- [This PR](https://github.com/ruby/ruby/pull/10510) optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` ## How does it work? This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. ## Why? I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. -- https://bugs.ruby-lang.org/

2 13

[ruby-core:118331] [Ruby master Misc#20583] https://ruby-doc.org/current redirects to Ruby 3.2.2
by Earlopain (A S) 16 Jun '24

16 Jun '24

Issue #20583 has been reported by Earlopain (A S). ---------------------------------------- Misc #20583: https://ruby-doc.org/current redirects to Ruby 3.2.2 https://bugs.ruby-lang.org/issues/20583 * Author: Earlopain (A S) * Status: Open ---------------------------------------- Not sure if this is documented somewhere. I was playing around with the url looking for something like this but its a bit out-of-date. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:118328] [Ruby master Bug#20582] Directory not empty
by wzh4464 (Zihan Wu) 16 Jun '24

16 Jun '24

Issue #20582 has been reported by wzh4464 (Zihan Wu). ---------------------------------------- Bug #20582: Directory not empty https://bugs.ruby-lang.org/issues/20582 * Author: wzh4464 (Zihan Wu) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- #### Summary When running `brew upgrade ruby`, several directories encountered the error: `Error: Directory not empty @ dir_s_rmdir /private/tmp/xxx`. After inspection, it was found that these directories contained `.DS_Store` files. #### Reproduce Process 1. **System Details:** - **OS Version:** macOS 14.5 (23F79) - **Chip:** Apple Silicon M1 2. **Steps to Reproduce:** 1. Ensure Homebrew is installed and up to date. 2. Run the following command to upgrade Ruby: ```sh brew upgrade ruby ``` 3. Observe the error during the upgrade process. #### Result of Reproduce Process During the upgrade process, several directories failed to be removed due to the presence of `.DS_Store` files. The specific error encountered was: ``` Error: Directory not empty @ dir_s_rmdir /private/tmp/homebrew-unpack20240616-1699-7vglx8/ruby/3.3.3/lib/ruby/gems/3.3.0 ``` #### Expected Result The expected result is a successful upgrade of Ruby without encountering any errors related to directory removal. The upgrade process should handle the presence of hidden files like `.DS_Store` and proceed without interruptions. #### Additional Information - **Ruby Version:** 3.3.3 - **Homebrew Version:** Homebrew 4.3.5-118-g766b9b8 - **Logs and Output:** The error logs indicating the presence of `.DS_Store` files in the temporary directories. #### Steps Taken to Resolve - Manually checked and confirmed the presence of `.DS_Store` files in the directories mentioned in the error logs. - Removed `.DS_Store` files manually and retried the upgrade process. #### Suggested Fix To prevent such errors in the future, the upgrade script should handle the presence of hidden files like `.DS_Store` and ensure they do not block the directory removal process. Adding a step to ignore or remove these files during cleanup would be beneficial. #### Maintainer Assignment Since this issue is encountered on macOS, it may be relevant to assign it to the current maintainer for macOS-related issues. #### Thank you! Thank you for your attention to this issue. Looking forward to a resolution. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117918] [Ruby master Bug#20494] Non-default directories are not searched when checking for a gmp header
by lish82 (Hiroki Katagiri) 15 Jun '24

15 Jun '24

Issue #20494 has been reported by lish82 (Hiroki Katagiri). ---------------------------------------- Bug #20494: Non-default directories are not searched when checking for a gmp header https://bugs.ruby-lang.org/issues/20494 * Author: lish82 (Hiroki Katagiri) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is my very first issue submission I would be happy to know if I missed anything --- I found that the ruby installed with `ruby-build` is not linked to the gmp library Environment: - OS: macOS Sonoma 14.5 - Directory that gmp is installed: /opt/homebrew (= installed via homebrew) I believe it was able to link to gmp before, so I tried to investigate the cause of the problem As a result, I found the followings: - During the execution of `./configure`, `checking for gmp.h...` is showing `no` - Specifying `--with-gmp-dir=/opt/homebrew` does not work - It has been occurring since 3.2.3 - When I built it on linux, it was successfully linked to gmp - gmp was installed in the default directory (`/usr/include`) - Revert the following changes and the problem goes away - https://github.com/ruby/ruby/pull/8449 Based on these observations, I believe the issue is that the directories specified with `--with-opt-dir` or `--with-gmp-dir` are not being included in the search path when checking for the gmp header I was able to solve the problem by changing the build process as follows ```patch diff --git a/configure.ac b/configure.ac index 18b4247..a440de6 100644 --- a/configure.ac +++ b/configure.ac @@ -1363,7 +1363,11 @@ AS_CASE("$target_cpu", [x64|x86_64|i[3-6]86*], [ RUBY_UNIVERSAL_CHECK_HEADER([x86_64, i386], x86intrin.h) AS_IF([test "x$with_gmp" != xno], - [AC_CHECK_HEADERS(gmp.h) + [original_CPPFLAGS="${CPPFLAGS}" + CPPFLAGS="${INCFLAGS} ${CPPFLAGS}" + AC_CHECK_HEADERS(gmp.h) + CPPFLAGS="${original_CPPFLAGS}" + unset original_CPPFLAGS AS_IF([test "x$ac_cv_header_gmp_h" != xno], AC_SEARCH_LIBS([__gmpz_init], [gmp], [AC_DEFINE(HAVE_LIBGMP, 1)]))]) ``` However, I am not familiar with ruby's build process or autotools, so I'm not sure if this is the correct approach --- I would appreciate it if someone could consider resolving this issue -- https://bugs.ruby-lang.org/

7 9

[ruby-core:118110] [Ruby master Bug#20515] --with-gmp is not working - GMP support won't be built
by sorah (Sorah Fukumori) 15 Jun '24

15 Jun '24

Issue #20515 has been reported by sorah (Sorah Fukumori). ---------------------------------------- Bug #20515: --with-gmp is not working - GMP support won't be built https://bugs.ruby-lang.org/issues/20515 * Author: sorah (Sorah Fukumori) * Status: Open * Assignee: sorah (Sorah Fukumori) * Target version: 3.3 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- gmp integration has been broken since https://bugs.ruby-lang.org/issues/20494 18eaf0be905e3e251423b42d6f4e56b7cae1bc3b including just released 3.3.2 due to missing HAVE_GMP_H. -- https://bugs.ruby-lang.org/

3 7

[ruby-core:117905] [Ruby master Bug#20493] Segfault on rb_io_getline_fast
by josegomezr (Jose Gomez) 15 Jun '24

15 Jun '24

Issue #20493 has been reported by josegomezr (Jose Gomez). ---------------------------------------- Bug #20493: Segfault on rb_io_getline_fast https://bugs.ruby-lang.org/issues/20493 * Author: josegomezr (Jose Gomez) * Status: Open * ruby -v: 3.3.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- We've spotted a consistent segfault when running bundle install with `--jobs 4` When running: `bundle install -j 4` we'd get a Segfault at: ``` /usr/lib64/ruby/3.3.0/rubygems/ext/builder.rb:93: [BUG] Segmentation fault at 0x0000000000000014 ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux-gnu] ``` Full [log is available here][0]. I could not find a shorter reproducer besides using bundler with `--jobs 4` or `--jobs 8`. Here's a sample command to trigger the behavior (it creates the Gemfile and calls bundler) [1]. We installed all debug symbols and narrowed down the location of the segfault to `rb_io_getline_fast` in io.c At [line 4001][3] `str` is `T_NONE`, which makes further usage down the line in [`io_enc_str`][4] raise a null pointer dereference. With the notes from [extension.rdoc - Appendix E. RB_GC_GUARD to protect from premature GC][8] I've prepared a patched ruby 3.3.1 package that does not segfault. It's on [OBS Project home:josegomezr:branches:ruby/ruby3.3][6]. Adding a `RB_GC_GUARD` on `rb_io_getline_fast` @ `io.c:4004` just before the return ```diff --- ruby3.3.orig/ruby-3.3.1/io.c +++ ruby3.3/ruby-3.3.1/io.c @@ -4004,6 +4004,7 @@ rb_io_getline_fast(rb_io_t *fptr, rb_enc ENC_CODERANGE_SET(str, cr); fptr->lineno++; + RB_GC_GUARD(str); return str; } ``` Fixes the segfault in our tests. `bundle` finish the installation and the image is built. I've set up a project in OBS to provide reproduceables. - [ruby3.3.1 package][5]. - [ruby3.3.1 base image with enough dependencies to reproduce][7] with [the reproducer script][1]. And the corresponding container is exported in the `containers-patched` repository. Here I leave the docker images generated by OBS: - 3.3.1 [without patches, segfaults.] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers/b… ``` - 3.3.1 [with patch, does not fail] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers-p… ``` [0]: https://gist.github.com/josegomezr/441c271cc731b0ec57213cb98743a699 [1]: https://gist.github.com/josegomezr/e17129bf2df33f3bea60e84a616a8322 [2]: https://gist.github.com/josegomezr/6f81878c979af334efee59b8f2225e58 [3]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4001 [4]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4003 [5]: https://build.opensuse.org/package/show/devel:languages:ruby/ruby3.3 [6]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby/ruby3… [7]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby:conta… [8]: https://github.com/ruby/ruby/blob/master/doc/extension.rdoc#label-Appendix+… -- https://bugs.ruby-lang.org/

4 13

[ruby-core:116182] [Ruby master Bug#20180] Inconsistent evaluation of `**{}` depending on position in array
by ozydingo (Andrew Schwartz) 15 Jun '24

15 Jun '24

Issue #20180 has been reported by ozydingo (Andrew Schwartz). ---------------------------------------- Bug #20180: Inconsistent evaluation of `**{}` depending on position in array https://bugs.ruby-lang.org/issues/20180 * Author: ozydingo (Andrew Schwartz) * Status: Open * Priority: Normal * ruby -v: 3.3 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduced on ruby:3.3 docker container The evaluation of `**{}` differs if it appears alone (evaluates as empty / no content) in an array vs after another element (evaluates as an empty Hash). ```rb args = []; kwargs = {} [*args] # => [] [**kwargs] # => [] [*args, **kwargs] # => [{}] [*args] + [**kwargs] == [*args, **kwargs] => false ``` I claim this violates the Principle of Least Surprise. I will admit that beyond a thin example I will give below, I am struggling to come up with a more convincing pragmatic reason that this should be addressed, but due to how surprising it is and the bugs it cause our team I wanted to submit it for tracking. This may be related to https://bugs.ruby-lang.org/issues/20064?tab=notes though the issues are distinct. Specifically, in my use case, I am writing a class responsible for adapting arguments in one form to another (json object to method args and vice versa). My tests broken when adding support for keyword args due to expectations of no args: ``` # RSpec expect(foo).to have_received(:bar).with([]) # This now need `with([], {})` ``` Again, this is a bit thin as by itself this isn't problematic as I can add the `{}`. However, this does require that my test knows more about the implementation that it should. That is, my implementation might be ``` if kwargs.present? call(*args, **kwargs) else call(*args) end ``` This change does not change the behavior of the class, but will break the test. I therefore think this behavior of `**kwargs` is problematic. -- https://bugs.ruby-lang.org/

6 7