June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118376] [Ruby master Feature#18762] Add an Array#undigits that compliments Integer#digits
by inopinatus (Joshua GOODALL) 23 Jun '24

23 Jun '24

Issue #18762 has been updated by inopinatus (Joshua GOODALL). I'll speak up for this feature. Whilst writing a base58 encoder/decoder today, I was surprised by the lack of an inverse to Integer#digits. In my benchmarking the reduce/inject variant is considerably faster than the sum variant, especially when we get into bignum territory. However, for consistency with other conversions, `Integer(array, base)` should work too, as `Integer(string, base)` currently does. The latter is already special-cased, so that could be extended to handle an array even if the method above doesn't end up as `Array#to_i`. ---------------------------------------- Feature #18762: Add an Array#undigits that compliments Integer#digits https://bugs.ruby-lang.org/issues/18762#change-108888 * Author: shan (Shannon Skipper) * Status: Open ---------------------------------------- I've found Integer#digits convenient and useful but several times have needed to go from the place-value notation back to an Integer after manipulation and wished there was a complimentary Array#undigits. ``` ruby class Array def undigits(base = 10) each_with_index.sum do |digit, exponent| digit * base**exponent end end end 42.digits.undigits #=> 42 42.digits(16).undigits(16) #=> 42 ``` Below is my stab at a Ruby implementation with behavior mirroring Integer#digits. ``` ruby class Array def undigits(base = 10) base_int = base.to_int raise TypeError, "wrong argument type #{base_int.class} (expected Integer)" unless base_int.is_a?(Integer) raise ArgumentError, 'negative radix' if base_int.negative? raise ArgumentError, "invalid radix #{base_int}" if base_int < 2 each_with_index.sum do |digit, exponent| raise MathDomainError, 'out of domain' if digit.negative? digit * base_int**exponent end end end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118372] [Ruby master Bug#20591] macOS head - all are using OpenSSL 1.1.1?
by MSP-Greg (Greg L) 22 Jun '24

22 Jun '24

Issue #20591 has been reported by MSP-Greg (Greg L). ---------------------------------------- Bug #20591: macOS head - all are using OpenSSL 1.1.1? https://bugs.ruby-lang.org/issues/20591 * Author: MSP-Greg (Greg L) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Currently, the macos.yml file used for GHA CI is using OpenSSL 1.1.1, and the same for the builds done in https://github.com/ruby/ruby-dev-builder. But, all the macOS builds of 3.1 thru 3.3 are using OpenSSL 3.x? See the 'Ruby Info' step in any of the macOS builds at https://github.com/MSP-Greg/github-actions-ruby-info/actions/runs/9583386691. I checked in a fork branch, and the macos.yml CI passes with OpenSSL@3. Additionally, no macos-14 jobs are being run, just macos-12 & macos-13. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118368] [Ruby master Feature#20590] Ensure `fork` isn't called when `raddrinfo`'s background thread is in `getaddrinfo`
by byroot (Jean Boussier) 21 Jun '24

21 Jun '24

Issue #20590 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20590: Ensure `fork` isn't called when `raddrinfo`'s background thread is in `getaddrinfo` https://bugs.ruby-lang.org/issues/20590 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- NB: Opening this as a feature because I don't have any clear bug report or repro script, but in a way this is a bug. ### Context For better or for worse, fork(2) remains the primary provider of parallelism in Ruby programs, and will likely continue to be for the foreseeable future. Even though it's frowned upon in many circles, and a lot of literature will simply state that only async-signal safe APIs are safe to use after `fork()`, in practice, most APIs work well as long as you are careful about not to fork while another thread is holding a pthread mutex, and the general advice is simply to not start any thread before calling `fork(2)`. This became much harder (if not impossible) to ensure in Ruby 3.3 following [Feature #19965]. Every call to `Addrinfo.getaddrinfo` now starts a native thread that will call `getaddrinfo(3)`. And unless I'm not reading the code correctly, that thread may even be left running if the call is interrupted. This is particularly problematic because, at least in the `glibc` implementation, `getaddrinfo(3)` do acquire a mutex. So if a fork happens while this mutex is held, the resulting child will be corrupted, and any call to `getaddrinfo(3)` in the child will deadlock. This is a fairly well-known fork-safety problem ([just one example](https://emptysqua.re/blog/getaddrinfo-deadlock/)). I don't have a reproducer to demonstrate this bug, but I hear several reports of fork issues happening to people upgrading to Ruby 3.3 that seem related or could be explained by this issue. ### Proposal I think we could reduce the impact of this problem by locking around `fork(2)` and `getaddrinfo(3) with a read-write lock. `Process.fork` would acquire the lock in write mode, and `getaddrinfo` would acquire it in read mode. The obvious downside of course is that an interrupted `addrinfo` call may take a very long time to timeout and release the lock, delaying the `Process.fork` call for a while, and that's far from ideal, but I don't have any better idea. I implemented a proof of concept at: https://github.com/ruby/ruby/pull/10864 cc @mame @ko1 -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118316] [Ruby master Feature#20580] Pipe Operator accepting lamnda
by martinosis (Martin Chabot) 21 Jun '24

21 Jun '24

Issue #20580 has been reported by martinosis (Martin Chabot). ---------------------------------------- Feature #20580: Pipe Operator accepting lamnda https://bugs.ruby-lang.org/issues/20580 * Author: martinosis (Martin Chabot) * Status: Open ---------------------------------------- I think that the pipe operator was not reflecting the actual pipe operator in functional programming language. In Elixir, Elm, F# etc. The pipe operator takes a value and applies it on the lambda at the right of the operator. Example ``` add_one = -> a { a + 1 } add_two = -> a { a + 2 } 2 |> add_one |> add_two == 5 ``` In combination with the >> operator, some interesting thing can be done. ``` 2 |> add_one >> add_two ``` you can refactor to ``` add_tree = add_one >> add_two 2 |> add_tree ``` I currently use the `then` method on Object to do the equivalent. However this takes more characters and you need to put the closing bracket at the end of the line. ``` 2.then(&add_one >> add_tree) ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118128] [Ruby master Bug#20516] The version of rexml in ruby 3.3.2 has not been updated since 3.2.6.
by naitoh (Jun NAITOH) 21 Jun '24

21 Jun '24

Issue #20516 has been reported by naitoh (Jun NAITOH). ---------------------------------------- Bug #20516: The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. https://bugs.ruby-lang.org/issues/20516 * Author: naitoh (Jun NAITOH) * Status: Open * ruby -v: ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The version of rexml in ruby 3.3.2 has not been updated since 3.2.6. This is still a DoS vulnerable version. https://www.ruby-lang.org/en/news/2024/05/16/dos-rexml-cve-2024-35176/ ``` $ ruby -v ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin22] $ gem list rexml *** LOCAL GEMS *** rexml (3.2.6) ``` -- https://bugs.ruby-lang.org/

5 12

[ruby-core:117746] [Ruby master Bug#20462] Native threads are no longer reused
by tenderlovemaking (Aaron Patterson) 20 Jun '24

20 Jun '24

Issue #20462 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20462: Native threads are no longer reused https://bugs.ruby-lang.org/issues/20462 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ruby used to reuse native threads in order to amortize the cost of making a pthread. For example this program: ```ruby ntids = 1000.times.map { Thread.new { Thread.current.native_thread_id }.value } p ntids.uniq.length ``` With Ruby 3.2.0, this would return 1. With Ruby 3.3.x, it returns 1000. It means we cannot amortize the cost of a pthread for short lived threads. I was able to bisect this to commit be1bbd5b7d40ad863ab35097765d3754726bbd54. But the change is big so I don't know how to fix it. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:118343] [Ruby master Bug#20584] Multiple irbrc files loaded
by aosho235 (Aoyama Shotaro) 18 Jun '24

18 Jun '24

Issue #20584 has been reported by aosho235 (Aoyama Shotaro). ---------------------------------------- Bug #20584: Multiple irbrc files loaded https://bugs.ruby-lang.org/issues/20584 * Author: aosho235 (Aoyama Shotaro) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Both ~/.irbrc and ./.irbrc are loaded on startup of irb. Only the first found one should be loaded. 3.2.2 is so. > The path to the configuration file is the first found among: https://docs.ruby-lang.org/en/3.3/IRB.html ``` $ ruby --version ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117498] [Ruby master Feature#20425] Optimize forwarding callers and callees
by tenderlovemaking (Aaron Patterson) 18 Jun '24

18 Jun '24

Issue #20425 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #20425: Optimize forwarding callers and callees https://bugs.ruby-lang.org/issues/20425 * Author: tenderlovemaking (Aaron Patterson) * Status: Open ---------------------------------------- [This PR](https://github.com/ruby/ruby/pull/10510) optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` ## How does it work? This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. ## Why? I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. -- https://bugs.ruby-lang.org/

2 13

[ruby-core:118331] [Ruby master Misc#20583] https://ruby-doc.org/current redirects to Ruby 3.2.2
by Earlopain (A S) 16 Jun '24

16 Jun '24

Issue #20583 has been reported by Earlopain (A S). ---------------------------------------- Misc #20583: https://ruby-doc.org/current redirects to Ruby 3.2.2 https://bugs.ruby-lang.org/issues/20583 * Author: Earlopain (A S) * Status: Open ---------------------------------------- Not sure if this is documented somewhere. I was playing around with the url looking for something like this but its a bit out-of-date. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:118328] [Ruby master Bug#20582] Directory not empty
by wzh4464 (Zihan Wu) 16 Jun '24

16 Jun '24

Issue #20582 has been reported by wzh4464 (Zihan Wu). ---------------------------------------- Bug #20582: Directory not empty https://bugs.ruby-lang.org/issues/20582 * Author: wzh4464 (Zihan Wu) * Status: Open * ruby -v: ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- #### Summary When running `brew upgrade ruby`, several directories encountered the error: `Error: Directory not empty @ dir_s_rmdir /private/tmp/xxx`. After inspection, it was found that these directories contained `.DS_Store` files. #### Reproduce Process 1. **System Details:** - **OS Version:** macOS 14.5 (23F79) - **Chip:** Apple Silicon M1 2. **Steps to Reproduce:** 1. Ensure Homebrew is installed and up to date. 2. Run the following command to upgrade Ruby: ```sh brew upgrade ruby ``` 3. Observe the error during the upgrade process. #### Result of Reproduce Process During the upgrade process, several directories failed to be removed due to the presence of `.DS_Store` files. The specific error encountered was: ``` Error: Directory not empty @ dir_s_rmdir /private/tmp/homebrew-unpack20240616-1699-7vglx8/ruby/3.3.3/lib/ruby/gems/3.3.0 ``` #### Expected Result The expected result is a successful upgrade of Ruby without encountering any errors related to directory removal. The upgrade process should handle the presence of hidden files like `.DS_Store` and proceed without interruptions. #### Additional Information - **Ruby Version:** 3.3.3 - **Homebrew Version:** Homebrew 4.3.5-118-g766b9b8 - **Logs and Output:** The error logs indicating the presence of `.DS_Store` files in the temporary directories. #### Steps Taken to Resolve - Manually checked and confirmed the presence of `.DS_Store` files in the directories mentioned in the error logs. - Removed `.DS_Store` files manually and retried the upgrade process. #### Suggested Fix To prevent such errors in the future, the upgrade script should handle the presence of hidden files like `.DS_Store` and ensure they do not block the directory removal process. Adding a step to ignore or remove these files during cleanup would be beneficial. #### Maintainer Assignment Since this issue is encountered on macOS, it may be relevant to assign it to the current maintainer for macOS-related issues. #### Thank you! Thank you for your attention to this issue. Looking forward to a resolution. -- https://bugs.ruby-lang.org/

2 1