January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116356] [Ruby master Bug#20198] Threaded DNS resolver does not propagate errno to the calling thread
by kjtsanaktsidis (KJ Tsanaktsidis) 14 Mar '24

14 Mar '24

Issue #20198 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20198: Threaded DNS resolver does not propagate errno to the calling thread https://bugs.ruby-lang.org/issues/20198 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- If we get a return value of `EAI_SYSTEM` from `getaddrinfo`, we transform that into an appropriate `Errno::` exception on the Ruby side. However, because we now run the actual call to `getaddrinfo` in a thread, we lose that `errno` value (because `errno` is thread-local). So, what we actually raise in case of `EAI_SYSTEM` is just the last error which happened on the calling thread - e.g. this `ECHILD` which presumably got set in the bowels of pthreads somewhere: ``` 1) Socket::IPSocket#getaddress raises an error on unknown hostnames ERROR Expected SocketError but got: Errno::ECHILD (No child processes - getaddrinfo) /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:22:in `getaddress' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:22:in `block (3 levels) in <top (required)>' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:21:in `block (2 levels) in <top (required)>' /home/runner/work/ruby/ruby/src/spec/ruby/library/socket/ipsocket/getaddress_spec.rb:4:in `<top (required)>' ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116073] [Ruby master Bug#20161] Memory leak in regexp grapheme clusters
by peterzhu2118 (Peter Zhu) 14 Mar '24

14 Mar '24

Issue #20161 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20161: Memory leak in regexp grapheme clusters https://bugs.ruby-lang.org/issues/20161 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9447 The cc->mbuf gets overwritten, so we need to free it to not leak memory. For example: ```ruby str = "hello world".encode(Encoding::UTF_32LE) 10.times do 1_000.times do str.grapheme_clusters end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 15536 15760 15920 16144 16304 16480 16640 16784 17008 17280 ``` After: ``` 15584 15584 15760 15824 15888 15888 15888 15888 16048 16112 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116489] [Ruby master Bug#20224] Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3
by yui-knk (Kaneko Yuichiro) 14 Mar '24

14 Mar '24

Issue #20224 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Bug #20224: Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3 https://bugs.ruby-lang.org/issues/20224 * Author: yui-knk (Kaneko Yuichiro) * Status: Closed * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- It's nice if we can backport https://github.com/ruby/ruby/pull/9634 (3d19409637de1462b6790d2a92344bf0a10d8c52) to Ruby 3.3 branch. Even so previous codes work as expected but it depends on how parse.c works and misleading. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113966] [Ruby master Bug#19742] Introduce `Module#anonymous?`
by ioquatix (Samuel Williams) 12 Mar '24

12 Mar '24

Issue #19742 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19742: Introduce `Module#anonymous?` https://bugs.ruby-lang.org/issues/19742 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- As a follow-on <from https://bugs.ruby-lang.org/issues/19521>, I'd like propose we introduce `Module#anonymous?`. In some situations, like logging/formatting, serialisation/deserialization, debugging or meta-programming, we might like to know if a class is a proper constant or not. However, this brings about some other issues which might need to be discussed. After assigning a constant, then removing it, the internal state of Ruby still believes that the class name is permanent, even thought it's no longer true. e.g. ``` m = Module.new m.anonymous? # true M = m m.anonyomous # false Object.send(:remove_const, :M) M # uninitialized constant M (NameError) M.anonymous? # false ``` Because RCLASS data structure is not updated after the constant is removed, internally the state still has a "permanent class name". I want to use this proposal to discuss this issue and whether there is anything we should do about such behaviour (or even if it's desirable). Proposed PR: https://github.com/ruby/ruby/pull/7966 cc @fxn -- https://bugs.ruby-lang.org/

9 46

[ruby-core:115925] [Ruby master Bug#20096] Ruby 3.2.2 win32/registry: Junk appended to Windows Registry String Value
by jay4rubydev (Jay M) 07 Mar '24

07 Mar '24

Issue #20096 has been reported by jay4rubydev (Jay M). ---------------------------------------- Bug #20096: Ruby 3.2.2 win32/registry: Junk appended to Windows Registry String Value https://bugs.ruby-lang.org/issues/20096 * Author: jay4rubydev (Jay M) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mswin64_140] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ruby Version: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mswin64_140] Compiler: MSVC 2019 - Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30147 for x64. Issue: win32/registry adds junk to Windows Registry String Value Code: require 'win32/registry' win_oracle_key = "SOFTWARE\\MicroSoft" reg=Win32::Registry::HKEY_LOCAL_MACHINE.open(win_oracle_key, Win32::Registry::KEY_ALL_ACCESS) inst_loc_key = "inst_loc" inv_dir="C:\\Program Files\\Tester\\ModuleInfo" reg[inst_loc_key] = inv_dir Result: Registry contains: C:\Program Files\Tester\ModuleInfo爀 Observation: Looks like memory overread Expected Result - without the junk: C:\Program Files\Tester\ModuleInfo After changing the code in registry.rb: From: def write(name, type, data) termsize = 0 case type when REG_SZ, REG_EXPAND_SZ data = data.encode(WCHAR) termsize = WCHAR_SIZE To: def write(name, type, data) termsize = 0 case type when REG_SZ, REG_EXPAND_SZ enc_data = data.encode(WCHAR) # Add NULL WCHAR for string data and don't set the termsize because # enc_data.bytesize will now include the size of the NULL character. enc_data += WCHAR_NUL # termsize = WCHAR_SIZE ... -- https://bugs.ruby-lang.org/

4 7

[ruby-core:115892] [Ruby master Bug#20085] Fiber.new{ }.resume causes Segmentation fault for Ruby 3.3.0 on aarch64-linux
by oleksii (Oleksii Leonov) 29 Feb '24

29 Feb '24

Issue #20085 has been reported by oleksii (Oleksii Leonov). ---------------------------------------- Bug #20085: Fiber.new{ }.resume causes Segmentation fault for Ruby 3.3.0 on aarch64-linux https://bugs.ruby-lang.org/issues/20085 * Author: oleksii (Oleksii Leonov) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` ruby -e "Fiber.new{}.resume" 0.170 -e:1: [BUG] Segmentation fault at 0x0036ffffb4f110f0 0.170 ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] 0.170 0.170 -- Control frame information ----------------------------------------------- 0.170 c:0003 p:---- s:0010 e:000009 CFUNC :resume 0.170 c:0002 p:0007 s:0006 E:0005e0 EVAL -e:1 [FINISH] 0.170 c:0001 p:0000 s:0003 E:000b50 DUMMY [FINISH] 0.170 0.170 -- Ruby level backtrace information ---------------------------------------- 0.170 -e:1:in `<main>' 0.170 -e:1:in `resume' 0.170 0.170 -- Threading information --------------------------------------------------- 0.170 Total ractor count: 1 0.170 Ruby thread count for this ractor: 1 0.170 0.170 -- Machine register context ------------------------------------------------ 0.170 x0: 0x0000aaab11b4f570 x1: 0x0000aaab11de4a40 x2: 0x0000ffffe5571990 0.170 x3: 0x0000ffff9ac5ff60 x4: 0x0000ffff9ac60018 x5: 0x0000ffff9ac80000 0.170 x6: 0x0000ffffb5458b88 x7: 0x0000000000000000 x18: 0x00000000007fffff 0.170 x19: 0x0000000000000000 x20: 0x0000000000000000 x21: 0x0000000000000000 0.170 x22: 0x0000000000000000 x23: 0x0000000000000000 x24: 0x0000000000000000 0.170 x25: 0x0000000000000000 x26: 0x0000000000000000 x27: 0x0000000000000000 0.170 x28: 0x0000000000000000 x29: 0x0000000000000000 sp: 0x0000ffff9ac60000 0.170 fau: 0x0036ffffb4f110f0 0.170 0.170 -- C level backtrace information ------------------------------------------- 0.171 Segmentation fault ``` - Ruby 3.3.0-rc1 and 3.2.2 work without Segmentation fault on both amd64 and arm64. - Ruby 3.3.0 works on amd64, but fails with Segmentation fault on arm64 (aarch64-linux) on Ubuntu 22.04 and Debian Bookworm. Dockerfile to reproduce: ``` FROM debian:bookworm # Set UTF-8 locale by default. ENV RBENV_ROOT=/usr/local/rbenv \ PATH=/usr/local/rbenv/bin:/usr/local/rbenv/shims:$PATH RUN ( \ export DEBIAN_FRONTEND=noninteractive \ && apt-get update \ && apt-get install -y --no-install-recommends \ ca-certificates curl git \ # standard dependencies for rbenv autoconf \ patch \ build-essential \ rustc \ libssl-dev \ libyaml-dev \ libreadline6-dev \ zlib1g-dev \ libgmp-dev \ libncurses5-dev \ libffi-dev \ libgdbm6 \ libgdbm-dev \ libdb-dev \ uuid-dev \ ) ENV RBENV_VERSION=3.3.0 RUN ( \ export DEBIAN_FRONTEND=noninteractive \ # Install rbenv & ruby-build && git clone https://github.com/rbenv/rbenv.git /usr/local/rbenv \ && git clone https://github.com/rbenv/ruby-build.git /usr/local/rbenv/plugins/ruby-build \ && /usr/local/rbenv/plugins/ruby-build/install.sh \ && echo 'export RBENV_ROOT=/usr/local/rbenv' >> /etc/profile.d/rbenv.sh \ && echo 'export PATH=/usr/local/rbenv/bin:$PATH' >> /etc/profile.d/rbenv.sh \ && echo 'eval "$(rbenv init -)"' >> /etc/profile.d/rbenv.sh \ && echo 'export RBENV_ROOT=/usr/local/rbenv' >> /root/.bashrc \ && echo 'export PATH=/usr/local/rbenv/bin:$PATH' >> /root/.bashrc \ && echo 'eval "$(rbenv init -)"' >> /root/.bashrc \ && eval "$(rbenv init -)"; rbenv install $RBENV_VERSION \ && eval "$(rbenv init -)"; rbenv global $RBENV_VERSION \ ) RUN RUN ruby -e "Fiber.new{}.resume" ``` `docker build --platform linux/arm64 .` ---Files-------------------------------- Dockerfile (1.53 KB) -- https://bugs.ruby-lang.org/

13 26

[ruby-core:116387] [Ruby master Bug#20206] PTY.spawn seems to fail to capture the output of "echo foo" once in a while
by lacostej (Jerome Lacoste) 26 Feb '24

26 Feb '24

Issue #20206 has been reported by lacostej (Jerome Lacoste). ---------------------------------------- Bug #20206: PTY.spawn seems to fail to capture the output of "echo foo" once in a while https://bugs.ruby-lang.org/issues/20206 * Author: lacostej (Jerome Lacoste) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- We use PTY.spawn to call "echo foo", and on Mac it seems to randomly fail, capturing an empty output every now and then. On Linux, the failure doesn't seem to happen. The following code 1. contains 2 ways of capturing the output from PTY.spawn. Both seem to show the same issue (`run_command` and `run_command2`) 2. invokes the external `stress` program. This helps to trigger the issue more often. ``` ruby require 'pty' require 'expect' def run_command(command) output = [] PTY.spawn(command) do |command_stdout, command_stdin, pid| begin command_stdout.each do |l| line = l.chomp output << line end rescue Errno::EIO # This is expected on some linux systems, that indicates that the subcommand finished # and we kept trying to read, ignore it ensure command_stdout.close command_stdin.close Process.wait(pid) end end raise "#{$?.exited?} #{$?.stopped?} #{$?.signaled?} - #{$?.stopsig} - #{$?.termsig} -" unless $?.exitstatus == 0 [$?.exitstatus, output.join("\n")] end def run_command2(command) output = [] PTY.spawn(command) do |command_stdout, command_stdin, pid| output = "" begin a = command_stdout.expect(/foo.*/, 5) output = a[0] if a ensure command_stdout.close command_stdin.close Process.wait(pid) end end raise "#{$?.exited?} #{$?.stopped?} #{$?.signaled?} - #{$?.stopsig} - #{$?.termsig} -" unless $?.exitstatus == 0 [$?.exitstatus, output] end def test_spawn(command) status, output = run_command(command) errors = [] errors << "status was '#{status}'" unless status == 0 errors << "output was '#{output}'" unless output == "foo" raise errors.join(" - ") unless errors.empty? end t = nil pid = nil if ENV['STRESS'] t = Thread.new do |t| puts "Spawning stress" pid = spawn("stress -c 16 -t 99", pgroup: true) puts "Waiting #{pid}" Process.wait(pid) puts "#{pid} DONE" end end command = "echo foo" if ARGV.count == 1 command = ARGV[0] end puts "Will run command: '#{command}'" errors = 0 2000.times do |i| begin test_spawn(command) rescue => e puts "ERROR #{i}: #{e}" errors += 1 end end if t begin Process.kill(:SIGKILL, -pid) rescue Errno::ESRCH # already dead, ignore end t.join end raise "Failed #{errors} times" unless errors == 0 ``` Here are some ways of reproducing the issue ``` ruby test_pty.rb STRESS=y ruby test_pty.rb ``` Here's an example of how it fails on circleci. https://app.circleci.com/pipelines/github/lacostej/cienvs/33/workflows/d6d8… Tested on ruby 2.6 to ruby 3.3.0 on Mac. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:115985] [Ruby master Feature#20108] Introduction of Happy Eyeballs Version 2 (RFC8305) in Socket.tcp
by shioimm (Misaki Shioi) 26 Feb '24

26 Feb '24

Issue #20108 has been reported by shioimm (Misaki Shioi). ---------------------------------------- Feature #20108: Introduction of Happy Eyeballs Version 2 (RFC8305) in Socket.tcp https://bugs.ruby-lang.org/issues/20108 * Author: shioimm (Misaki Shioi) * Status: Open * Priority: Normal ---------------------------------------- This is an implementation of Happy Eyeballs version 2 (RFC 8305) in Socket.tcp. ### Background Currently, `Socket.tcp` synchronously resolves names and makes connection attempts with `Addrinfo::foreach.` This implementation has the following two problems. 1. In hostname resolution, the program stops until the DNS server responds to all DNS queries. 2. In a connection attempt, while an IP address is trying to connect to the destination host and is taking time, the program stops, and other resolved IP addresses cannot try to connect. ### Proposal "Happy Eyeballs" ([RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305)) is an algorithm to solve this kind of problem. It avoids delays to the user whenever possible and also uses IPv6 preferentially. I implemented it into `Socket.tcp` by using `Addrinfo.getaddrinfo` in each thread spawned per address family to resolve the hostname asynchronously, and using `Socket::connect_nonblock` to try to connect with multiple addrinfo in parallel. See https://github.com/ruby/ruby/pull/9374 ### Outcome This change eliminates a fatal defect in the following cases. #### Case 1. One of the A or AAAA DNS queries does not return ```ruby require 'socket' class Addrinfo class << self # Current Socket.tcp depends on foreach def foreach(nodename, service, family=nil, socktype=nil, protocol=nil, flags=nil, timeout: nil, &block) getaddrinfo(nodename, service, Socket::AF_INET6, socktype, protocol, flags, timeout: timeout) .concat(getaddrinfo(nodename, service, Socket::AF_INET, socktype, protocol, flags, timeout: timeout)) .each(&block) end def getaddrinfo(_, _, family, *_) case family when Socket::AF_INET6 then sleep when Socket::AF_INET then [Addrinfo.tcp("127.0.0.1", 4567)] end end end end Socket.tcp("localhost", 4567) ``` Because the current `Socket.tcp` cannot resolve IPv6 names, the program stops in this case. It cannot start to connect with IPv4 address. Though `Socket.tcp` with HEv2 can promptly start a connection attempt with IPv4 address in this case. #### Case 2. Server does not promptly return ack for syn of either IPv4 / IPv6 address family ```ruby require 'socket' fork do socket = Socket.new(Socket::AF_INET6, :STREAM) socket.setsockopt(:SOCKET, :REUSEADDR, true) socket.bind(Socket.pack_sockaddr_in(4567, '::1')) sleep socket.listen(1) connection, _ = socket.accept connection.close socket.close end fork do socket = Socket.new(Socket::AF_INET, :STREAM) socket.setsockopt(:SOCKET, :REUSEADDR, true) socket.bind(Socket.pack_sockaddr_in(4567, '127.0.0.1')) socket.listen(1) connection, _ = socket.accept connection.close socket.close end Socket.tcp("localhost", 4567) ``` The current `Socket.tcp` tries to connect serially, so when its first name resolves an IPv6 address and initiates a connection to an IPv6 server, this server does not return an ACK, and the program stops. Though `Socket.tcp` with HEv2 starts to connect sequentially and in parallel so a connection can be established promptly at the socket that attempted to connect to the IPv4 server. In exchange, the performance of `Socket.tcp` with HEv2 will be degraded. ``` 100.times { Socket.tcp("www.ruby-lang.org", 80) } # Socket.tcp (Before) 0.123809 # Socket.tcp (After) 0.224684 ``` This is due to the addition of the creation of IO objects, Thread objects, etc., and calls to `IO::select` in the implementation. -- https://bugs.ruby-lang.org/

4 13

[ruby-core:115169] [Ruby master Bug#19973] Duplicate keyword argument names don't always warn
by tenderlovemaking (Aaron Patterson) 25 Feb '24

25 Feb '24

Issue #19973 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #19973: Duplicate keyword argument names don't always warn https://bugs.ruby-lang.org/issues/19973 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-24T19:38:50Z cleanup 3525a9bd22) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Calling a method with duplicate keyword arguments doesn't warn when it could: ```ruby def bar a:, b: a + b end # Warning bar(a: 1, b: 3, a: 2) z = { b: 123 } # No warning bar(a: 1, **z, a: 2) ``` The first call to `bar` gives a warning about duplicate keyword args, but the second call doesn't. I think both cases should emit a warning. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116516] [Ruby master Bug#20229] Empty keyword splat in array not removed in ARGSPUSH case
by jeremyevans0 (Jeremy Evans) 20 Feb '24

20 Feb '24

Issue #20229 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20229: Empty keyword splat in array not removed in ARGSPUSH case https://bugs.ruby-lang.org/issues/20229 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- For the following code: ```ruby a = [] kw = {} [*a, **kw] ``` Ruby 2.0-3.3 return `[{}]`, when the desired result (since 2.7) is `[]`. I discovered this while working on an optimization for the ARGSCAT case, where `[*a, 1, **kw]` allocates two arrays instead of a single array. My optimization fixes that by adding a `pushtoarraykwsplat` instruction as a replacement for `newarraykwsplat`. I've used `pushtoarraykwsplat` in the ARGSPUSH case to fix this bug. My pull request for the bug fix and optimization: https://github.com/ruby/ruby/pull/9766 -- https://bugs.ruby-lang.org/

1 2