Issue #20209 has been reported by alanwu (Alan Wu).
----------------------------------------
Bug #20209: YJIT can leak memory by retaining objects with singleton class
https://bugs.ruby-lang.org/issues/20209
* Author: alanwu (Alan Wu)
* Status: Closed
* Priority: Normal
* Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED
----------------------------------------
We've received [reports](https://github.com/ruby/ruby/pull/9693) of YJIT causing memory leaks in production Rails apps by keeping objects that have singleton classes alive. The symptom is similar to #19436. We have found a workaround with https://github.com/ruby/ruby/pull/9693 and would like to have it in the next 3.3 point release.
--
https://bugs.ruby-lang.org/
Issue #20208 has been reported by jprokop (Jarek Prokop).
----------------------------------------
Bug #20208: Net::HTTP errors with Errno::EAFNOSUPPORT when setting local_host with Addrinfo
https://bugs.ruby-lang.org/issues/20208
* Author: jprokop (Jarek Prokop)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
A bug was found when dealing with Ruby tests downstream. One of our builders has a specific networking configuration, resulting in Ruby incorrectly binding a socket, resulting in exception Errno::EAFNOSUPPORT,
despite localhost being IPv6 capable.
It is reproducible with Ruby 3.3, and reasonably current master (git hash a846d391d38b34fcc4f90adef967c166c923bd56).
Reproduction environment:
The networking configuration has to be in a specific state. The regular interface (such as eth0) has to have ipv6 disabled while localhost is IPv6 enabled.
I have tracked the problem to a commit adding AI_ADDRCONFIG flag: https://github.com/ruby/ruby/commit/d2ba8ea54a4089959afdeecdd963e3c4ff39174…
If I revert the commit or just simply set 2 ifdefs that are present in the diff with `HAVE_CONST_AI_ADDRCONFIG` to 0, the problem no longer occurs.
I have used vagrant with fedora/39-cloud-base box with the above mentioned git hash. However, I'd note that I reproduced it also on RHEL 8 and RHEL 9.
The VM has the following interfaces:
~~~
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:e3:aa:c1 brd ff:ff:ff:ff:ff:ff
altname enp0s5
altname ens5
inet 192.168.122.209/24 brd 192.168.122.255 scope global dynamic noprefixroute eth0
valid_lft 2099sec preferred_lft 2099sec
inet6 fe80::f5fe:e8a4:8f83:4a8f/64 scope link tentative noprefixroute
valid_lft forever preferred_lft forever
~~~
Disable IPv6 of eth0 and leave only lo with IPv6:
~~~
$ sudo sysctl "net.ipv6.conf.eth0.disable_ipv6=1"
~~~
Confirm the result:
~~~
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:e3:aa:c1 brd ff:ff:ff:ff:ff:ff
altname enp0s5
altname ens5
inet 192.168.122.209/24 brd 192.168.122.255 scope global dynamic noprefixroute eth0
valid_lft 3587sec preferred_lft 3587sec
~~~
inet6 is no longer present on eth0, but still present in lo.
Then we can copy what TestNetHTTPLocalBind is doing in setup, as that is one of the failing tests and use it for a reproducer:
~~~
$ ruby -rnet/http -e 'http = Net::HTTP.new("localhost", 8080); http.local_host = Addrinfo.tcp("localhost", 8080).ip_address; p http.get("/")'
/usr/share/ruby/net/http.rb:1603:in `initialize': Failed to open TCP connection to localhost:8080 (Address family not supported by protocol - bind(2) for "::1" port ) (Errno::EAFNOSUPPORT)
from /usr/share/ruby/net/http.rb:1603:in `open'
from /usr/share/ruby/net/http.rb:1603:in `block in connect'
from /usr/share/ruby/timeout.rb:186:in `block in timeout'
from /usr/share/ruby/timeout.rb:193:in `timeout'
from /usr/share/ruby/net/http.rb:1601:in `connect'
from /usr/share/ruby/net/http.rb:1580:in `do_start'
from /usr/share/ruby/net/http.rb:1569:in `start'
from /usr/share/ruby/net/http.rb:2297:in `request'
from /usr/share/ruby/net/http.rb:1917:in `get'
from -e:1:in `<main>'
/usr/share/ruby/net/http.rb:1603:in `initialize': Address family not supported by protocol - bind(2) for "::1" port (Errno::EAFNOSUPPORT)
from /usr/share/ruby/net/http.rb:1603:in `open'
from /usr/share/ruby/net/http.rb:1603:in `block in connect'
from /usr/share/ruby/timeout.rb:186:in `block in timeout'
from /usr/share/ruby/timeout.rb:193:in `timeout'
from /usr/share/ruby/net/http.rb:1601:in `connect'
from /usr/share/ruby/net/http.rb:1580:in `do_start'
from /usr/share/ruby/net/http.rb:1569:in `start'
from /usr/share/ruby/net/http.rb:2297:in `request'
from /usr/share/ruby/net/http.rb:1917:in `get'
from -e:1:in `<main>'
~~~
The script:
~~~
http = Net::HTTP.new("localhost", 8080)
http.local_host = Addrinfo.tcp("localhost", 8080).ip_address
p http.get("/")
~~~
Without setting the `http.local_host` attribute using Addrinfo, the reproducer does not fail with EAFNOSUPPORT. Whether `port` is specified or `nil` does not make a difference.
Whether there is a server listening on 8080 or not does not make a difference, the script fails with the errno regardless.
I have collected `strace` that points to a possible cause:
~~~
$ strace ruby -rnet/http -e 'http = Net::HTTP.new("localhost", 8080); http.local_host = Addrinfo.tcp("localhost", 8080).ip_address; p http.get("/")' 2>&1 | grep AF_INET
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_TCP) = 5
bind(5, {sa_family=AF_INET6, sin6_port=htons(0), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_scope_id=0}, 28) = -1 EAFNOSUPPORT (Address family not supported by protocol)
~~~
A socket is created with AF_INET and later is bound with AF_INET6, that is not correct behavior as far as I can tell.
Full strace is attached.
Observed failures in Ruby test suite related to this issue:
~~~
109) Error:
TestNetHTTPLocalBind#test_bind_to_local_port:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:37337 (Address family not supported by protocol - bind(2) for "::1" port 45395)
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1282:in `test_bind_to_local_port'
110) Error:
TestNetHTTPLocalBind#test_bind_to_local_host:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:46329 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1267:in `test_bind_to_local_host'
111) Error:
TestNetHTTPForceEncoding#test_response_body_encoding_false:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:41749 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1308:in `fe_request'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1312:in `test_response_body_encoding_false'
112) Error:
TestNetHTTPForceEncoding#test_response_body_encoding_string_without_content_type:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:42775 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1308:in `fe_request'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1330:in `test_response_body_encoding_string_without_content_type'
113) Error:
TestNetHTTPForceEncoding#test_response_body_encoding_true_with_content_type:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:36895 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1308:in `fe_request'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1324:in `test_response_body_encoding_true_with_content_type'
114) Error:
TestNetHTTPForceEncoding#test_response_body_encoding_encoding_without_content_type:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:37115 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1308:in `fe_request'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1336:in `test_response_body_encoding_encoding_without_content_type'
115) Error:
TestNetHTTPForceEncoding#test_response_body_encoding_true_without_content_type:
Errno::EAFNOSUPPORT: Failed to open TCP connection to localhost:37799 (Address family not supported by protocol - bind(2) for "::1" port )
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `open'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1603:in `block in connect'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:186:in `block in timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/timeout.rb:193:in `timeout'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1601:in `connect'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1580:in `do_start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1569:in `start'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:2297:in `request'
/builddir/build/BUILD/ruby-3.3.0/lib/net/http.rb:1917:in `get'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1308:in `fe_request'
/builddir/build/BUILD/ruby-3.3.0/test/net/http/test_http.rb:1318:in `test_response_body_encoding_true_without_content_type'
~~~
Related failures from specs:
~~~
1)
An exception occurred during: before :each
TCPSocket#local_address using IPv6 using an implicit hostname the returned Addrinfo uses the correct IP address ERROR
Errno::ECONNREFUSED: Connection refused - connect(2) for nil port 37121
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/local_address_spec.rb:59:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/local_address_spec.rb:59:in `new'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/local_address_spec.rb:59:in `block (4 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/local_address_spec.rb:4:in `<top (required)>'
2)
An exception occurred during: before :each
TCPSocket#remote_address using IPv6 using an implicit hostname the returned Addrinfo uses the correct IP address ERROR
Errno::ECONNREFUSED: Connection refused - connect(2) for nil port 39823
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/remote_address_spec.rb:58:in `initialize'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/remote_address_spec.rb:58:in `new'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/remote_address_spec.rb:58:in `block (4 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.3.0/spec/ruby/library/socket/tcpsocket/remote_address_spec.rb:4:in `<top (required)>'
~~~
---Files--------------------------------
strace_log.txt (304 KB)
--
https://bugs.ruby-lang.org/
Issue #20213 has been reported by jeremyevans0 (Jeremy Evans).
----------------------------------------
Bug #20213: zsuper with keyword splat without explicit keywords incorrectly uses mutable keyword splat
https://bugs.ruby-lang.org/issues/20213
* Author: jeremyevans0 (Jeremy Evans)
* Status: Open
* Priority: Normal
* Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED
----------------------------------------
As the subject states, the super call in this code is compiled incorrectly:
```ruby
extend(Module.new{def a(**k) k[:a] = 1 end})
extend(Module.new{def a(**k) p k; super; p k end})
a
# Expected output, actual output on Ruby 2.0-3.2:
{}
{}
# Actual output on Ruby 3.3 and master
{}
{:a=>1}
```
The zsuper call here uses VM_CALL_KW_SPLAT_MUT:
```
invokesuper <calldata!argc:1, FCALL|SUPER|ZSUPER|KW_SPLAT|KW_SPLAT_MUT>, nil
```
That is not correct, because as the example shows, if the super method accepts a keyword splat, the super method can modify the keyword splat, and changes are reflected in the caller.
I submitted a pull request to fix this: https://github.com/ruby/ruby/pull/9710, and marked this for backporting to 3.3.
--
https://bugs.ruby-lang.org/
Issue #20197 has been reported by osyoyu (Daisuke Aritomo).
----------------------------------------
Bug #20197: Postponed job invocations are significantly reduced in Ruby 3.3
https://bugs.ruby-lang.org/issues/20197
* Author: osyoyu (Daisuke Aritomo)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The number of postponed job invocations has been significantly reduced in Ruby 3.3.
While my understanding is that postponed jobs provide no guarantee of how soon registered callbacks will fire, I believe the current rate is too low for practical usage, especially for profilers such as StackProf.
A git bisect led me to https://github.com/ruby/ruby/commit/1f0304218cf00e05a4a126196676ba221ebf91f6 which obviously seems to be related, but I'm not sure why.
## Repro
### Expected
The job fires (nearly) every 100 ms.
```
% ruby bin/test.rb # runs for 3 seconds
count: 1
count: 2
(snip)
count: 29
```
### Actual
The job fires only once.
```
% ruby bin/test.rb
count: 1
count: 2
(snip)
count: 29
```
### Code
```ruby
require 'mycext'
time = Time.now
th = Thread.new do
loop do
sleep 0.01
break if Time.now - time > 3 # run for 3 seconds
end
end
th.join
```
```c
#include <pthread.h>
#include <stdio.h>
#include <time.h>
#include "ruby.h"
#include "ruby/debug.h"
int called_count;
void
postponed_job(void *ptr)
{
called_count++;
printf("count: %d\n", called_count);
}
_Noreturn void *
pthread_main(void *_)
{
while (1) {
rb_postponed_job_register_one(0, postponed_job, NULL);
// Sleep for 100 ms
struct timespec ts;
ts.tv_sec = 0;
ts.tv_nsec = 100 * 1000 * 1000;
nanosleep(&ts, NULL);
}
}
RUBY_FUNC_EXPORTED void
Init_mycext(void)
{
called_count = 0;
pthread_t pthread;
pthread_create(&pthread, NULL, pthread_main, NULL);
}
```
--
https://bugs.ruby-lang.org/
Issue #20153 has been reported by k0kubun (Takashi Kokubun).
----------------------------------------
Bug #20153: Backport 7f9c174102 to fix --yjit-stats with RubyVM::YJIT.enable
https://bugs.ruby-lang.org/issues/20153
* Author: k0kubun (Takashi Kokubun)
* Status: Open
* Priority: Normal
* Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED
----------------------------------------
Ruby 3.3.0 ignored --yjit-stats when `RubyVM::YJIT.enable` (no argument) is used, which was an unintended behavior.
https://github.com/ruby/ruby/pull/9415 should be backported to ruby_3_3.
--
https://bugs.ruby-lang.org/
Issue #20214 has been reported by k0kubun (Takashi Kokubun).
----------------------------------------
Bug #20214: Backport https://github.com/ruby/ruby/pull/9711 to fix exits on Ruby 3.3's new instruction
https://bugs.ruby-lang.org/issues/20214
* Author: k0kubun (Takashi Kokubun)
* Status: Closed
* Priority: Normal
* Assignee: naruse (Yui NARUSE)
* Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED
----------------------------------------
Ruby 3.3.0 YJIT missed the support for the instruction that was added shortly before the 3.3.0 release. It's used in Rails, and we didn't mean to exit on such method calls.
It'd be nice if we can fix the issue in Ruby 3.3.1 by backporting https://github.com/ruby/ruby/pull/9711.
--
https://bugs.ruby-lang.org/
Issue #19787 has been reported by joshuay03 (Joshua Young).
----------------------------------------
Feature #19787: Add Enumerable#uniq_map, Enumerable::Lazy#uniq_map, Array#uniq_map and Array#uniq_map!
https://bugs.ruby-lang.org/issues/19787
* Author: joshuay03 (Joshua Young)
* Status: Open
* Priority: Normal
----------------------------------------
I would like to propose a collection of new methods, `Enumerable#uniq_map`, `Enumerable::Lazy#uniq_map`, `Array#uniq_map` and `Array#uniq_map!`.
TL;DR: It's a drop in replacement for `.map { ... }.uniq`, with better performance.
I've quite often had to map over an array and get its unique elements. It occurred to me when doing so recently that Ruby doesn't have a short form method for doing that, similar to how `flat_map { ... }` replaces `.map { ... }.flatten` and `filter_map { ... }` replaces `.map { ... }.compact` (with minor differences). I think these new methods could be beneficial both in terms of better performance and writing more succinct code.
I have already got a draft PR up with some initial benchmarks in the description: https://github.com/ruby/ruby/pull/8140.
--
https://bugs.ruby-lang.org/
Issue #20225 has been reported by make_now_just (Hiroya Fujinami).
----------------------------------------
Bug #20225: Inconsistent behavior of regex matching for a regex has a null loop
https://bugs.ruby-lang.org/issues/20225
* Author: make_now_just (Hiroya Fujinami)
* Status: Open
* Priority: Normal
* Assignee: make_now_just (Hiroya Fujinami)
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Usually, in Ruby (Onigmo), when a null loop (a loop consuming no characters) occurs on regex matching, this loop is terminated. But, if a loop has a capture and some complex condition is satisfied, this causes backtracking. This behavior invokes unexpected results, for example,
```ruby
p /(?:.B.(?<a>(?:[C-Z]|.)*)+){2}/ =~ "ABCABC" # => nil
p /(?:.B.(?:(?:[C-Z]|.)*)+){2}/ =~ "ABCABC" # => 0
```
Because the above regex has a capture and the below does not, different matching results are returned. It is not very intuitive that the presence of a capture changes the matching result.
The detailed condition for changing the null-loop behavior is 1) a previous capture in this loop holds the empty string, and 2) this capture's position is different from the current matching position. This condition is checked in `STACK_NULL_CHECK_MEMST` (https://github.com/ruby/ruby/blob/bbb7ab906ec64b963bd4b5d37e47b14796d64371/…).
Perhaps, you cannot understand what this condition means. Don't worry, I also cannot understand. This condition has been introduced for at least 20 years, and no one may remember the reason for this necessity. (If you know, please tell me!) Even if there is a reason, I believe that there is no reasonable authority for allowing counter-intuitive behavior, such as the above example.
This behavior can also cause memoization to be buggy. Memoization relies on the fact that backtracking only depends on positions and states (byte-code offsets of a regex). However, this condition additionally refers to captures, and the memoization is broken.
My proposal is to **correct this inconsistent behavior**. Specifically, a null loop should be determined solely on the basis of whether the matching position has changed, without referring to captures.
This fix changes the behavior of regex matching, but I believe that the probability that this will actually cause backward compatibility problems is remarkably low. This is because I have never seen any mention of this puzzling behavior before.
--
https://bugs.ruby-lang.org/
Issue #20150 has been reported by peterzhu2118 (Peter Zhu).
----------------------------------------
Bug #20150: Memory leak in grapheme clusters
https://bugs.ruby-lang.org/issues/20150
* Author: peterzhu2118 (Peter Zhu)
* Status: Open
* Priority: Normal
* Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED
----------------------------------------
GitHub PR: https://github.com/ruby/ruby/pull/9414
String#grapheme_cluters and String#each_grapheme_cluster leaks memory because if the string is not UTF-8, then the created regex will not be freed.
For example:
```ruby
str = "hello world".encode(Encoding::UTF_32LE)
10.times do
1_000.times do
str.grapheme_clusters
end
puts `ps -o rss= -p #{$$}`
end
```
Before:
```
26000
42256
59008
75792
92528
109232
125936
142672
159392
176160
```
After:
```
9264
9504
9808
10000
10128
10224
10352
10544
10704
10896
```
--
https://bugs.ruby-lang.org/