Issue #19436 has been reported by byroot (Jean Boussier).
----------------------------------------
Bug #19436: Call Cache for singleton methods can lead to "memory leaks"
https://bugs.ruby-lang.org/issues/19436
* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Using "memory leaks" with quotes, because strictly speaking the memory isn't leaked, but it can nonetheless lead to large memory overheads.
### Minimal Reproduction
```ruby
module Foo
def bar
end
end
def call_bar(obj)
# Here the call cache we'll keep a ref on the method_entry
# which then keep a ref on the singleton_class, making that
# instance immortal until the method is called again with
# another instance.
# The reference chain is IMEMO(callcache) -> IMEMO(ment) -> ICLASS -> CLASS(singleton) -> OBJECT
obj.bar
end
obj = Object.new
obj.extend(Foo)
call_bar(obj)
id = obj.object_id
obj = nil
4.times { GC.start }
p ObjectSpace._id2ref(id)
```
### Explanation
Call caches keep a strong reference onto the "callable method entry" (CME), which itself keeps a strong reference on the called object class
and in the cache of a singleton class, it keeps a strong reference onto the `attached_object` (instance).
This means that any call site that calls a singleton method, will effectively keep a strong reference onto the last receiver.
If the method is frequently called it's not too bad, but if it's infrequently called, it's effectively a (bounded) memory leak.
And if the `attached_object` is big, the wasted memory can be very substantial.
### Practical Implications
Once relative common API impacted by this is [Rails' `extending` API](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#met…. This API allow to extend a "query result set" with a module.
These query results set can sometimes be very big, especially since they keep references to the instantiated `ActiveRecord::Base` instances etc.
### Possible Solutions
#### Only keep a weak reference to the CME
The fairly "obvious" solution is to keep a weak reference to the CME, that's what I explored in https://github.com/ruby/ruby/pull/7272, and it seems to work.
However in debug mode It does fail on an assertion during compaction, but it's isn't quite clear to me what the impact is.
Additionally, something that makes me think this would be the right solution, is that call caches already try to avoid marking the class:
```c
# vm_callinfo.h:275
struct rb_callcache {
const VALUE flags;
/* inline cache: key */
const VALUE klass; // should not mark it because klass can not be free'd
// because of this marking. When klass is collected,
// cc will be cleared (cc->klass = 0) at vm_ccs_free().
```
So it appears that the class being also marked through the CME is some kind of oversight?
#### Don't cache based on some heuristics
If the above isn't possible or too complicated, an alternative would be to not cache CMEs found in singleton classes, except if it's the the singleton class of a `Class` or `Module`.
It would make repeated calls to such methods slower, but the assumption is that it's unlikely that these CME would live very long.
#### Make `Class#attached_object` a weak reference
Alternatively we could make the `attached_object` a weak reference, which would drastically limit the amount of memory that may be leaked in such scenario.
The downside is that `Class#attached_object` was very recently exposed in Ruby 3.2.0, so it means changing its semantic a bit.
cc @peterzhu2118 @ko1
--
https://bugs.ruby-lang.org/
Issue #19785 has been reported by peterzhu2118 (Peter Zhu).
----------------------------------------
Feature #19785: Deprecate `RUBY_GC_HEAP_INIT_SLOTS `
https://bugs.ruby-lang.org/issues/19785
* Author: peterzhu2118 (Peter Zhu)
* Status: Open
* Priority: Normal
----------------------------------------
GitHub PR: https://github.com/ruby/ruby/pull/8116
The `RUBY_GC_HEAP_INIT_SLOTS` environment variable is replaced by
`RUBY_GC_HEAP_INIT_SIZE_%d_SLOTS` (introduced in commit
[3ab3455](https://github.com/ruby/ruby/commit/3ab34551450c7a3a3e1ae0b24bf6b7…),
which allows for more fine-grained tuning of size pools. So it doesn't make
sense to keep the environment variable `RUBY_GC_HEAP_INIT_SLOTS`.
I'm proposing to deprecate `RUBY_GC_HEAP_INIT_SLOTS` and eventually remove it.
--
https://bugs.ruby-lang.org/
Issue #19783 has been reported by peterzhu2118 (Peter Zhu).
----------------------------------------
Feature #19783: Weak References in the GC
https://bugs.ruby-lang.org/issues/19783
* Author: peterzhu2118 (Peter Zhu)
* Status: Open
* Priority: Normal
----------------------------------------
GitHub PR: https://github.com/ruby/ruby/pull/8113
I'm proposing support for weak references in the Ruby garbage collector. This
feature adds a new function called `void rb_gc_mark_weak(VALUE *ptr)` which
marks `*ptr` as weak, meaning that if no other object strongly marks `*ptr`
(using `rb_gc_mark` or `rb_gc_mark_movable`), then it will be overwritten with
`*ptr = Qundef`.
Weak references are implemented using a buffer in `objspace` that stores all
the `ptr` in the latest marking phase. After marking has finished, we iterate
over the buffer and check if the `*ptr` is a dead object. If it is, then we
set `*ptr = Qundef`.
Weak references are implemented on the callable method entry (CME) of
callcaches, which fixes issue #19436.
Weak references are also implemented on `ObjectSpace::WeakMap` and
`ObjectSpace::WeakKeyMap`, which have:
- Significantly simpler implementations because we no longer need to have
multiple tables and do not need to define finalizers on the objects.
- Support for compaction because finalizers pin objects and we no longer need
to define finalizers on the objects.
- Much faster performance (see [benchmarks](#microbenchmarks)).
## Benchmark results
### YJIT-bench
We see largely no change in performance or memory usage after this feature.
```
-------------- --------- ---------- --------- ----------- ---------- --------- -------------- -----------
bench base (ms) stddev (%) RSS (MiB) branch (ms) stddev (%) RSS (MiB) branch 1st itr base/branch
activerecord 72.3 2.2 51.9 72.9 2.2 51.9 0.99 0.99
chunky-png 889.2 0.3 43.9 874.5 0.3 42.5 1.02 1.02
erubi-rails 21.2 13.5 90.7 21.0 13.3 90.9 1.01 1.01
hexapdf 2557.0 0.8 157.1 2559.2 0.7 197.1 1.01 1.00
liquid-c 65.2 0.4 34.5 65.4 0.4 34.5 0.99 1.00
liquid-compile 62.5 0.4 30.9 62.2 0.4 31.0 1.00 1.01
liquid-render 164.6 0.4 33.1 162.6 0.3 33.1 1.01 1.01
mail 133.3 0.1 46.4 134.4 0.2 46.4 1.03 0.99
psych-load 2066.6 0.2 31.6 2083.6 0.1 31.6 0.99 0.99
railsbench 2027.0 0.5 88.8 2019.4 0.5 89.0 1.01 1.00
ruby-lsp 65.6 3.0 90.1 65.4 3.1 88.5 1.00 1.00
sequel 73.1 1.1 36.6 73.1 1.1 36.6 1.00 1.00
-------------- --------- ---------- --------- ----------- ---------- --------- -------------- -----------
```
### Microbenchmarks
We can see signficantly improved performance in `ObjectSpace::WeakMap`, with
`ObjectSpace::WeakMap#[]=` being nearly 3x faster.
Base:
```
ObjectSpace::WeakMap#[]=
1.037M (± 0.5%) i/s - 5.262M in 5.072833s
ObjectSpace::WeakMap#[]
12.367M (± 0.9%) i/s - 62.479M in 5.052365s
```
Branch:
```
ObjectSpace::WeakMap#[]=
3.054M (± 0.3%) i/s - 15.448M in 5.058783s
ObjectSpace::WeakMap#[]
15.796M (± 4.8%) i/s - 79.245M in 5.028583s
```
Code:
```ruby
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "benchmark-ips"
end
wmap = ObjectSpace::WeakMap.new
key = Object.new
val = Object.new
wmap[key] = val
Benchmark.ips do |x|
x.report("ObjectSpace::WeakMap#[]=") do |times|
i = 0
while i < times
wmap[Object.new] = Object.new
i += 1
end
end
x.report("ObjectSpace::WeakMap#[]") do |times|
i = 0
while i < times
wmap[key]
wmap[val] # does not exist
i += 1
end
end
end
```
--
https://bugs.ruby-lang.org/
Issue #19766 has been reported by mame (Yusuke Endoh).
----------------------------------------
Misc #19766: DevMeeting-2023-08-24
https://bugs.ruby-lang.org/issues/19766
* Author: mame (Yusuke Endoh)
* Status: Open
* Priority: Normal
----------------------------------------
# The next dev meeting
**Date: 2023/08/24 13:00-17:00** (JST)
Log: *TBD*
- Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker.
- Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly.
- Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue).
- We will write a record of the discussion in the file or to each ticket in English.
- All activities are best-effort (keep in mind that most of us are volunteer developers).
- The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time.
- *DO NOT* discuss then on this ticket, please.
# Call for agenda items
If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format:
```
* [Ticket ref] Ticket title (your name)
* Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.)
```
Example:
```
* [Feature #14609] `Kernel#p` without args shows the receiver (ko1)
* I feel this feature is very useful and some people say :+1: so let discuss this feature.
```
- It is recommended to add a comment by 2023/08/21. We hold a preparatory meeting to create an agenda a few days before the dev-meeting.
- The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format.
- Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion.
--
https://bugs.ruby-lang.org/
Issue #19761 has been reported by nobu (Nobuyoshi Nakada).
----------------------------------------
Bug #19761: AddressSanitizer fails with Thread and Process
https://bugs.ruby-lang.org/issues/19761
* Author: nobu (Nobuyoshi Nakada)
* Status: Open
* Priority: Normal
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
With AddressSanitizer on macOS, this code crashes.
```ruby
system('')
Thread.new {}
300_000.times do
String.new(capacity: 1000)
end
```
```
==62870==ERROR: AddressSanitizer failed to deallocate 0x8000 (32768) bytes at address 0x62d00000a400
AddressSanitizer: CHECK failed: sanitizer_posix.cpp:61 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0) (tid=857549)
<empty stack>
Assertion Failed: ../debug/vm_core.h:1853:rb_current_execution_context:!expect_ec || ec != NULL
ruby 3.3.0dev (2023-07-08T02:54:37Z tmp 0083edc3a0) [x86_64-darwin22]
-- Crash Report log information --------------------------------------------
See Crash Report log file in one of the following locations:
* ~/Library/Logs/DiagnosticReports
* /Library/Logs/DiagnosticReports
for more details.
Don't forget to include the above Crash Report log file in bug reports.
-- C level backtrace information -------------------------------------------
```
--
https://bugs.ruby-lang.org/
Issue #19770 has been reported by Schachi65 (Joachim Schachermayer).
----------------------------------------
Bug #19770: TLS / Certificate Hostname Verification against IP fails with "address family must be specified"
https://bugs.ruby-lang.org/issues/19770
* Author: Schachi65 (Joachim Schachermayer)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.4
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
I dont't use Ruby directly but indirectly via Fluentd-1.16.1 (https://www.fluentd.org/). Fluentd uses Ruby 3.1.4.
The problem is, that a TLS connection from a Fluentd Client to a Fluentd Server fails, if the connection is established using the IP address of the server and tls_verify_hostname is switched to on. In the servers certificate, in the SubjectAlternativeNames extention, there is the servers IP address set with the correct value. I get the error message " **address family must be specified** ".
**Probable cause** :
ext/openssl/lib/openssl/ **ssl.rb** line 288:
Here the Methode IPAddr.new(hostname) is called with just one argument:
return true if san.value == IPAddr.new( **hostname** ).hton
lib/ **ipaddr.rb** line 594:
Here the exception is raised with the observed error message:
when Socket::AF_UNSPEC
**raise** AddressFamilyError, " **address family must be specified** "
The exception is raised, because the variable "family" has the value " **Socket::AF_UNSPEC** " because thats the default value of "family" if there is no additional parameter in the call to "initialize()".
The documumentation of "initialize()" says in line 575/576/577: "Although the address family is determined automatically from a specified string, you can specify one explicitly by the optional second argument."
But that automatic address determination is done too late. If no second parameter is given, it is not done at all because that exception is raised (assumption: the addr i always a string).
**Probable solution** :
If the address family is set to Socket::AF_UNSPEC with the call of "initialize()", dont't raise an exception now but do the in the comment announced automatic adress family detection here.
--
https://bugs.ruby-lang.org/
Issue #19791 has been reported by stuartb (Stuart Bury).
----------------------------------------
Bug #19791: Debugger does not handle delete key correctly
https://bugs.ruby-lang.org/issues/19791
* Author: stuartb (Stuart Bury)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux-musl]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
When entering the debugger using `binding.debugger` or the `rdbg` command, pressing the delete key on the keyboard to delete characters at the REPL merely prints "^[[3~" to the console, and does not actually delete anything. This is **very** inconvenient. Can this be fixed somehow? I really like the new debugger, but my swearing like a sailor every time I hit "delete" is starting to really annoy my wife...
It is worth noting that if I run the `pry` command and then, within pry, call `binding.debugger`, the delete key operates correctly.
I believe the issue is related in some way to readline support, or the lack thereof – but my Ruby installation was compiled with readline support. I've tried everything I can think of, and spent some time on Google trying to solve it, but the problem remains. It happens in Alpine 3.18 and also in Debian 12.
--
https://bugs.ruby-lang.org/
Issue #19325 has been reported by dsisnero (Dominic Sisneros).
----------------------------------------
Bug #19325: Windows support lacking.
https://bugs.ruby-lang.org/issues/19325
* Author: dsisnero (Dominic Sisneros)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Ruby's support on windows has always been second class. With some of the recent decisions, windows support is falling even more behind. Recent developments in mjit and yjit that exclude windows are two glaring issues that should be corrected. Googling 'percent of windows vs other operating systems' and it shows windows has a share of 76%. Ceding that users to python and other programming languages has to be one of the reasons python continues get more market share from ruby. With rust having first class windows support and threading support, is there a reason why yjit is not able to work on windows? Also, windows compiler support has matured enough and vcpkg support has evolved enough that it seems it should be possible to finally get a ruby version without having to use msys2. Even Crystal language has a version that runs on windows without needing msys2.
--
https://bugs.ruby-lang.org/