Issue #19160 has been reported by kaiquekandykoga (Kaíque Koga).
----------------------------------------
Bug #19160: cmp_clamp arguments
https://bugs.ruby-lang.org/issues/19160
* Author: kaiquekandykoga (Kaíque Koga)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-freebsd13.1]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
If clamp receives min higher than max, it will raise an exception. The message says *min argument must be smaller than max argument* , but min can actually be equal to max too.
Patch https://github.com/ruby/ruby/pull/6802.
--
https://bugs.ruby-lang.org/
Issue #19362 has been reported by zverok (Victor Shepelev).
----------------------------------------
Bug #19362: #dup on Proc doesn't call initialize_dup
https://bugs.ruby-lang.org/issues/19362
* Author: zverok (Victor Shepelev)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
In #17545, `#dup` had changed to create an instance of the subclass.
It, though, doesn't invoke `initialize_dup` of the subclass, unlike other standard classes.
```ruby
class MyAry < Array
def initialize_dup(...)
p(self.class, ...)
super
end
end
class MyString < String
def initialize_dup(...)
p(self.class, ...)
super
end
end
class MyProc < Proc
def initialize_dup(...)
p(self.class, ...)
super
end
end
MyString.new('test').dup # prints MyString, "test"
MyAry.new(['test']).dup # prints MyAry, ["test"]
MyProc.new { 'test' }.dup # doesn't print anything
```
This makes the change in #17545 useless: while inheriting from core classes is indeed marginal, one of author's intention might be carrying additional information with the Proc instance, and bypassing `#initialize_dup` makes it impossible to maintain this information.
It seems that actually `#initialize_dup` is also invoked on the core classes themselves, but ignored on `Proc`.
```ruby
class Array
def initialize_dup(...)
p(self.class, ...)
super
end
end
class String
def initialize_dup(...)
p(self.class, ...)
super
end
end
class Proc
def initialize_dup(...)
p(self.class, ...)
super
end
end
'test'.dup # prints String, "test"
['test'].dup # prints Array, ["test"]
Proc.new { 'test' }.dup # doesn't print anything
```
Which is an even more marginal problem but still an inconsistency.
--
https://bugs.ruby-lang.org/
Issue #19430 has been reported by mame (Yusuke Endoh).
----------------------------------------
Feature #19430: Contribution wanted: DNS lookup by c-ares library
https://bugs.ruby-lang.org/issues/19430
* Author: mame (Yusuke Endoh)
* Status: Open
* Priority: Normal
----------------------------------------
## Problem
At the present time, Ruby uses `getaddrinfo(3)` to resolve names. Because this function is synchronous, we cannot interrupt the thread performing name resolution until the DNS server returns a response.
We can see this behavior by setting blackhole.webpagetest.org (72.66.115.13) as a DNS server, which swallows all packets, and resolving any name:
```
# cat /etc/resolv.conf
nameserver 72.66.115.13
# ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)'
^C^C^C^C
```
As we see, Ctrl+C does not stop ruby.
The current workaround that users can take is to do name resolution in a Ruby thread.
```ruby
Thread.new { Addrinfo.getaddrinfo("www.ruby-lang.org", 80) }.value
```
The thread that calls this code is interruptible. (Note that the newly created thread itself will be stuck until the DNS lookup exceeds the time out.)
## Proposal
We can solve this problem by using c-ares, which is an asynchronous name resolver, as a backend of `Addrinfo.getaddrinfo`, etc. (@sorah told me about this library, thanks!)
https://c-ares.org/
I have created a PoC patch.
https://github.com/mame/ruby/commit/547806146993bbc25984011d423dcc0f913b211c
By applying this patch, we can interrupt `Addrinfo.getaddrinfo` by Ctrl+C.
```
# cat /etc/resolv.conf
nameserver 72.66.115.13
# ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)'
^C-e:1:in `getaddrinfo': Interrupt
from -e:1:in `<main>'
```
## Discussion
### About c-ares
According to the site of c-ares, some major tools including libcurl, Wireshark, and Apache Arrow are already using c-ares. In the language interpreter, node.js seems to be using c-ares.
I am honestly not sure about the compatibility of c-ares with `getaddrinfo(3)`. I guess there is no major incompatibility because I have not experienced any name resolution problem of curl. @akr (who is the author and maintainer of Ruby's socket library) suggested to check if OS-specific name resolution, e.g., WINS on Windows, NIS on Solaris, etc., is supported. He also said that it may be acceptable even if they are not supported.
Whether to bundle c-ares source code with ruby would require further discussion. If this proposal is accepted, then c-ares will become a de facto essential dependency for practical use, like gmp, in my opinion. Incidentally, node.js bundles c-ares: https://github.com/nodejs/node/tree/main/deps/cares
### Alternative approaches
Recent glibc provides `getaddrinfo_a(3)` which performs asynchronous name resolution. However, this function has a fatal problem of being incompatible with `fork(2)`, which is heavily used in the Ruby ecosystem. In fact, the attempt to use `getaddrinfo_a(3)` (#17134) has been revert because it fails rails tests. (#17220)
Another alternative is to have a worker pthread inside Ruby that calls getaddrinfo(3). Instead of calling getaddrinfo(3) directly, `Addrinfo.getaddrinfo` would ask the worker to resolve a name and wait for a response. This method should be able to implement cancellation. (Simply put, this means reimplementation of getaddrinfo_a(3) on our own, taking into account of `fork(2).)
This has the advantages: not adding dependencies on external libraries and not having compatibility issues with `getaddrinfo(3)`. However, it is considerably more difficult to implement and maintain. An internal pthread may have a non-trivial impact on the execution efficiency and memory usage. Also, we may need to implement a mechanism to dynamically change the number of workers depending on the load.
It would be ideal if we could try and evaluate both approaches. But my current impression is that using c-ares is the quickest and best compromise.
## Contribution wanted
I have made it up to the PoC, but don't have much time to complete this. @naruse suggested me to create a ticket asking for contributions. Is anyone interested in this?
* This patch changes `rsock_getaddrinfo` to accept a timeout argument. There are several places where Qnil is passed as a timeout (where I add `// TODO` in the PoC). We need to consider what timeout we should pass.
* This cares only `getaddrinfo`, but we also need to care `getnameinfo` (and something else if any). There may be some issues I'm not aware of.
* I have not yet tested this PoC seriously. It would be great if we could evaluate it with some real apps.
Also, it would be great to hear from someone who knows more about c-ares.
--
https://bugs.ruby-lang.org/
Issue #19288 has been reported by maciej.mensfeld (Maciej Mensfeld).
----------------------------------------
Bug #19288: Ractor JSON parsing significantly slower than linear parsing
https://bugs.ruby-lang.org/issues/19288
* Author: maciej.mensfeld (Maciej Mensfeld)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
a simple benchmark:
```ruby
require 'json'
require 'benchmark'
CONCURRENT = 5
RACTORS = true
ELEMENTS = 100_000
data = CONCURRENT.times.map do
ELEMENTS.times.map do
{
rand => rand,
rand => rand,
rand => rand,
rand => rand
}.to_json
end
end
ractors = CONCURRENT.times.map do
Ractor.new do
Ractor.receive.each { JSON.parse(_1) }
end
end
result = Benchmark.measure do
if RACTORS
CONCURRENT.times do |i|
ractors[i].send(data[i], move: false)
end
ractors.each(&:take)
else
# Linear without any threads
data.each do |piece|
piece.each { JSON.parse(_1) }
end
end
end
puts result
```
Gives following results on my 8 core machine:
```shell
# without ractors:
2.731748 0.003993 2.735741 ( 2.736349)
# with ractors
12.580452 5.089802 17.670254 ( 5.209755)
```
I would expect Ractors not to be two times slower on the CPU intense work.
--
https://bugs.ruby-lang.org/
Issue #19297 has been reported by vo.x (Vit Ondruch).
----------------------------------------
Bug #19297: Don't download content from internet to execute Ruby test suite
https://bugs.ruby-lang.org/issues/19297
* Author: vo.x (Vit Ondruch)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Trying to build Ruby 3.2.0 for Fedora and execute its test suite via `make check` as we always did [1], the test suite suddenly fails (while it was working with commit:git|c5eefb7f37):
~~~
... snip ...
C-API Util function ruby_strtod
- converts a string to a double and returns the remaining string
- returns 0 and the full string if there's no numerical value
Finished in 45.737677 seconds
3827 files, 31635 examples, 177877 expectations, 0 failures, 0 errors, 0 tagged
./miniruby -I/builddir/build/BUILD/ruby-3.2.0/lib -I. -I.ext/common /builddir/build/BUILD/ruby-3.2.0/tool/runruby.rb --extout=.ext -- --disable-gems -C "/builddir/build/BUILD/ruby-3.2.0" bin/gem install --no-document \
--install-dir .bundle --conservative "bundler" "rake" "rspec:~> 3" #"ruby-prof"
ERROR: Could not find a valid gem 'bundler' (>= 0), here is why:
Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz)
ERROR: Could not find a valid gem 'rspec' (~> 3), here is why:
Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz)
make: Leaving directory '/builddir/build/BUILD/ruby-3.2.0/redhat-linux-build'
make: *** [uncommon.mk:1464: yes-test-syntax-suggest-prepare] Error 2
~~~
This is obviously due to the test suite trying to download `rspec` from the internet, while Fedora builders does not have internet access (and won't ever have for security reasons). If I am not mistaken, this is caused by commit:git|cae53842735237ccf71a13873fd0d1ae7f165582. Now
1) Can this be fixed?
2) Can the tarball be always self contained?
[1]: https://src.fedoraproject.org/rpms/ruby/blob/631163e3b8a51ed610528181aabe0d…
--
https://bugs.ruby-lang.org/
Issue #19281 has been reported by tompng (tomoya ishida).
----------------------------------------
Bug #19281: SyntaxError if first argument of command call has semicolon inside parenthesis
https://bugs.ruby-lang.org/issues/19281
* Author: tompng (tomoya ishida)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
These are syntax error
~~~ruby
p (1;2),(3),(4)
p (;),(),()
a.b (1;2),(3),(4)
a.b (;),(),()
~~~
I expect it to be syntax ok because the code below is syntax ok.
~~~ruby
p (1),(2;3),(4;5)
p (),(;),(;)
a.b (1),(2;3),(4;5)
a.b (),(;),(;)
~~~
It will be easy to traverse sexp if the sexp of first argument is same as others
~~~ruby
Ripper.sexp "p (),(),()"
# =>
[:program,
[[:command,
[:@ident, "p", [1, 0]],
[:args_add_block,
[[:paren, false], # [:paren, [[:void_stmt]]]
[:paren, [[:void_stmt]]],
[:paren, [[:void_stmt]]]],
false]]]]
Ripper.sexp "p (1),(2),(3)"
# =>
[:program,
[[:command,
[:@ident, "p", [1, 0]],
[:args_add_block,
[[:paren, [:@int, "1", [1, 3]]], # [:paren, [[:@int, "1", [1, 3]]]]
[:paren, [[:@int, "2", [1, 7]]]],
[:paren, [[:@int, "3", [1, 11]]]]],
false]]]]
~~~
--
https://bugs.ruby-lang.org/
Issue #19230 has been reported by mame (Yusuke Endoh).
----------------------------------------
Bug #19230: The openssl backend of securerandom is no longer needed
https://bugs.ruby-lang.org/issues/19230
* Author: mame (Yusuke Endoh)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
securerandom first checks if Random.urandom is available ([Line 77](https://github.com/ruby/securerandom/blob/5bfe7d6c163f7a8a45af8d2fc377f…), and if not available, it uses the openssl backend as a degeneration.
However, the openssl backend does not work because it internally uses Random.urandom ([Line 55](https://github.com/ruby/securerandom/blob/5bfe7d6c163f7a8a45af8d2fc377f…) to create a seed.
This issue is found by @hanachin.
```
$ ruby -ve 'def Random.urandom(*); raise; end; require "securerandom"; p SecureRandom.bytes(10)'
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
-e:1: warning: method redefined; discarding old urandom
-e:1:in `urandom': unhandled exception
from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:75:in `singleton class'
from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:42:in `<module:SecureRandom>'
from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:41:in `<top (required)>'
from <internal:/home/mame/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:/home/mame/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from -e:1:in `<main>'
```
There has been this bug since commit:abae70d6ed63054d7d01bd6cd80c1b5b98b93ba3, which made the urandom backend as default and left the openssl backend just for degeneration. I think no one need the openssl anymore because no one has reported this bug for such a long time.
How about removing it?
```diff
diff --git a/lib/securerandom.rb b/lib/securerandom.rb
index 07ae048634..32b76a2137 100644
--- a/lib/securerandom.rb
+++ b/lib/securerandom.rb
@@ -14,7 +14,6 @@
#
# It supports the following secure random number generators:
#
-# * openssl
# * /dev/urandom
# * Win32
#
@@ -46,21 +45,6 @@ def bytes(n)
private
- def gen_random_openssl(n)
- @pid = 0 unless defined?(@pid)
- pid = $$
- unless @pid == pid
- now = Process.clock_gettime(Process::CLOCK_REALTIME, :nanosecond)
- OpenSSL::Random.random_add([now, @pid, pid].join(""), 0.0)
- seed = Random.urandom(16)
- if (seed)
- OpenSSL::Random.random_add(seed, 16)
- end
- @pid = pid
- end
- return OpenSSL::Random.random_bytes(n)
- end
-
def gen_random_urandom(n)
ret = Random.urandom(n)
unless ret
@@ -77,13 +61,7 @@ def gen_random_urandom(n)
Random.urandom(1)
alias gen_random gen_random_urandom
rescue RuntimeError
- begin
- require 'openssl'
- rescue NoMethodError
- raise NotImplementedError, "No random device"
- else
- alias gen_random gen_random_openssl
- end
+ raise NotImplementedError, "No random device"
end
public :gen_random
```
--
https://bugs.ruby-lang.org/
Issue #19436 has been reported by byroot (Jean Boussier).
----------------------------------------
Bug #19436: Call Cache for singleton methods can lead to "memory leaks"
https://bugs.ruby-lang.org/issues/19436
* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Using "memory leaks" with quotes, because strictly speaking the memory isn't leaked, but it can nonetheless lead to large memory overheads.
### Minimal Reproduction
```ruby
module Foo
def bar
end
end
def call_bar(obj)
# Here the call cache we'll keep a ref on the method_entry
# which then keep a ref on the singleton_class, making that
# instance immortal until the method is called again with
# another instance.
# The reference chain is IMEMO(callcache) -> IMEMO(ment) -> ICLASS -> CLASS(singleton) -> OBJECT
obj.bar
end
obj = Object.new
obj.extend(Foo)
call_bar(obj)
id = obj.object_id
obj = nil
4.times { GC.start }
p ObjectSpace._id2ref(id)
```
### Explanation
Call caches keep a strong reference onto the "callable method entry" (CME), which itself keeps a strong reference on the called object class
and in the cache of a singleton class, it keeps a strong reference onto the `attached_object` (instance).
This means that any call site that calls a singleton method, will effectively keep a strong reference onto the last receiver.
If the method is frequently called it's not too bad, but if it's infrequently called, it's effectively a (bounded) memory leak.
And if the `attached_object` is big, the wasted memory can be very substantial.
### Practical Implications
Once relative common API impacted by this is [Rails' `extending` API](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#met…. This API allow to extend a "query result set" with a module.
These query results set can sometimes be very big, especially since they keep references to the instantiated `ActiveRecord::Base` instances etc.
### Possible Solutions
#### Only keep a weak reference to the CME
The fairly "obvious" solution is to keep a weak reference to the CME, that's what I explored in https://github.com/ruby/ruby/pull/7272, and it seems to work.
However in debug mode It does fail on an assertion during compaction, but it's isn't quite clear to me what the impact is.
Additionally, something that makes me think this would be the right solution, is that call caches already try to avoid marking the class:
```c
# vm_callinfo.h:275
struct rb_callcache {
const VALUE flags;
/* inline cache: key */
const VALUE klass; // should not mark it because klass can not be free'd
// because of this marking. When klass is collected,
// cc will be cleared (cc->klass = 0) at vm_ccs_free().
```
So it appears that the class being also marked through the CME is some kind of oversight?
#### Don't cache based on some heuristics
If the above isn't possible or too complicated, an alternative would be to not cache CMEs found in singleton classes, except if it's the the singleton class of a `Class` or `Module`.
It would make repeated calls to such methods slower, but the assumption is that it's unlikely that these CME would live very long.
#### Make `Class#attached_object` a weak reference
Alternatively we could make the `attached_object` a weak reference, which would drastically limit the amount of memory that may be leaked in such scenario.
The downside is that `Class#attached_object` was very recently exposed in Ruby 3.2.0, so it means changing its semantic a bit.
cc @peterzhu2118 @ko1
--
https://bugs.ruby-lang.org/
Issue #19325 has been reported by dsisnero (Dominic Sisneros).
----------------------------------------
Bug #19325: Windows support lacking.
https://bugs.ruby-lang.org/issues/19325
* Author: dsisnero (Dominic Sisneros)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Ruby's support on windows has always been second class. With some of the recent decisions, windows support is falling even more behind. Recent developments in mjit and yjit that exclude windows are two glaring issues that should be corrected. Googling 'percent of windows vs other operating systems' and it shows windows has a share of 76%. Ceding that users to python and other programming languages has to be one of the reasons python continues get more market share from ruby. With rust having first class windows support and threading support, is there a reason why yjit is not able to work on windows? Also, windows compiler support has matured enough and vcpkg support has evolved enough that it seems it should be possible to finally get a ruby version without having to use msys2. Even Crystal language has a version that runs on windows without needing msys2.
--
https://bugs.ruby-lang.org/