Issue #18814 has been updated by shyouhei (Shyouhei Urabe).
I heard from @ko1 in person that _he_ doesn't need this feature to implement his ractor pool. From what I understand he would have a single ruby-level Queue instance for all incoming messages and instead of pushing those messages to each ractors one by one, he would let everyone pull messages from that queue at once.
But I might have missed his details. @ko1 please explain your detailed reason why golang's `len()` is not a good idea.
----------------------------------------
Feature #18814: Ractor: add method to query incoming message queue size
https://bugs.ruby-lang.org/issues/18814#change-101060
* Author: phigrofi (Philipp Großelfinger)
* Status: Open
* Priority: Normal
----------------------------------------
## Abstract
A simple method to query the current size of a Ractor's incoming queue from outside. Can be used to decide on the sender's side if a message is sent or postponed.
## Background
Ractors have an infinite incoming message queue. When messages are sent to a Ractor it is not possible to check the current count of elements in the queue. A workaround would be: The receiving Ractor could immediately accept each message and put them into a separate queue and keep track of their count. Then the sending Ractor could query the count from the receiving Ractor as a message.
While this message exchange would be short and simple, it still requires the receiving Ractor to process the "queue-count" message and respond to it.
## Proposal
The Ractor implementation already keeps track of the current incoming message fill level in the field `sync.incoming_queue.cnt`. A simple method in the ruby code of Ractor could expose this number so that it is simple to query the queue size from outside. This works without any interaction of the queried Ractor.
The code would work as follows:
``` ruby
ractor = Ractor.new do
loop { sleep(1) }
end
ractor.queue_size #=> 0
ractor << "message"
ractor.queue_size #=> 1
ractor << "message"
ractor.queue_size #=> 2
```
## Use cases
1. Avoid queue overflow by checking queue size from outside before sending further messages.
2. Incoming queue sizes can be monitored.
## Discussion
The proposal makes it much easier to prevent overflow of a message queue than managing a separate queue inside of a Ractor and keeping track of its element count. I think also having a separate queue where the count needs to communicated, ignores the concept of a Ractor's incoming message queue and makes it quite complicated.
## See also
In this [issue](https://bugs.ruby-lang.org/issues/17679) a middleman solution was proposed which keeps track of a separate queue count.
--
https://bugs.ruby-lang.org/
Issue #19275 has been reported by xtkoba (Tee KOBAYASHI).
----------------------------------------
Bug #19275: Bundled gems extensions are not installed with 3.2.0 release tarball
https://bugs.ruby-lang.org/issues/19275
* Author: xtkoba (Tee KOBAYASHI)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [aarch64-linux-android]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
Bundled gems extensions are not installed with 3.2.0 release tarball when cross building for `aarch64-linux-android` target.
Possibly related to #19271.
Excerpt from build log:
```
2022-12-28T12:39:39.5065133Z linking shared-object rbs_extension.so
2022-12-28T12:39:39.5317220Z Successfully remade target file '../../../../../.bundle/extensions/aarch64-linux-android/3.2.0/rbs-2.8.2/rbs_extension.so'.
[...]
2022-12-28T12:40:06.9333744Z rm -rf .bundle/extensions/aarch64-linux-android/
```
Seems like bundled gems extensions are built but removed afterwards.
The workaround we took is to patch `common.mk` so that `outdate-bundled-gems` is not triggered:
```patch
--- a/common.mk
+++ b/common.mk
@@ -1375,7 +1375,6 @@
refresh-gems: update-bundled_gems prepare-gems
prepare-gems: $(HAVE_BASERUBY:yes=update-gems) $(HAVE_BASERUBY:yes=extract-gems)
-prepare-gems: $(DOT_WAIT) $(HAVE_BASERUBY:yes=outdate-bundled-gems)
extract-gems: $(HAVE_BASERUBY:yes=update-gems)
update-gems$(gnumake:yes=-sequential): PHONY
```
Full build log is attached.
---Files--------------------------------
build.log.xz (245 KB)
--
https://bugs.ruby-lang.org/
Issue #18518 has been updated by Eregon (Benoit Daloze).
This is still not fixed.
I think CRuby should check if RHS is bigger than 2**31 and if so raise an exception immediately instead of taking a lot of time and run into OOM:
https://github.com/ruby/ruby/commit/5df711844586312891bb466dbc72265490488d9…
----------------------------------------
Bug #18518: NoMemoryError + [FATAL] failed to allocate memory for twice 1 << large
https://bugs.ruby-lang.org/issues/18518#change-101055
* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Repro:
```ruby
exp = 2**40 # also fails with bignum e.g. 2**64
def exc
begin
yield
rescue NoMemoryError => e
p :NoMemoryError
end
end
p exp
exc { (1 << exp) }
exc { (-1 << exp) }
exc { (bignum_value << exp) }
exc { (-bignum_value << exp) }
```
Output:
```
$ ruby -v mri_oom.rb
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
mri_oom.rb:7: warning: assigned but unused variable - e
1099511627776
:NoMemoryError
[FATAL] failed to allocate memory
```
3.1.0 seems fine:
```
$ ruby -v mri_oom.rb
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-linux]
mri_oom.rb:7: warning: assigned but unused variable - e
1099511627776
:NoMemoryError
:NoMemoryError
:NoMemoryError
:NoMemoryError
```
--
https://bugs.ruby-lang.org/
Issue #18285 has been updated by zverok (Victor Shepelev).
> The vast majority of classes out there don't define a custom inspect. So Ruby should do something reasonable for these cases.
That's true.
(I still maintain that it could/should be a good community practice to define better `#inspect`s for better debuggability, because eventual `p self` in pry session of a controller is painful as well, but that's a different question.)
----------------------------------------
Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call
https://bugs.ruby-lang.org/issues/18285#change-101054
* Author: ivoanjo (Ivo Anjo)
* Status: Open
* Priority: Normal
----------------------------------------
Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>.
### Background
While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception.
### How to reproduce
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'benchmark-ips'
end
puts RUBY_DESCRIPTION
class GemInformation
# ...
def get_no_method_error
method_does_not_exist
rescue => e
e
end
def get_runtime_error
raise 'Another Error'
rescue => e
e
end
def inspect # <-- expensive method gets called when calling NoMethodError#message
Gem::Specification._all.inspect
end
end
NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error
RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message }
x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message }
x.compare!
end
```
### Expectation and result
Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception.
In reality:
```
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
no method error message cost
115.390 (± 1.7%) i/s - 580.000 in 5.027822s
runtime error message cost
6.938M (± 0.5%) i/s - 35.334M in 5.092617s
Comparison:
runtime error message cost: 6938381.6 i/s
no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower
```
### Suggested solutions
1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>)
2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1.
--
https://bugs.ruby-lang.org/
Issue #18285 has been updated by byroot (Jean Boussier).
> Note that in the example provided in the original issue
I just realized my response was confusing. To clarify I meant `ActionController::Base#inspect`. Which resolve to `Object#inspect`. Controllers simply are the root of a very large object graph.
----------------------------------------
Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call
https://bugs.ruby-lang.org/issues/18285#change-101053
* Author: ivoanjo (Ivo Anjo)
* Status: Open
* Priority: Normal
----------------------------------------
Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>.
### Background
While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception.
### How to reproduce
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'benchmark-ips'
end
puts RUBY_DESCRIPTION
class GemInformation
# ...
def get_no_method_error
method_does_not_exist
rescue => e
e
end
def get_runtime_error
raise 'Another Error'
rescue => e
e
end
def inspect # <-- expensive method gets called when calling NoMethodError#message
Gem::Specification._all.inspect
end
end
NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error
RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message }
x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message }
x.compare!
end
```
### Expectation and result
Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception.
In reality:
```
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
no method error message cost
115.390 (± 1.7%) i/s - 580.000 in 5.027822s
runtime error message cost
6.938M (± 0.5%) i/s - 35.334M in 5.092617s
Comparison:
runtime error message cost: 6938381.6 i/s
no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower
```
### Suggested solutions
1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>)
2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1.
--
https://bugs.ruby-lang.org/
Issue #18285 has been updated by byroot (Jean Boussier).
> What if #inspect is very inefficient? In this case a) actually, yes, I believe that it is client's code responsibility
Note that in the example provided in the original issue, the slow `#inspect` is simply the default `Object#inspect`.
The vast majority of classes out there don't define a custom inspect. So Ruby should do something reasonable for these cases.
----------------------------------------
Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call
https://bugs.ruby-lang.org/issues/18285#change-101052
* Author: ivoanjo (Ivo Anjo)
* Status: Open
* Priority: Normal
----------------------------------------
Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>.
### Background
While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception.
### How to reproduce
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'benchmark-ips'
end
puts RUBY_DESCRIPTION
class GemInformation
# ...
def get_no_method_error
method_does_not_exist
rescue => e
e
end
def get_runtime_error
raise 'Another Error'
rescue => e
e
end
def inspect # <-- expensive method gets called when calling NoMethodError#message
Gem::Specification._all.inspect
end
end
NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error
RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message }
x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message }
x.compare!
end
```
### Expectation and result
Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception.
In reality:
```
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
no method error message cost
115.390 (± 1.7%) i/s - 580.000 in 5.027822s
runtime error message cost
6.938M (± 0.5%) i/s - 35.334M in 5.092617s
Comparison:
runtime error message cost: 6938381.6 i/s
no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower
```
### Suggested solutions
1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>)
2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1.
--
https://bugs.ruby-lang.org/
Issue #18285 has been updated by zverok (Victor Shepelev).
@byroot I am not denying the problem exists or saying it shouldn't be handled (and I acknowledge my Rails remark was inappropriate).
I am just saying that the way of fixing it by replacing the object representation with class name breaks many good use cases and, in general, seems less friendly to me.
It always felt nice and informative.
Interesting to note that the latest Ruby releases tend to add _more_ context for errors (with `did_you_mean` and `error_highlight`), and it is considered a good thing, while this change seems to go in the exactly opposite direction.
Actually, thinking a bit more about this, maybe a good compromise solution would be to preserve this change but add `#inspect` of the object to `NoMethodError#detailed_message`?.. It seems to be introduced exactly to handle the balance between the efficiency/brevity of the "core" message and helpfulness.
As for the problems which `#inspect` in (any) error message leads to, I believe there are two related but different ones?
1. What if `#inspect` output is too long? Intuitively, I would say that there are many techniques of mechanical shortening of the string to the desired limit. Say, modern IRB handles it quite gracefully (this is a copy-paste of FULL IRB session, the `...` is what it did):
```
>> ary = [{a: 1, foo: :bar}] * 100
=>
[{:a=>1, :foo=>:bar},
...
```
2. What if `#inspect` is very inefficient? In this case a) actually, yes, I believe that it is client's code responsibility (and it is not "sharp knife"-class responsibility, it is rather "if you've implemented `to_s` to return a number, nothing would work properly"), but b) maybe the `detailed_message` idea helps mitigating this, anyway?..
----------------------------------------
Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call
https://bugs.ruby-lang.org/issues/18285#change-101051
* Author: ivoanjo (Ivo Anjo)
* Status: Open
* Priority: Normal
----------------------------------------
Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>.
### Background
While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception.
### How to reproduce
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'benchmark-ips'
end
puts RUBY_DESCRIPTION
class GemInformation
# ...
def get_no_method_error
method_does_not_exist
rescue => e
e
end
def get_runtime_error
raise 'Another Error'
rescue => e
e
end
def inspect # <-- expensive method gets called when calling NoMethodError#message
Gem::Specification._all.inspect
end
end
NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error
RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message }
x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message }
x.compare!
end
```
### Expectation and result
Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception.
In reality:
```
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
no method error message cost
115.390 (± 1.7%) i/s - 580.000 in 5.027822s
runtime error message cost
6.938M (± 0.5%) i/s - 35.334M in 5.092617s
Comparison:
runtime error message cost: 6938381.6 i/s
no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower
```
### Suggested solutions
1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>)
2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1.
--
https://bugs.ruby-lang.org/
Issue #19150 has been reported by Eregon (Benoit Daloze).
----------------------------------------
Bug #19150: pack/unpack silently ignores unknown directives
https://bugs.ruby-lang.org/issues/19150
* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
But I believe it should be an error instead.
Typically when a parser sees a syntax error it should fail not continue silently.
For instance `[1].pack('<L')` succeeds and only emits a warning if `$VERBOSE` is true.
This behavior caused confusion in https://github.com/oracle/truffleruby/issues/2791
I think it should fail with an `ArgumentError` instead.
Extracted from https://bugs.ruby-lang.org/issues/19108#note-3
--
https://bugs.ruby-lang.org/
Issue #18285 has been updated by byroot (Jean Boussier).
> As far as I understand, the problem originates in Rails, which (as they do) doesn't try to follow good practices for #inspect (which is honestly frequently irritating)
This absolutely isn't specific to Rails. It can happen with any object that has a large number of reference.
Try your `transform_keys` example on a large API payload you received and you'll produce a gigantic error message.
"People are holding it wrong" isn't a proper way to address issues.
----------------------------------------
Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call
https://bugs.ruby-lang.org/issues/18285#change-101041
* Author: ivoanjo (Ivo Anjo)
* Status: Open
* Priority: Normal
----------------------------------------
Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>.
### Background
While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception.
### How to reproduce
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'benchmark-ips'
end
puts RUBY_DESCRIPTION
class GemInformation
# ...
def get_no_method_error
method_does_not_exist
rescue => e
e
end
def get_runtime_error
raise 'Another Error'
rescue => e
e
end
def inspect # <-- expensive method gets called when calling NoMethodError#message
Gem::Specification._all.inspect
end
end
NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error
RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message }
x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message }
x.compare!
end
```
### Expectation and result
Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception.
In reality:
```
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
no method error message cost
115.390 (± 1.7%) i/s - 580.000 in 5.027822s
runtime error message cost
6.938M (± 0.5%) i/s - 35.334M in 5.092617s
Comparison:
runtime error message cost: 6938381.6 i/s
no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower
```
### Suggested solutions
1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>)
2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1.
--
https://bugs.ruby-lang.org/