January 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111934] [Ruby master Bug#18381] Default vs Bundled gems
by hsbt (Hiroshi SHIBATA) 20 Jan '23

20 Jan '23

Issue #18381 has been updated by hsbt (Hiroshi SHIBATA). I'll document this. ---------------------------------------- Bug #18381: Default vs Bundled gems https://bugs.ruby-lang.org/issues/18381#change-101362 * Author: vo.x (Vit Ondruch) * Status: Assigned * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) * ruby -v: ruby 3.1.0dev (2021-12-01 master 0b53a8895f) [x86_64-linux] * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- Could there be documented what is the difference between default and bundled gems? E.g. * Are the Bundled gems considered a part of StdLib or not? * What is general strategy for the future, seeing more and more gems moving from default to bundled gems in Ruby 3.1 * Default gems does not need to be listed in Gemfile where Bundled gems needs to be listed in Gemfile * Default gems can be used without RubyGems where Bundled gems requires RubyGems -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111807] [Ruby master Bug#19337] What is the difference between "default" gems and "bundled" gems?
by zzak (Zak Scott) 20 Jan '23

20 Jan '23

Issue #19337 has been reported by zzak (Zak Scott). ---------------------------------------- Bug #19337: What is the difference between "default" gems and "bundled" gems? https://bugs.ruby-lang.org/issues/19337 * Author: zzak (Zak Scott) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Although I found [some answers](https://nts.strzibny.name/ruby-stdlib-default-bundled-gems/) on the [internet](https://stdgems.org), it seems like this information should be available from Ruby itself, if it's not already. Perhaps this is something we could add here? Open to other suggestions. https://github.com/ruby/ruby/blob/master/doc/standard_library.rdoc That blog pretty much answers the questions that I had fwiw. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:111917] [Ruby master Bug#19356] Backport f7b72462aa27716370c6bea1f2c240983aca9a55
by shugo (Shugo Maeda) 20 Jan '23

20 Jan '23

Issue #19356 has been reported by shugo (Shugo Maeda). ---------------------------------------- Bug #19356: Backport f7b72462aa27716370c6bea1f2c240983aca9a55 https://bugs.ruby-lang.org/issues/19356 * Author: shugo (Shugo Maeda) * Status: Open * Priority: Normal * Assignee: naruse (Yui NARUSE) * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- Please backport [f7b72462aa27716370c6bea1f2c240983aca9a55](https://github.com/ruby/ruby/comm…. In Feature #19314, we concluded that the return value of String#bytesplice should be changed from the source string to the receiver, because the source string is useless and confusing when extra arguments are added. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111522] [Ruby master Bug#19286] What should kwargs' arity be?
by matsuda (Akira Matsuda) 20 Jan '23

20 Jan '23

Issue #19286 has been reported by matsuda (Akira Matsuda). ---------------------------------------- Bug #19286: What should kwargs' arity be? https://bugs.ruby-lang.org/issues/19286 * Author: matsuda (Akira Matsuda) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2022-12-28T16:43:05Z master cada537040) +YJIT [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello, guys. It's time for a quick Ruby quiz. Q: What is this method's arity? def f(a:, b:) end It requires two arguments, hence it should be 2? Or if we call this method with one argument, the error message says "wrong number of arguments (given 1, expected 0; required keywords: a, b) (ArgumentError)", which means the arity is 0, maybe? A: The answer is, $ all-ruby -e 'p method(def f(a:, b:) end).arity' ruby-2.1.0-preview1 0 ... ruby-2.1.0 0 ruby-2.1.1 -1 ruby-2.1.2 1 ... ruby-3.1.0 1 it's been 1 since 2.1.2. But why 1? Why not 2 nor 0? I asked this question to the ruby-core people, and ko1's answer was that even he has no idea what the number 1 means. ![](random_-_ruby-lang_-_Slack.png) So I thought it'd be worth asking this question here. ---Files-------------------------------- random_-_ruby-lang_-_Slack.png (40.1 KB) -- https://bugs.ruby-lang.org/

5 5

[ruby-core:111308] [Ruby master Bug#19237] Hash default_proc is not thread-safe to lazy-initialize value for a given key
by Eregon (Benoit Daloze) 20 Jan '23

20 Jan '23

Issue #19237 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Bug #19237: Hash default_proc is not thread-safe to lazy-initialize value for a given key https://bugs.ruby-lang.org/issues/19237 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- ```ruby 1000.times do h = Hash.new do |hash, key| # Note, with [] instead of Array.new below is seems to work # but then e.g. anything else in this block like Thread.pass also breaks it hash[key] = Array.new end go = false threads = 100.times.map do Thread.new do Thread.pass until go h[:na] << true end end go = true threads.each(&:join) raise "Expected 100 elements but was #{h[:na].size}" if h[:na].size != 100 end ``` gives (in 3 runs): ``` concurrent_hash_default_proc.rb:17:in `block in <main>': Expected 100 elements but was 1 (RuntimeError) concurrent_hash_default_proc.rb:17:in `block in <main>': Expected 100 elements but was 3 (RuntimeError) concurrent_hash_default_proc.rb:17:in `block in <main>': Expected 100 elements but was 2 (RuntimeError) ``` So what happens is the same Hash entry gets assigned multiple times, which feels quite unexpected and cause fairly surprising bugs (e.g. elements "disappearing" etc). Any thoughts on how to solve that in CRuby? In my PhD thesis one way I found is to actually pass a different object than the Hash itself as the first argument to the block. See https://eregon.me/blog/assets/research/thesis-thread-safe-data-representati… page 83 `Idiomatic Concurrent Hash Operations`. In short, it replaces `[]=` calls in the initializer block with `put_if_absent` by passing a different object than the Concurrent::Hash itself, which overrides `[]=` and delegates the rest. #19069 could be another way but there are compatibility issues (e.g. storing when it previously would not for `Hash.new { |h,k| k * 3 }`. There is also the question whether the block should be allowed to execute more than once for a given key. I think that is difficult to solve (probably the only way is via some lock, but then that can lead to deadlocks), and less important than ensuring the value is only assigned once for a given key. --- Note that concurrent-ruby has the same issue because it uses ::Array/::Hash on CRuby: https://github.com/ruby-concurrency/concurrent-ruby/issues/970#issuecomment… There is also more discussions about trade-offs there which apply to `Hash` too. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111921] [Ruby master Bug#18518] NoMemoryError + [FATAL] failed to allocate memory for twice 1 << large
by mame (Yusuke Endoh) 20 Jan '23

20 Jan '23

Issue #18518 has been updated by mame (Yusuke Endoh). Discussed at the dev meeting. Are there any real-world use cases (other than rubyspec) where you would like to prohibit `integer << large`? @matz was initially positive about prohibiting huge object generation as early failure. However, there was little reason to prohibit only `Integer#<<`, so he considered to prohibit any Bignum generation larger than a threshold size, such as Integer#*. However, there was little reason to prohibit only Bignum, so he wanted that Array and String should be prohibited from generating objects larger than the threshold size. However, we sometimes use File.read with a file larger than 2GB. We reached that it would be difficult to determine a reasonable size threshold. The discussion ran out of time here. ---------------------------------------- Bug #18518: NoMemoryError + [FATAL] failed to allocate memory for twice 1 << large https://bugs.ruby-lang.org/issues/18518#change-101344 * Author: Eregon (Benoit Daloze) * Status: Rejected * Priority: Normal * ruby -v: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Repro: ```ruby exp = 2**40 # also fails with bignum e.g. 2**64 def exc begin yield rescue NoMemoryError => e p :NoMemoryError end end p exp exc { (1 << exp) } exc { (-1 << exp) } exc { (bignum_value << exp) } exc { (-bignum_value << exp) } ``` Output: ``` $ ruby -v mri_oom.rb ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] mri_oom.rb:7: warning: assigned but unused variable - e 1099511627776 :NoMemoryError [FATAL] failed to allocate memory ``` 3.1.0 seems fine: ``` $ ruby -v mri_oom.rb ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-linux] mri_oom.rb:7: warning: assigned but unused variable - e 1099511627776 :NoMemoryError :NoMemoryError :NoMemoryError :NoMemoryError ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111920] [Ruby master Feature#18285] NoMethodError#message uses a lot of CPU/is really expensive to call
by mame (Yusuke Endoh) 20 Jan '23

20 Jan '23

Issue #18285 has been updated by mame (Yusuke Endoh). Discussed at the dev meeting. @matz is positive about the change and said he would give it a try. The message format is slightly changed as follows: * If the receiver is a Class, `... for class <class name>` should be printed. * If the receiver is a Module, `... for module <module name>` should be printed. * If the receiver is an extended object (i.e., has a singleton class), `any_to_s` should be used. * Otherwise, `... for an instance of <class name>` should be printed. Examples: ``` 42.time #=> undefined method `time' for an instance of Integer (NoMethodError) class Foo privatee #=> undefined local variable or method 'privatee' for class Foo (NoMethodError) end def (o=Object.new).foo end o.bar #=> undefined method `bar' for #<Object: 0xdeadbeef(any_to_s)> (NoMethodError) ``` I'll update my PR later. When we use the new message format and find it inconvenient, we will adjust it accordingly. ---------------------------------------- Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call https://bugs.ruby-lang.org/issues/18285#change-101343 * Author: ivoanjo (Ivo Anjo) * Status: Open * Priority: Normal ---------------------------------------- Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>. ### Background While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception. ### How to reproduce ```ruby require 'bundler/inline' gemfile do source 'https://rubygems.org' gem 'benchmark-ips' end puts RUBY_DESCRIPTION class GemInformation # ... def get_no_method_error method_does_not_exist rescue => e e end def get_runtime_error raise 'Another Error' rescue => e e end def inspect # <-- expensive method gets called when calling NoMethodError#message Gem::Specification._all.inspect end end NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error Benchmark.ips do |x| x.config(:time => 5, :warmup => 2) x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message } x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message } x.compare! end ``` ### Expectation and result Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception. In reality: ``` ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] no method error message cost 115.390 (± 1.7%) i/s - 580.000 in 5.027822s runtime error message cost 6.938M (± 0.5%) i/s - 35.334M in 5.092617s Comparison: runtime error message cost: 6938381.6 i/s no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower ``` ### Suggested solutions 1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>) 2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111339] [Ruby master Feature#19245] Strict mode for Array#pack that doesn't silently truncate numbers that are too large for the given directive
by byroot (Jean Boussier) 20 Jan '23

20 Jan '23

Issue #19245 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19245: Strict mode for Array#pack that doesn't silently truncate numbers that are too large for the given directive https://bugs.ruby-lang.org/issues/19245 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- ```ruby >> [256].pack("C").unpack1("C") => 0 >> [257].pack("C").unpack1("C") => 1 ``` This is specified: ```ruby it "encodes the least significant 32 bits of a negative number" do [ [[-0x0000_0021], "\xdf\xff\xff\xff"], [[-0x0000_4321], "\xdf\xbc\xff\xff"], [[-0x0065_4321], "\xdf\xbc\x9a\xff"], [[-0x7865_4321], "\xdf\xbc\x9a\x87"] ].should be_computed_by(:pack, pack_format()) end ``` But not documented in `Array#pack`. I think that in many case this may lead to silent bugs. ### Possible solutions We could have a strict version of `pack`, either `pack(template, strict: true)` or `pack!(template)`. Or alternatively if we think this is never a desired behavior, we could change `pack` to first warn on truncation and later raise. -- https://bugs.ruby-lang.org/

5 7

[ruby-core:111746] [Ruby master Bug#19327] Backport commits 3be2acfafd3b3c6168e2266c7c6561d143d7ae5c and 9726736006b3f74635fd8af05814fe0908e2cf84
by peterzhu2118 (Peter Zhu) 20 Jan '23

20 Jan '23

Issue #19327 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19327: Backport commits 3be2acfafd3b3c6168e2266c7c6561d143d7ae5c and 9726736006b3f74635fd8af05814fe0908e2cf84 https://bugs.ruby-lang.org/issues/19327 * Author: peterzhu2118 (Peter Zhu) * Status: Closed * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- Please backport commits [3be2acfafd3b3c6168e2266c7c6561d143d7ae5c](https://github.com/ruby/ruby/comm… and [9726736006b3f74635fd8af05814fe0908e2cf84](https://github.com/ruby/ruby/comm…. This commits fixes a bug where string objects are not correctly re-embedding during GC compaction. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111915] [Ruby master Bug#15507] Thread#raise is delivered to GC context
by kjtsanaktsidis (KJ Tsanaktsidis) 19 Jan '23

19 Jan '23

Issue #15507 has been updated by kjtsanaktsidis (KJ Tsanaktsidis). ❤️ Thank you for fixing this @jeremyevans - I was debugging some instances of our Ruby 2.7 app's test suite hanging on exit, and narrowed it down to this reproduction: ``` require 'tempfile' def forget_about_a_tempfile Tempfile.new('garbage') end def tempfile_making_thread loop do forget_about_a_tempfile GC.start end end 10.times do Thread.new { tempfile_making_thread } end sleep 0.5 ``` I git bisect'd the fix to this commit. I guess the `eTerminateSignal` that kills threads on process exit has the same problem as an explicit `Thread#raise`. ---------------------------------------- Bug #15507: Thread#raise is delivered to GC context https://bugs.ruby-lang.org/issues/15507#change-101338 * Author: larskanis (Lars Kanis) * Status: Closed * Priority: Normal * ruby -v: ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux] * Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- Since I started the development of [Eventbox](https://github.com/larskanis/eventbox) I noticed sporadic failures of the Eventbox test suite on MRI. [One issue](https://github.com/larskanis/eventbox/issues/2) has been fixed in the meantime, but I was now able to minimize 3 others to a reproducible script. This is the first one: This script combines a GC finalizer with Thread#raise: ```ruby Thread.abort_on_exception = true class Finalizer def self.define ObjectSpace.define_finalizer(String.new, self.method(:gc)) end def self.gc(object_id) puts "gc #{object_id} #{caller[0]}" end end class Deadlocker class Stop < RuntimeError end def self.run th = Thread.handle_interrupt(Exception => :never) do Thread.new do begin Thread.handle_interrupt(Stop => :on_blocking) do 100.times { String.new } # Trigger GC sleep 5 # Wait for Stop exception raise "not interrupted" end rescue Stop end end end th.raise Stop th.join end end 100.times do Finalizer.define # This alone works well Deadlocker.run # This alone works equally, but both interfere badly end ``` The program output looks similar to: ```ruby $ ruby -d --disable-gems interrupt-in-gc-error.rb Exception `Deadlocker::Stop' at interrupt-in-gc-error.rb:23 - Deadlocker::Stop Exception `Deadlocker::Stop' at interrupt-in-gc-error.rb:23 - Deadlocker::Stop [...] Exception `Deadlocker::Stop' at interrupt-in-gc-error.rb:9 - Deadlocker::Stop gc 47133824012760 interrupt-in-gc-error.rb:22:in `call' gc 47133824022800 interrupt-in-gc-error.rb:22:in `call' [...] Exception `RuntimeError' at interrupt-in-gc-error.rb:24 - not interrupted #<Thread:0x000055bc65ac8f38@interrupt-in-gc-error.rb:19 run> terminated with exception (report_on_exception is true): Traceback (most recent call last): 2: from interrupt-in-gc-error.rb:21:in `block (2 levels) in run' 1: from interrupt-in-gc-error.rb:21:in `handle_interrupt' interrupt-in-gc-error.rb:24:in `block (3 levels) in run': not interrupted (RuntimeError) ``` The debug output shows, that the `Stop` exception is delivered several times, so that the sleep call is properly interrupted. But if the interrupt is sent just in the moment when the finalizer is active, it is discarded and doesn't abort the `sleep` call. IMHO interrupts shouldn't be delivered to any GC/trap context. At least they should be masked in this context. This issue is present on all older MRI versions. However it doesn't appear on JRuby-9.2.5.0. So they seem to have solved the GC/interrupt relationship somehow. -- https://bugs.ruby-lang.org/

1 0