January 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111400] [Ruby master Bug#19254] Enabling YJIT configuration option breaks rspec-core test suite
by vo.x (Vit Ondruch) 11 Jan '23

11 Jan '23

Issue #19254 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19254: Enabling YJIT configuration option breaks rspec-core test suite https://bugs.ruby-lang.org/issues/19254 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0dev (2022-12-23 master c5eefb7f37) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- In preparation for Ruby 3.2, we have enabled YJIT in Fedora: https://src.fedoraproject.org/rpms/ruby/c/3c1be9f9c2c1d8679eebb9a185fefa15b… Since that moment, rspec-core test suite started to fail (see the attached log for all details): ~~~ ... snip ... 1) RSpec::Core::Example#run memory leaks, see GH-321, GH-1921 releases references to the examples / their ivars Failure/Error: expect(get_all.call).to eq opts.fetch(:post_gc) expected: [] got: ["after_all", "before_all"] (compared using ==) # ./spec/rspec/core/example_spec.rb:469:in `expect_gc' # ./spec/rspec/core/example_spec.rb:492:in `block (4 levels) in <top (required)>' # ./spec/support/sandboxing.rb:16:in `block (3 levels) in <top (required)>' # ./spec/support/sandboxing.rb:7:in `block (2 levels) in <top (required)>' Finished in 8.98 seconds (files took 0.47612 seconds to load) 2209 examples, 1 failure, 4 pending ~~~ Please note that the YJIT was not enabled during runtime, just the support was enabled. Disabling the YJIT supports makes the test case pass. [1]: https://download.copr.fedorainfracloud.org/results/vondruch/ruby-3.2/fedora… [2]: https://copr.fedorainfracloud.org/coprs/vondruch/ruby-3.2/package/rubygem-r… ---Files-------------------------------- builder-live.log.gz (28.7 KB) -- https://bugs.ruby-lang.org/

5 6

[ruby-core:111783] [Ruby master Bug#19332] Thread::Queue#pop raises ArgumentError with
by schnittchen (Thomas Stratmann) 11 Jan '23

11 Jan '23

Issue #19332 has been reported by schnittchen (Thomas Stratmann). ---------------------------------------- Bug #19332: Thread::Queue#pop raises ArgumentError with https://bugs.ruby-lang.org/issues/19332 * Author: schnittchen (Thomas Stratmann) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ✓ ruby --version ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] ✓ irb Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7.1 Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2 irb(main):001:0> require "thread" => false irb(main):002:0> q = Thread::Queue.new => #<Thread::Queue:0x00007f5ec57c2da8> irb(main):003:0> q.pop(non_block: true) <internal:thread_sync>:14:in `pop': unknown keyword: :non_block (ArgumentError) from (irb):3:in `<main>' from /home/thomas/.rubies/ruby-3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>' from /home/thomas/.rubies/ruby-3.2.0/bin/irb:25:in `load'·· from /home/thomas/.rubies/ruby-3.2.0/bin/irb:25:in `<main>' irb(main):004:0> According to the documentation, I expected a ThreadError to be raised. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:111775] [Ruby master Bug#19330] ruby 3.2.0 parameter autosplat breaks call using (*args, &block)
by rockorequin (rocko requin) 11 Jan '23

11 Jan '23

Issue #19330 has been reported by rockorequin (rocko requin). ---------------------------------------- Bug #19330: ruby 3.2.0 parameter autosplat breaks call using (*args, &block) https://bugs.ruby-lang.org/issues/19330 * Author: rockorequin (rocko requin) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following code in Rails 6.1.7 activerecord relation.rb no longer works with ruby 3.2.0, because the call "instance_exec(*args, &block)" raises an ArgumentException: ``` def _exec_scope(*args, &block) # :nodoc: @delegate_to_klass = true _scoping(nil) { instance_exec(*args, &block) || self } ensure @delegate_to_klass = false end ``` I think it may be due to the bugfix https://bugs.ruby-lang.org/issues/18633 ("proc { |a, **kw| a } autosplats and treats empty kwargs specially"). There is more info at https://github.com/rails/rails/issues/46934, including this code to reproduce the issue: ``` gem 'rails', '=6.1.7' require 'active_record' class Test < ActiveRecord::Base scope :test, ->(arg: nil) {} end Test.test(arg: 1) ``` Another user in that issue report has indicated that the syntax "save(**)" in suppressor.rb raises the same exception in ruby 3.2.0: ``` def save(**) # :nodoc: SuppressorRegistry.suppressed[self.class.name] ? true : super end ``` Is this intentional, ie is the syntax "instance_exec(*args, &block)" etc no longer valid in ruby 3.2.0? If so, could a note indicating this incompatibility perhaps be added to the release notes for 3.2.0? -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111772] [Ruby master Feature#18190] Split `Random::Formatter` from securerandom
by Eregon (Benoit Daloze) 10 Jan '23

10 Jan '23

Issue #18190 has been updated by Eregon (Benoit Daloze). To clarify, random/formatter is now a stdlib (not loaded by default). `Random::Formatter` is defined in core but has few methods before `require 'random/formatter'`: ``` $ ruby -e 'puts Random::Formatter.instance_methods(false)' rand random_number $ ruby -rrandom/formatter -e 'puts Random::Formatter.instance_methods(false)' random_number uuid rand alphanumeric hex random_bytes base64 urlsafe_base64 ``` ---------------------------------------- Feature #18190: Split `Random::Formatter` from securerandom https://bugs.ruby-lang.org/issues/18190#change-101184 * Author: nobu (Nobuyoshi Nakada) * Status: Closed * Priority: Normal ---------------------------------------- Now `Random::Formatter` methods are defined in `securerandom.rb`, since it was split from `SecureRandom` module historically. However this module does not need to be `SecureRandom` but just to respond to `bytes` method. I propose to move `Random::Formatter` module to another file, `random_formatter.rb` or `random/formatter.rb`. And keep only that file in ruby core and remove from securerandom library in future. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111770] [Ruby master Feature#18285] NoMethodError#message uses a lot of CPU/is really expensive to call
by Hanmac (Hans Mackowiak) 10 Jan '23

10 Jan '23

Issue #18285 has been updated by Hanmac (Hans Mackowiak). maybe we could add some conditions when the object should be printed instead of the class for NoMethodError? * if object overwrites `method_missing` and maybe `respond_to_missing` then print the object instead of the class * if the object has included or prepend modules into its singleton class, then print the object instead of real class these are the cases i can think of that could affect the methods an object can respond to that aren't related to its class ---------------------------------------- Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call https://bugs.ruby-lang.org/issues/18285#change-101178 * Author: ivoanjo (Ivo Anjo) * Status: Open * Priority: Normal ---------------------------------------- Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>. ### Background While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception. ### How to reproduce ```ruby require 'bundler/inline' gemfile do source 'https://rubygems.org' gem 'benchmark-ips' end puts RUBY_DESCRIPTION class GemInformation # ... def get_no_method_error method_does_not_exist rescue => e e end def get_runtime_error raise 'Another Error' rescue => e e end def inspect # <-- expensive method gets called when calling NoMethodError#message Gem::Specification._all.inspect end end NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error Benchmark.ips do |x| x.config(:time => 5, :warmup => 2) x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message } x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message } x.compare! end ``` ### Expectation and result Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception. In reality: ``` ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] no method error message cost 115.390 (± 1.7%) i/s - 580.000 in 5.027822s runtime error message cost 6.938M (± 0.5%) i/s - 35.334M in 5.092617s Comparison: runtime error message cost: 6938381.6 i/s no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower ``` ### Suggested solutions 1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>) 2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111769] [Ruby master Bug#19005] Ruby interpreter compiled XCode 14 cannot build some native gems on macOS
by stanhu (Stan Hu) 10 Jan '23

10 Jan '23

Issue #19005 has been updated by stanhu (Stan Hu). nobu (Nobuyoshi Nakada) wrote in #note-33: > Adding `-no_fixup_chains -undefined dynamic_lookup` together will break old compilers. > `-no_fixup_chains` should be checked solely **before** `-undefined dynamic_lookup`, at least. Good point. I've updated https://github.com/ruby/ruby/pull/7086. ---------------------------------------- Bug #19005: Ruby interpreter compiled XCode 14 cannot build some native gems on macOS https://bugs.ruby-lang.org/issues/19005#change-101177 * Author: stanhu (Stan Hu) * Status: Open * Priority: Normal * ruby -v: ruby 2.7.6p219 (2022-04-12 revision 44c8bfa984) [arm64-darwin21] * Backport: 2.7: DONE, 3.0: DONE, 3.1: DONE ---------------------------------------- This seems related to https://bugs.ruby-lang.org/issues/18912 and https://bugs.ruby-lang.org/issues/18981 . Steps to reproduce: 1. Upgrade to XCode 14. 2. Compile a new Ruby interpreter. I used the version provided in https://github.com/ruby/ruby/pull/6297 with `./configure --prefix=/tmp/ruby --with-openssl-dir=$(brew --prefix openssl(a)1.1) --with-readline-dir=$(brew --prefix readline) --enable-shared`. 3. Confirm that `-Wl,-undefined,dynamic_lookup` is no longer available: ``` irb(main):001:0> RbConfig::CONFIG['DLDFLAGS'] => "-Wl,-multiply_defined,suppress" ``` 4. Ran `gem install pg_query` (`gem install ffi-yajl` will also fail). Error: ``` linking shared-object pg_query/pg_query.bundle Undefined symbols for architecture arm64: "Init_pg_query", referenced from: -exported_symbol[s_list] command line option (maybe you meant: _Init_pg_query) ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) ``` I can workaround the problem by doing: ``` gem install pg_query -- --with-ldflags="-Wl,-undefined,dynamic_lookup" ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111768] [Ruby master Feature#18285] NoMethodError#message uses a lot of CPU/is really expensive to call
by mame (Yusuke Endoh) 10 Jan '23

10 Jan '23

Issue #18285 has been updated by mame (Yusuke Endoh). zverok (Victor Shepelev) wrote in #note-26: > Say, in a complicated algorithm working with nested data structures, `undefined method transform_keys for object Array` is significantly worse than `undefined method transform_keys for [{id: 1, name: 'John'}]`. I think this is arguable. Is the receiver often as simple as `[{id: 1, name: 'John'}]`? And how are the contents of the receiver helpful to debug the bug? This topic is not general, but specific to NoMethodError message. I wonder if the contents of the receiver are really so critical to debug a bug of NoMethodError. On the other hand, I have experienced multiple times that a length #inspect swept away the terminal logs and obscured the important NoMethodError message. As far as I remember, the only information I needed for debug was the receiver class, not its contents. Rather than talking about it with an imagination, we should actually live with the new message format and experience how (in)convenient it is, I think. ---------------------------------------- Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call https://bugs.ruby-lang.org/issues/18285#change-101176 * Author: ivoanjo (Ivo Anjo) * Status: Open * Priority: Normal ---------------------------------------- Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>. ### Background While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception. ### How to reproduce ```ruby require 'bundler/inline' gemfile do source 'https://rubygems.org' gem 'benchmark-ips' end puts RUBY_DESCRIPTION class GemInformation # ... def get_no_method_error method_does_not_exist rescue => e e end def get_runtime_error raise 'Another Error' rescue => e e end def inspect # <-- expensive method gets called when calling NoMethodError#message Gem::Specification._all.inspect end end NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error Benchmark.ips do |x| x.config(:time => 5, :warmup => 2) x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message } x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message } x.compare! end ``` ### Expectation and result Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception. In reality: ``` ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] no method error message cost 115.390 (± 1.7%) i/s - 580.000 in 5.027822s runtime error message cost 6.938M (± 0.5%) i/s - 35.334M in 5.092617s Comparison: runtime error message cost: 6938381.6 i/s no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower ``` ### Suggested solutions 1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>) 2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111765] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention
by k0kubun (Takashi Kokubun) 10 Jan '23

10 Jan '23

Issue #18439 has been updated by k0kubun (Takashi Kokubun). Subject changed from Support YJIT for VC++ to YJIT: Support Microsoft x86 calling convention What's used for compiling YJIT itself, which was changed from a C compiler to `rustc`, wasn't a huge concern for supporting Windows. Given that @usa has an idea for the `mmap` part, the main challenge for supporting Windows would be ABI support in JITed code. I changed the ticket description to clarify what's to be done. ---------------------------------------- Feature #18439: YJIT: Support Microsoft x86 calling convention https://bugs.ruby-lang.org/issues/18439#change-101170 * Author: usa (Usaku NAKAMURA) * Status: Assigned * Priority: Normal * Assignee: yjit ---------------------------------------- I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master. And, I found we need more changes to actually enabled YJIT for VC++, at least: - YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`. Really? - Maybe ABI deffers between VC++ and YJIT's expectation. Can I get support to fix above? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111763] [Ruby master Feature#18439] Support YJIT for VC++
by hsbt (Hiroshi SHIBATA) 10 Jan '23

10 Jan '23

Issue #18439 has been updated by hsbt (Hiroshi SHIBATA). Status changed from Open to Assigned Assignee changed from maximecb (Maxime Chevalier-Boisvert) to yjit YJIT has been rewrote with rust in Ruby 3.2. Should we close this? ---------------------------------------- Feature #18439: Support YJIT for VC++ https://bugs.ruby-lang.org/issues/18439#change-101167 * Author: usa (Usaku NAKAMURA) * Status: Assigned * Priority: Normal * Assignee: yjit ---------------------------------------- I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master. And, I found we need more changes to actually enabled YJIT for VC++, at least: - YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`. Really? - Maybe ABI deffers between VC++ and YJIT's expectation. Can I get support to fix above? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111402] [Ruby master Bug#19255] YJIT: global symbol leakage
by nobu (Nobuyoshi Nakada) 10 Jan '23

10 Jan '23

Issue #19255 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19255: YJIT: global symbol leakage https://bugs.ruby-lang.org/issues/19255 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED ---------------------------------------- When YJIT support is enabled, libyjit.a adds thousands leaking global symbols. With this patch: ```diff diff --git a/template/Makefile.in b/template/Makefile.in index 145631386d8..6191bcbeb88 100644 --- a/template/Makefile.in +++ b/template/Makefile.in @@ -720,5 +720,5 @@ mjit_build_dir.$(SOEXT): $(MJIT_MIN_HEADER) $(srcdir)/ruby-runner.c ruby-runner. $(OUTFLAG)$@ $(srcdir)/ruby-runner.c # yes-test-basic: leaked-globals -leaked-globals: $(COMMONOBJS) prog $(tooldir)/leaked-globals PHONY - $(Q) $(XRUBY) $(tooldir)/leaked-globals NM=$(NM) SYMBOL_PREFIX=$(SYMBOL_PREFIX) PLATFORM=$(hdrdir)/ruby/$(PLATFORM_DIR).h $(srcdir)/configure.ac $(COMMONOBJS) +leaked-globals: $(COMMONOBJS) $(YJIT_LIBS) prog $(tooldir)/leaked-globals PHONY + $(Q) $(XRUBY) $(tooldir)/leaked-globals NM=$(NM) SYMBOL_PREFIX=$(SYMBOL_PREFIX) PLATFORM=$(hdrdir)/ruby/$(PLATFORM_DIR).h $(srcdir)/configure.ac $(COMMONOBJS) $(YJIT_LIBS) ``` `make leaked-globals` reports **2915 un-prefixed symbols leaked**. -- https://bugs.ruby-lang.org/

2 2