January 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112113] [Ruby master Feature#18285] NoMethodError#message uses a lot of CPU/is really expensive to call
by mame (Yusuke Endoh) 30 Jan '23

30 Jan '23

Issue #18285 has been updated by mame (Yusuke Endoh). I have updated my PR: https://github.com/ruby/ruby/pull/6950 As I was working on updating the PR, the following message change caught my attention. ``` old: undefined method `foo' for main:Object new: undefined method `foo' for #<Object:0x00007f230602cd28> ``` Because the toplevel (`main`) is an extended object, `any_to_s` rule is applied. But I think the new message is too confusing. I talked with @matz, and we decided to handle the toplevel specially: ``` new: undefined method `foo' for main ``` I found no other confusing examples during updating my PR. ---------------------------------------- Feature #18285: NoMethodError#message uses a lot of CPU/is really expensive to call https://bugs.ruby-lang.org/issues/18285#change-101550 * Author: ivoanjo (Ivo Anjo) * Status: Open * Priority: Normal ---------------------------------------- Hello there! I'm working at Datadog on the ddtrace gem -- https://github.com/DataDog/dd-trace-rb and we ran into this issue on one of our internal testing applications. I also blogged about this issue in <https://ivoanjo.me/blog/2021/11/01/nomethoderror-ruby-cost/>. ### Background While testing an application that threw a lot of `NoMethodError`s in a Rails controller (this was used for validation), we discovered that service performance was very much impacted when we were logging these exceptions. While investigating with a profiler, the performance impact was caused by calls to `NoMethodError#message`, because this Rails controller had a quite complex `#inspect` method, that was getting called every time we tried to get the `#message` from the exception. ### How to reproduce ```ruby require 'bundler/inline' gemfile do source 'https://rubygems.org' gem 'benchmark-ips' end puts RUBY_DESCRIPTION class GemInformation # ... def get_no_method_error method_does_not_exist rescue => e e end def get_runtime_error raise 'Another Error' rescue => e e end def inspect # <-- expensive method gets called when calling NoMethodError#message Gem::Specification._all.inspect end end NO_METHOD_ERROR_INSTANCE = GemInformation.new.get_no_method_error RUNTIME_ERROR_INSTANCE = GemInformation.new.get_runtime_error Benchmark.ips do |x| x.config(:time => 5, :warmup => 2) x.report("no method error message cost") { NO_METHOD_ERROR_INSTANCE.message } x.report("runtime error message cost") { RUNTIME_ERROR_INSTANCE.message } x.compare! end ``` ### Expectation and result Getting the `#message` from a `NoMethodError` should be no costly than getting it from any other exception. In reality: ``` ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] no method error message cost 115.390 (± 1.7%) i/s - 580.000 in 5.027822s runtime error message cost 6.938M (± 0.5%) i/s - 35.334M in 5.092617s Comparison: runtime error message cost: 6938381.6 i/s no method error message cost: 115.4 i/s - 60130.02x (± 0.00) slower ``` ### Suggested solutions 1. Do not call `#inspect` on the object on which the method was not found (see <https://github.com/ruby/ruby/blob/e0915ba67964d843832148aeca29a1f8244ca7b1/…>) 2. Cache result of calling `#message` after the first call. Ideally this should be done together with suggestion 1. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112112] [Ruby master Misc#16671] BASERUBY version policy
by hsbt (Hiroshi SHIBATA) 30 Jan '23

30 Jan '23

Issue #16671 has been updated by hsbt (Hiroshi SHIBATA). CentOS 7 will be EOL at 2024/06 and CentOS 8+ will be not provided in the future. I know Red Hat switched them to CentOS Stream. Unfortunately, CentOS Stream (and RHEL) is not major distribution for Ruby developers. ---------------------------------------- Misc #16671: BASERUBY version policy https://bugs.ruby-lang.org/issues/16671#change-101549 * Author: ko1 (Koichi Sasada) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- Ruby 2.7 (MRI) requires Ruby 2.2 or later (== BASERUBY) to build from repository. Tarball does not need any installed Ruby. To build latest Ruby from repository, you need to build Ruby 2.2 and later from a tarball. Can we make the BASERUBY version update policy? # Background To use `ISeq#to_a` (specific format) Ruby 2.2 or later is needed from Ruby 2.7. The oldest version used by RubyCI machines was Ruby 2.2, so I determined update. I needed to file a ticket about this version bump, sorry. --- related: https://bugs.ruby-lang.org/issues/16668 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112111] [Ruby master Misc#16671] BASERUBY version policy
by vo.x (Vit Ondruch) 30 Jan '23

30 Jan '23

Issue #16671 has been updated by vo.x (Vit Ondruch). Why Ubuntu / Debian and not CentOS / RHEL 7? IOW I think it would be probably better to base the policy on Ruby releases then on external factor such as Linux distribution versions. ---------------------------------------- Misc #16671: BASERUBY version policy https://bugs.ruby-lang.org/issues/16671#change-101548 * Author: ko1 (Koichi Sasada) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- Ruby 2.7 (MRI) requires Ruby 2.2 or later (== BASERUBY) to build from repository. Tarball does not need any installed Ruby. To build latest Ruby from repository, you need to build Ruby 2.2 and later from a tarball. Can we make the BASERUBY version update policy? # Background To use `ISeq#to_a` (specific format) Ruby 2.2 or later is needed from Ruby 2.7. The oldest version used by RubyCI machines was Ruby 2.2, so I determined update. I needed to file a ticket about this version bump, sorry. --- related: https://bugs.ruby-lang.org/issues/16668 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112106] [Ruby master Misc#16671] BASERUBY version policy
by matz (Yukihiro Matsumoto) 29 Jan '23

29 Jan '23

Issue #16671 has been updated by matz (Yukihiro Matsumoto). I agree with upgrading BASERUBY. The version should be determined by the latest version of the distribution (Debian?). Matz. ---------------------------------------- Misc #16671: BASERUBY version policy https://bugs.ruby-lang.org/issues/16671#change-101543 * Author: ko1 (Koichi Sasada) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- Ruby 2.7 (MRI) requires Ruby 2.2 or later (== BASERUBY) to build from repository. Tarball does not need any installed Ruby. To build latest Ruby from repository, you need to build Ruby 2.2 and later from a tarball. Can we make the BASERUBY version update policy? # Background To use `ISeq#to_a` (specific format) Ruby 2.2 or later is needed from Ruby 2.7. The oldest version used by RubyCI machines was Ruby 2.2, so I determined update. I needed to file a ticket about this version bump, sorry. --- related: https://bugs.ruby-lang.org/issues/16668 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112102] [Ruby master Misc#19096] [Question] Time with `-00:00` offset is in UTC
by andrykonchin (Andrew Konchin) 28 Jan '23

28 Jan '23

Issue #19096 has been updated by andrykonchin (Andrew Konchin). @nobu Thank you! ---------------------------------------- Misc #19096: [Question] Time with `-00:00` offset is in UTC https://bugs.ruby-lang.org/issues/19096#change-101539 * Author: andrykonchin (Andrew Konchin) * Status: Closed * Priority: Normal ---------------------------------------- It's a bit unexpected but ```ruby Time.new(2022, 1, 1, 0, 0, 0, "-00:00").utc? # => true ``` But time with `+00:00` or `0` offset is treated as not UTC time: ```ruby Time.new(2022, 1, 1, 0, 0, 0, "+00:00").utc? # => false Time.new(2022, 1, 1, 0, 0, 0, 0).utc? # => false ``` Is it an intentional behaviour? In this case could you please clarify the reason why it works this way? --- ``` ruby -v ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112101] [Ruby master Misc#19096] [Question] Time with `-00:00` offset is in UTC
by nobu (Nobuyoshi Nakada) 28 Jan '23

28 Jan '23

Issue #19096 has been updated by nobu (Nobuyoshi Nakada). Status changed from Open to Closed It is commit:git|296a2cab. See also the related [changesets](https://bugs.ruby-lang.org/issues/17544?tab=changesets). ---------------------------------------- Misc #19096: [Question] Time with `-00:00` offset is in UTC https://bugs.ruby-lang.org/issues/19096#change-101538 * Author: andrykonchin (Andrew Konchin) * Status: Closed * Priority: Normal ---------------------------------------- It's a bit unexpected but ```ruby Time.new(2022, 1, 1, 0, 0, 0, "-00:00").utc? # => true ``` But time with `+00:00` or `0` offset is treated as not UTC time: ```ruby Time.new(2022, 1, 1, 0, 0, 0, "+00:00").utc? # => false Time.new(2022, 1, 1, 0, 0, 0, 0).utc? # => false ``` Is it an intentional behaviour? In this case could you please clarify the reason why it works this way? --- ``` ruby -v ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112100] [Ruby master Misc#19096] [Question] Time with `-00:00` offset is in UTC
by andrykonchin (Andrew Konchin) 28 Jan '23

28 Jan '23

Issue #19096 has been updated by andrykonchin (Andrew Konchin). @zverok Thank you for pointing at the original ticket. TBH I was waiting so long for some official confirmation because I haven't found any mentioning that https://bugs.ruby-lang.org/issues/17544 affects not only `Time#strftime` method but also a way how `utc offset` argument (in Time construction methods) is handled. Even [added tests](https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revis… don't check this behaviour. ---------------------------------------- Misc #19096: [Question] Time with `-00:00` offset is in UTC https://bugs.ruby-lang.org/issues/19096#change-101536 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal ---------------------------------------- It's a bit unexpected but ```ruby Time.new(2022, 1, 1, 0, 0, 0, "-00:00").utc? # => true ``` But time with `+00:00` or `0` offset is treated as not UTC time: ```ruby Time.new(2022, 1, 1, 0, 0, 0, "+00:00").utc? # => false Time.new(2022, 1, 1, 0, 0, 0, 0).utc? # => false ``` Is it an intentional behaviour? In this case could you please clarify the reason why it works this way? --- ``` ruby -v ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111287] [Ruby master Bug#19235] StringIO.new(encoding: "ext_enc:int_enc") doesn't set external encoding
by andrykonchin (Andrew Konchin) 28 Jan '23

28 Jan '23

Issue #19235 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19235: StringIO.new(encoding: "ext_enc:int_enc") doesn't set external encoding https://bugs.ruby-lang.org/issues/19235 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * ruby -v: 3.1.2 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- I noticed a divergence between IO `:encoding` option documentation and StringIO's behaviour: ```ruby io = StringIO.new('', 'w', encoding: 'utf-8:ISO-8859-1') io.external_encoding # => #<Encoding:ISO-8859-1> ``` The IO class behaves in expected way: ```ruby f = File.open("test.txt", 'w', encoding: 'UTF-8:ISO-8859-1') f.external_encoding # => #<Encoding:UTF-8> ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:112082] [Ruby master Bug#18567] Depending on default gems in stdlib gems when not needed considered harmful
by Eregon (Benoit Daloze) 27 Jan '23

27 Jan '23

Issue #18567 has been updated by Eregon (Benoit Daloze). hsbt (Hiroshi SHIBATA) wrote in #note-37: > I can work it with reasonable effort, not a requirement. I am fine with that, and would appreciate that very much. I guess there won't be many new default gems anyway, it seems almost every stdlib is a default gem nowadays. > Because I'm not truffleruby developer and it's not my primary work. I understand, but unfortunately the process of making new default gems actually breaks things and it seems best to avoid breaking as much as possible. The main issue so far is we learn too late about it, when it's already broken, vs spend a bit of time to fix it before people depend on the gem. > You don't know, I already worked it. In my experience, `jruby` and `truffleruby` jobs sometimes failed with CRuby changes. This fails are abandoned usually. I didn't know, thank you. In case it doesn't work, I would love if you can just mention it to me for TruffleRuby/`@headius` for JRuby. That's the ideal time to fix it, when creating the new default gem. > You should work truffleruby support for default gems repo before your proposal. I am working on that, see e.g.: * https://github.com/oracle/truffleruby/issues/2644 * https://github.com/ruby/bigdecimal/pull/247 * https://github.com/eregon/truffleruby-gem-tracker/blob/fcf6bc0057f72275fdad… ---------------------------------------- Bug #18567: Depending on default gems in stdlib gems when not needed considered harmful https://bugs.ruby-lang.org/issues/18567#change-101516 * Author: Eregon (Benoit Daloze) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- CRuby over time moves more and more code to default gems, to bundled gems and or even stops shipping some gems which used to be stdlibs (#5481). I believe the main motivation for that is being to fix security issues without needing to make a CRuby release, and that makes sense. There are however multiple unwanted side effects of this: 1. Removing gems from stdlib (e.g., #17873) is a breaking change, which makes upgrading to a new Ruby version more difficult. I think this should only be done if there is a clear gain. Being a default gem is already enough to fix a security issue without a CRuby release. 2. When any gem depends on a default gem, it tends to break on all Ruby implementations except CRuby, and for older Ruby versions. Let's focus on this second point. There can be good reasons to depend on a specific version (or ~>/>=) of a default gem, for instance to ensure a given security issue is addressed. In other cases, I think there is no value to depend explicitly on a default gem, it would work without an explicit dependency since it is still "in stdlib". And it is actually harmful to depend on default gems for JRuby, TruffleRuby and older Ruby versions, because the default gem does typically not work there yet, but the stdlib works just fine! The reason for that is simple, those gems used to be stdlib and so were implemented directly in the Ruby implementation. Also depending on default gem will typically be redundant with what's in stdlib, which is then a waste of network, time and disk. For larger default gems (e.g., openssl), I believe the solution is those gems to support JRuby, TruffleRuby, etc. This is useful so the behavior for a given version of the gem is compatible between Ruby implementations, has the same security fixes, etc. It is however a large effort and overhead to do this, and it only makes sense if people are going to need to depend on such a gem explicitly (either for security or new features in a given version), otherwise the version in stdlib is good enough and much simpler. Here are I think some clear cases of default gems which are clearly more overhead than what they gain: * io-wait: just a few methods very tight to IO internals, should really be core * io-nonblock just a few methods very tight to IO internals, should really be core * digest: has a public header and so versioning it doesn't work. Also it makes sense to reuse e.g. MessageDigest on JVM for better performance. * strscan: this accesses a lot of Regexp internal, it would fit better in each implementation repo as a non-gem stdlib. These are all small, they are all fairly stable, and it's unclear why they are even default gems in the first place. They also seem fairly unlikely to have security issues. So this is what I propose: * Do not depend on default gems in stdlib gems unless necessary (for security or feature), or unless we know the next version of Ruby will no longer ship that gem. An example is `net-protocol` depending needlessly on `io-wait`, I'll make a PR for that. * I think those gems listed just above should no longer be default gems in the future to clarify the situation. They should either be core or regular non-gem stdlib. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112080] [Ruby master Bug#18567] Depending on default gems in stdlib gems when not needed considered harmful
by hsbt (Hiroshi SHIBATA) 27 Jan '23

27 Jan '23

Issue #18567 has been updated by hsbt (Hiroshi SHIBATA). I don't agree it. I can work it with reasonable effort, not a requirement. Because I'm not truffleruby developer and it's not my primary work. You don't know, I already worked it. In my experience, `jruby` and `truffleruby` jobs sometimes failed with CRuby changes. This fails are abandoned usually. see. * https://github.com/ruby/cgi/commit/7ac6f1c9098992ab145980edfb4129f692586cfc * https://github.com/ruby/delegate/commit/d0f0be1b630bd2c3376f40b0d47dafcbede… * etc. You should work truffleruby support for default gems repo before your proposal. ---------------------------------------- Bug #18567: Depending on default gems in stdlib gems when not needed considered harmful https://bugs.ruby-lang.org/issues/18567#change-101514 * Author: Eregon (Benoit Daloze) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- CRuby over time moves more and more code to default gems, to bundled gems and or even stops shipping some gems which used to be stdlibs (#5481). I believe the main motivation for that is being to fix security issues without needing to make a CRuby release, and that makes sense. There are however multiple unwanted side effects of this: 1. Removing gems from stdlib (e.g., #17873) is a breaking change, which makes upgrading to a new Ruby version more difficult. I think this should only be done if there is a clear gain. Being a default gem is already enough to fix a security issue without a CRuby release. 2. When any gem depends on a default gem, it tends to break on all Ruby implementations except CRuby, and for older Ruby versions. Let's focus on this second point. There can be good reasons to depend on a specific version (or ~>/>=) of a default gem, for instance to ensure a given security issue is addressed. In other cases, I think there is no value to depend explicitly on a default gem, it would work without an explicit dependency since it is still "in stdlib". And it is actually harmful to depend on default gems for JRuby, TruffleRuby and older Ruby versions, because the default gem does typically not work there yet, but the stdlib works just fine! The reason for that is simple, those gems used to be stdlib and so were implemented directly in the Ruby implementation. Also depending on default gem will typically be redundant with what's in stdlib, which is then a waste of network, time and disk. For larger default gems (e.g., openssl), I believe the solution is those gems to support JRuby, TruffleRuby, etc. This is useful so the behavior for a given version of the gem is compatible between Ruby implementations, has the same security fixes, etc. It is however a large effort and overhead to do this, and it only makes sense if people are going to need to depend on such a gem explicitly (either for security or new features in a given version), otherwise the version in stdlib is good enough and much simpler. Here are I think some clear cases of default gems which are clearly more overhead than what they gain: * io-wait: just a few methods very tight to IO internals, should really be core * io-nonblock just a few methods very tight to IO internals, should really be core * digest: has a public header and so versioning it doesn't work. Also it makes sense to reuse e.g. MessageDigest on JVM for better performance. * strscan: this accesses a lot of Regexp internal, it would fit better in each implementation repo as a non-gem stdlib. These are all small, they are all fairly stable, and it's unclear why they are even default gems in the first place. They also seem fairly unlikely to have security issues. So this is what I propose: * Do not depend on default gems in stdlib gems unless necessary (for security or feature), or unless we know the next version of Ruby will no longer ship that gem. An example is `net-protocol` depending needlessly on `io-wait`, I'll make a PR for that. * I think those gems listed just above should no longer be default gems in the future to clarify the situation. They should either be core or regular non-gem stdlib. -- https://bugs.ruby-lang.org/

1 0