January 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112155] [Ruby master Bug#19395] Process forking within non-main Ractor creates child stuck in busy loop
by luke-gru (Luke Gruber) 16 Feb '24

16 Feb '24

Issue #19395 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19395: Process forking within non-main Ractor creates child stuck in busy loop https://bugs.ruby-lang.org/issues/19395 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby def test_fork_in_ractor r2 = Ractor.new do pid = fork do exit Ractor.count end pid end pid = r2.take puts "Process #{Process.pid} waiting for #{pid}" _pid, status = Process.waitpid2(pid) # stuck forever if status.exitstatus != 1 raise "status is #{status.exitstatus}" end end test_fork_in_ractor() ``` $ top # shows CPU usage is high for child process -- https://bugs.ruby-lang.org/

2 5

[ruby-core:111342] [Ruby master Bug#19246] Rebuilding the loaded feature index much slower in Ruby 3.1
by thomthom (Thomas Thomassen) 17 Jan '24

17 Jan '24

Issue #19246 has been reported by thomthom (Thomas Thomassen). ---------------------------------------- Bug #19246: Rebuilding the loaded feature index much slower in Ruby 3.1 https://bugs.ruby-lang.org/issues/19246 * Author: thomthom (Thomas Thomassen) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Some background to this issue: (This is a case that is unconventional usage of Ruby, but I hope you bear with me.) We ship the Ruby interpreter with our desktop applications for plugin support in our application (SketchUp). One feature we have had since, at least 2006 (maybe earlier-hard to track history beyond that) is that we had a custom alternate `require` method: `Sketchup.require`. This allows the users of our API to load encrypted Ruby files. This originally used `rb_provide` to add the path to the encrypted file into the list of loaded feature. However, somewhere between Ruby 2.2 and 2.5 there was some string optimisations made and the function `rb_provide` would not use a copy of the string passed to it. Instead it just held on to a pointer reference. In our case that string came from user-land, being passed in from `Sketchup.require` and would eventually be garbage collected and cause access violation crashes. To work around that we changed our custom `Sketchup.require` to push to `$LOADED_FEATURES` directly. There was a small penalty to the index being rebuilt after that, but it was negligible. Recently we tried to upgrade the Ruby interpreter in our application from 2.7 to 3.1 and found a major performance reduction when using our `Sketchup.require. As in, a plugin that would load in half a second would now spend 30 seconds. From https://bugs.ruby-lang.org/issues/18452 it sounds like there is _some_ expected extra penalty due to changes in how the index is built. But should it really be this much? Example minimal repro to simulate the issue: ``` # frozen_string_literal: true require 'benchmark' iterations = 200 foo_files = iterations.times.map { |i| "#{__dir__}/tmp/foo-#{i}.rb" } foo_files.each { |f| File.write(f, "") } bar_files = iterations.times.map { |i| "#{__dir__}/tmp/bar-#{i}.rb" } bar_files.each { |f| File.write(f, "") } biz_files = iterations.times.map { |i| "#{__dir__}/tmp/biz-#{i}.rb" } biz_files.each { |f| File.write(f, "") } Benchmark.bm do |x| x.report('normal') { foo_files.each { |file| require file } } x.report('loaded_features') { foo_files.each { |file| require file $LOADED_FEATURES << "#{file}-fake.rb" } } x.report('normal again') { biz_files.each { |file| require file } } end ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby27.bat ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.000000 0.031000 0.031000 ( 0.078483) loaded_features 0.015000 0.000000 0.015000 ( 0.038759) normal again 0.016000 0.032000 0.048000 ( 0.076940) ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby30.bat ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.000000 0.031000 0.031000 ( 0.074733) loaded_features 0.032000 0.000000 0.032000 ( 0.038898) normal again 0.000000 0.047000 0.047000 ( 0.076343) ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby31.bat ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x64-mingw-ucrt] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.016000 0.031000 0.047000 ( 0.132633) loaded_features 1.969000 11.500000 13.469000 ( 18.395761) normal again 0.031000 0.125000 0.156000 ( 0.249130) ``` Right now we're exploring options to deal with this. Because the performance degradation is a blocker for us upgrading. We also have 16 years of plugins created by third party developer that makes it impossible for us to drop this feature. Some options as-is, none of which are ideal: 1. We revert to using `rb_provide` but ensure the string passed in is not owned by Ruby, instead building a list of strings that we keep around for the duration of the application process. The problem is that some of our plugin developers have on occasion released plugins that will touch `$LOADED_FEATURES`, and if such a plugin is installed on a user machine it might cause the application to become unresponsive for minutes. The other non-ideal issue with using `rb_provide` is that we're also using that in ways it wasn't really intended (from that I understand). And it's not an official API? 2. We create a separate way for our `Sketchup.require` to keep track of it's loaded features, but then that would diverge even more from the behaviour of `require`. Replicating `require` functionality is not trivial and would be prone to subtle errors and possible diverge. It also doesn't address our issue that there is code out there in existing plugins that touches `$LOADED_FEATURES`. (And it's not something we can just ask people to clean up. From previous experience old versions stick around for a long time and is very hard to purge from circulation.) I have two questions for the Ruby mantainers: 1. Would it be reasonable to see an API for adding/removing/checking `$LOADED_FEATURE` that would allow for a more ideal implementation of a custom `require` functionality? 2. Is the performance difference in rebuilding the loaded feature index really expected to be as high as what we're seeing? An increase of nearly 100 times? Is there something there that might be addressed to make the rebuild to be less expensive against? (This would really help to address our challenges with third party plugins occasionally touching the global.) -- https://bugs.ruby-lang.org/

9 21

[ruby-core:112010] [Ruby master Bug#19375] File objects are currently shareable
by luke-gru (Luke Gruber) 29 Dec '23

29 Dec '23

Issue #19375 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19375: File objects are currently shareable https://bugs.ruby-lang.org/issues/19375 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I don't know the internals of file.c but I don't think files are thread-safe. -- https://bugs.ruby-lang.org/

1 2

[ruby-core:111739] [Ruby master Feature#19324] Enumerator.product => Enumerable#product
by zverok (Victor Shepelev) 23 Dec '23

23 Dec '23

Issue #19324 has been reported by zverok (Victor Shepelev). ---------------------------------------- Feature #19324: Enumerator.product => Enumerable#product https://bugs.ruby-lang.org/issues/19324 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- I know it might be too late after introducing a feature and releasing a version, but I find `Enumerator.product` quite confusing, and can't find any justification in #18685. **Problem 1: It is `Array#product` but `Enumerator.product`** ```ruby [1, 2].product([4, 5]) # => [[1, 4], [1, 5], [2, 4], [2, 5]] # Usually, when we add methods to Enumerable/Enumerator which # already array had before, it is symmetric, say... [1, nil, 2, 3].compact #=> [1, 2, 3] [1, nil, 2, 3].lazy.compact.first(2) #=> [1, 2] # But not in this case: [1, 2].lazy.product([4, 5]).first(2) # undefined method `product' for #<Enumerator::Lazy: [1, 2]> (NoMethodError) # Because you "just" need to change it to: Enumerator.product([1, 2].lazy, [4, 5]).first(2) # => [[1, 4], [1, 5]] ``` No other method was "promoted" from Array this way And in general, I believe core methods tend to belong to the first object in the expression and not be free module methods, Elixir style. **Problem 2: It is one letter different from `Enumerator.produce`** I understand I might be biased here (as a person who proposed `produce`), and that method is not as popular (yet?) as I hoped, but still, two methods that do completely different things and differ by one letter, both being somewhat vague verbs (so it is easy to confuse them unless you did a lot of math and "product" is firmly set for set product in your head). I believe that EITHER of two problems would be concerning enough, but the combination of them seems to be a strong enough argument to make the change?.. (Maybe with graceful deprecation of module method in one next version, but, considering the Ruby 3.2 is just released, maybe vice versa, fix the problem in the next minor release?..) -- https://bugs.ruby-lang.org/

6 17

[ruby-core:111988] [Ruby master Feature#19370] Anonymous parameters for blocks?
by zverok (Victor Shepelev) 23 Dec '23

23 Dec '23

Issue #19370 has been reported by zverok (Victor Shepelev). ---------------------------------------- Feature #19370: Anonymous parameters for blocks? https://bugs.ruby-lang.org/issues/19370 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- Just to clarify: are anonymous parameters delegation is planned to support in blocks? It would be a nice addition, if it is possible to implement: ```ruby # data in form [request method, URL, params]: [ [:get, 'https://google.com', {q: 'Ruby'}, {'User-Argent': 'Google-Chrome'}], [:post, 'https://gist.github.com', 'body'], # ... ].each { |method, *| request(method.to_s.upcase, *) } ``` ...and at the very least, consistent with what the method definition can have. If they are NOT planned to be implemented, I believe that at least error messages should be made much clearer, because currently, this would happen while running the code above: > no anonymous rest parameter (SyntaxError) I understand the reason (the `request` clause doesn't "see" anonymous parameter of the **block**, and claims that current **method** doesn't have them), but it looks honestly confusing and inconsistent. -- https://bugs.ruby-lang.org/

5 9

[ruby-core:112119] [Ruby master Bug#19392] Endless method vs and/or
by zverok (Victor Shepelev) 09 Dec '23

09 Dec '23

Issue #19392 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #19392: Endless method vs and/or https://bugs.ruby-lang.org/issues/19392 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- [Discovered](https://twitter.com/lucianghinda/status/1617783952353406977) by Lucian Ghinda: ```ruby def test = puts("foo") and puts("bar") # prints "bar" immediately test # prints "foo" ``` It seems that it is a parser error, right?.. ```ruby RubyVM::AbstractSyntaxTree.parse('def test = puts("foo") and puts("bar")') # => # (SCOPE@1:0-1:38 # tbl: [] # args: nil # body: # (AND@1:0-1:38 # (DEFN@1:0-1:22 # mid: :test # body: # (SCOPE@1:0-1:22 # tbl: [] # args: # (ARGS@1:0-1:8 pre_num: 0 pre_init: nil opt: nil first_post: nil post_num: 0 post_init: nil rest: nil kw: nil kwrest: nil block: nil) # body: (FCALL@1:11-1:22 :puts (LIST@1:16-1:21 (STR@1:16-1:21 "foo") nil)))) # (FCALL@1:27-1:38 :puts (LIST@1:32-1:37 (STR@1:32-1:37 "bar") nil)))) ``` E.g. it is parsed as ```ruby (def test = puts("foo")) and (puts("bar")) ``` ...which is hardly intentional or have any practical use. The rightly parsed code in this case _can_ have practical use, like ```ruby def write(data) = File.write(@filename, data) == data.size or raise "Something went wrong" ``` -- https://bugs.ruby-lang.org/

12 19

[ruby-core:111074] [Ruby master Bug#19160] cmp_clamp arguments
by kaiquekandykoga 13 Nov '23

13 Nov '23

Issue #19160 has been reported by kaiquekandykoga (Kaíque Koga). ---------------------------------------- Bug #19160: cmp_clamp arguments https://bugs.ruby-lang.org/issues/19160 * Author: kaiquekandykoga (Kaíque Koga) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-freebsd13.1] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- If clamp receives min higher than max, it will raise an exception. The message says *min argument must be smaller than max argument* , but min can actually be equal to max too. Patch https://github.com/ruby/ruby/pull/6802. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111953] [Ruby master Bug#19362] #dup on Proc doesn't call initialize_dup
by zverok (Victor Shepelev) 26 Oct '23

26 Oct '23

Issue #19362 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #19362: #dup on Proc doesn't call initialize_dup https://bugs.ruby-lang.org/issues/19362 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- In #17545, `#dup` had changed to create an instance of the subclass. It, though, doesn't invoke `initialize_dup` of the subclass, unlike other standard classes. ```ruby class MyAry < Array def initialize_dup(...) p(self.class, ...) super end end class MyString < String def initialize_dup(...) p(self.class, ...) super end end class MyProc < Proc def initialize_dup(...) p(self.class, ...) super end end MyString.new('test').dup # prints MyString, "test" MyAry.new(['test']).dup # prints MyAry, ["test"] MyProc.new { 'test' }.dup # doesn't print anything ``` This makes the change in #17545 useless: while inheriting from core classes is indeed marginal, one of author's intention might be carrying additional information with the Proc instance, and bypassing `#initialize_dup` makes it impossible to maintain this information. It seems that actually `#initialize_dup` is also invoked on the core classes themselves, but ignored on `Proc`. ```ruby class Array def initialize_dup(...) p(self.class, ...) super end end class String def initialize_dup(...) p(self.class, ...) super end end class Proc def initialize_dup(...) p(self.class, ...) super end end 'test'.dup # prints String, "test" ['test'].dup # prints Array, ["test"] Proc.new { 'test' }.dup # doesn't print anything ``` Which is an even more marginal problem but still an inconsistency. -- https://bugs.ruby-lang.org/

3 7

[ruby-core:111572] [Ruby master Bug#19297] Don't download content from internet to execute Ruby test suite
by vo.x (Vit Ondruch) 06 Sep '23

06 Sep '23

Issue #19297 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19297: Don't download content from internet to execute Ruby test suite https://bugs.ruby-lang.org/issues/19297 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Trying to build Ruby 3.2.0 for Fedora and execute its test suite via `make check` as we always did [1], the test suite suddenly fails (while it was working with commit:git|c5eefb7f37): ~~~ ... snip ... C-API Util function ruby_strtod - converts a string to a double and returns the remaining string - returns 0 and the full string if there's no numerical value Finished in 45.737677 seconds 3827 files, 31635 examples, 177877 expectations, 0 failures, 0 errors, 0 tagged ./miniruby -I/builddir/build/BUILD/ruby-3.2.0/lib -I. -I.ext/common /builddir/build/BUILD/ruby-3.2.0/tool/runruby.rb --extout=.ext -- --disable-gems -C "/builddir/build/BUILD/ruby-3.2.0" bin/gem install --no-document \ --install-dir .bundle --conservative "bundler" "rake" "rspec:~> 3" #"ruby-prof" ERROR: Could not find a valid gem 'bundler' (>= 0), here is why: Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz) ERROR: Could not find a valid gem 'rspec' (~> 3), here is why: Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz) make: Leaving directory '/builddir/build/BUILD/ruby-3.2.0/redhat-linux-build' make: *** [uncommon.mk:1464: yes-test-syntax-suggest-prepare] Error 2 ~~~ This is obviously due to the test suite trying to download `rspec` from the internet, while Fedora builders does not have internet access (and won't ever have for security reasons). If I am not mistaken, this is caused by commit:git|cae53842735237ccf71a13873fd0d1ae7f165582. Now 1) Can this be fixed? 2) Can the tarball be always self contained? [1]: https://src.fedoraproject.org/rpms/ruby/blob/631163e3b8a51ed610528181aabe0d… -- https://bugs.ruby-lang.org/

8 11

[ruby-core:111499] [Ruby master Bug#19281] SyntaxError if first argument of command call has semicolon inside parenthesis
by tompng (tomoya ishida) 04 Sep '23

04 Sep '23

Issue #19281 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19281: SyntaxError if first argument of command call has semicolon inside parenthesis https://bugs.ruby-lang.org/issues/19281 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- These are syntax error ~~~ruby p (1;2),(3),(4) p (;),(),() a.b (1;2),(3),(4) a.b (;),(),() ~~~ I expect it to be syntax ok because the code below is syntax ok. ~~~ruby p (1),(2;3),(4;5) p (),(;),(;) a.b (1),(2;3),(4;5) a.b (),(;),(;) ~~~ It will be easy to traverse sexp if the sexp of first argument is same as others ~~~ruby Ripper.sexp "p (),(),()" # => [:program, [[:command, [:@ident, "p", [1, 0]], [:args_add_block, [[:paren, false], # [:paren, [[:void_stmt]]] [:paren, [[:void_stmt]]], [:paren, [[:void_stmt]]]], false]]]] Ripper.sexp "p (1),(2),(3)" # => [:program, [[:command, [:@ident, "p", [1, 0]], [:args_add_block, [[:paren, [:@int, "1", [1, 3]]], # [:paren, [[:@int, "1", [1, 3]]]] [:paren, [[:@int, "2", [1, 7]]]], [:paren, [[:@int, "3", [1, 11]]]]], false]]]] ~~~ -- https://bugs.ruby-lang.org/

3 2