September 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119251] [Ruby master Bug#20754] [doc] `Hash::new` is missing from rdoc for 3.4
by Earlopain (A S) 18 Sep '24

18 Sep '24

Issue #20754 has been reported by Earlopain (A S). ---------------------------------------- Bug #20754: [doc] `Hash::new` is missing from rdoc for 3.4 https://bugs.ruby-lang.org/issues/20754 * Author: Earlopain (A S) * Status: Open * ruby -v: 3.4-dev * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I was interested in the `capacity` argument added in https://github.com/ruby/ruby/pull/10357 but found any reference to it missing from https://docs.ruby-lang.org/en/master/Hash.html `::new` just links to docs from `BasicObject`. Docs for 3.3 contain the proper `new` method: https://docs.ruby-lang.org/en/3.3/Hash.html#method-c-new Maybe it has something to do with it moving into ruby code? Not sure. I haven't checked other classes that moved into ruby. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:119242] [Ruby master Bug#20751] Regression in Prism related to use of super in default argument values
by jeremyevans0 (Jeremy Evans) 17 Sep '24

17 Sep '24

Issue #20751 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20751: Regression in Prism related to use of super in default argument values https://bugs.ruby-lang.org/issues/20751 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONTNEED ---------------------------------------- All versions of parse.y dating at least back to Ruby 1.8.7, and Prism in both Ruby 3.3.5 and 3.4.0preview1, correctly parses this code: ```ruby class A def foo(b = nil || (return)) end end ``` The master branch, unless `--parser=parse.y` is used, now considers this a SyntaxError: ``` -: -:2: syntax error found (SyntaxError) 1 | class A > 2 | def foo(b = nil || (return)) | ^~~~~~ Invalid return in class/module body 3 | end 4 | end ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119199] [Ruby master Bug#20742] Trying to assign to a variable in statement modifier should emit a warning
by esad (Esad Hajdarevic) 17 Sep '24

17 Sep '24

Issue #20742 has been reported by esad (Esad Hajdarevic). ---------------------------------------- Bug #20742: Trying to assign to a variable in statement modifier should emit a warning https://bugs.ruby-lang.org/issues/20742 * Author: esad (Esad Hajdarevic) * Status: Open * ruby -v: 3.3.4 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- There is an example in Control Expressions documentation: ``` p a if a = 0.zero? # raises NameError “undefined local variable or method ‘a’”. ``` However, if we had already defined `a` there would be no exception raised. If one uses something like `p` for scratch variable, due to Kernel#p, also no exception is raised. Statement modifier is generally somewhat inverting the code flow (the right part is evaluated first then the left part), so it is not really obvious why binding variables shouldn't follow the same flow. A warning would be very beneficial. -- https://bugs.ruby-lang.org/

4 6

[ruby-core:119220] [Ruby master Bug#20749] Error message not shown in output with prism
by luke-gru (Luke Gruber) 17 Sep '24

17 Sep '24

Issue #20749 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #20749: Error message not shown in output with prism https://bugs.ruby-lang.org/issues/20749 * Author: luke-gru (Luke Gruber) * Status: Open * ruby -v: 3.4.0dev * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- test.rb: ```ruby p = Proc.new ``` ``` ./ruby --parser=parse.y -I../ruby/lib -I. -I.ext/x86_64-linux -I.ext/common -r./x86_64-linux-fake ../ruby/test.rb ../ruby/test.rb:1:in 'Proc.new': tried to create Proc object without a block (ArgumentError) from ../ruby/test.rb:1:in '<main>' ``` ``` ./ruby --parser=prism -I../ruby/lib -I. -I.ext/x86_64-linux -I.ext/common -r./x86_64-linux-fake ../ruby/test.rb ../ruby/test.rb:1:in 'Proc.new': ArgumentError from ../ruby/test.rb:1:in '<main>' ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:119214] [Ruby master Bug#20747] [prism] regression in retry precedence
by byroot (Jean Boussier) 16 Sep '24

16 Sep '24

Issue #20747 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20747: [prism] regression in retry precedence https://bugs.ruby-lang.org/issues/20747 * Author: byroot (Jean Boussier) * Status: Open * Assignee: kddnewton (Kevin Newton) * ruby -v: ruby 3.4.0dev (2024-09-15T01:06:11Z master 532af89e3b) +PRISM [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since prism has been made the default, our test suite now fail to boot with the following error: ```ruby syntax error found (SyntaxError) 298 | raise e 299 | else > 300 | retry | ^~~~~ Invalid retry after else 301 | end 302 | end ``` Here's the code in question, that reproduce the error: ```ruby def retry_thing(times: 1) retries ||= 0 begin yield rescue RuntimeError => e raise if (retries += 1) > times begin refresh! rescue raise e else retry end end end ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119207] [Ruby master Bug#20744] syntax error found (SyntaxError) `UTF-8 mixed within US-ASCII source` reported Ruby with Prism parser
by yahonda (Yasuo Honda) 16 Sep '24

16 Sep '24

Issue #20744 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #20744: syntax error found (SyntaxError) `UTF-8 mixed within US-ASCII source` reported Ruby with Prism parser https://bugs.ruby-lang.org/issues/20744 * Author: yahonda (Yasuo Honda) * Status: Open * ruby -v: ruby 3.4.0dev (2024-09-15T01:06:11Z master 532af89e3b) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I'm reporting this issue based on Rails Nightly CI against Ruby 3.4.0dev. https://buildkite.com/rails/rails-nightly/builds/1023#0191ed31-1d77-4705-ab… I have not able to create a minimum test case without Rails yet. ### Steps to reproduce ```ruby git clone https://github.com/rails/rails cd rails/actionview bin/test test/template/template_test.rb:278 ``` ### Expected behavior It should pass as it Runs parser.y parser ```ruby $ RUBYOPT=--parser=parse.y bin/test test/template/template_test.rb:278 /home/yahonda/.gem/ruby/3.4.0+0/gems/json-2.7.1/lib/json/common.rb:3: warning: ostruct was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0. You can add ostruct to your Gemfile or gemspec to silence this warning. Running 36 tests in a single process (parallelization threshold is 50) Run options: --seed 61197 # Running: . Finished in 0.037914s, 26.3756 runs/s, 79.1267 assertions/s. 1 runs, 3 assertions, 0 failures, 0 errors, 0 skips $ ``` ### Actual behavior It raises the following `syntax error found (SyntaxError)` ```ruby $ bin/test test/template/template_test.rb:278 /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.4.0+0/bundled_gems.rb:79:in 'Kernel.require': /home/yahonda/src/github.com/rails/rails/actionview/test/template/template_test.rb:278: syntax error found (SyntaxError) 276 | @template = new_template("<%# encoding: ISO-8859-1 %>\n<%# locals: (message: 'Hi!') %>\nhello \xFCmlat\n<%= message %>", virtual_path: nil) 277 | assert_equal Encoding::UTF_8, render.encoding > 278 | assert_match(/hello \u{fc}mlat\nHi!/, render) | ^~ UTF-8 mixed within US-ASCII source 279 | end 280 | end from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.4.0+0/bundled_gems.rb:79:in 'block (2 levels) in Kernel#replace_require' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:62:in 'block in Rails::TestUnit::Runner.load_tests' from <internal:array>:42:in 'Array#each' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:60:in 'Rails::TestUnit::Runner.load_tests' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:52:in 'Rails::TestUnit::Runner.run' from /home/yahonda/src/github.com/rails/rails/tools/test.rb:18:in '<top (required)>' from bin/test:5:in 'Kernel#require_relative' from bin/test:5:in '<main>' $ ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119219] [Ruby master Bug#20748] Issue with defined? given method call with block in prism compiler
by luke-gru (Luke Gruber) 16 Sep '24

16 Sep '24

Issue #20748 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #20748: Issue with defined? given method call with block in prism compiler https://bugs.ruby-lang.org/issues/20748 * Author: luke-gru (Luke Gruber) * Status: Open * ruby -v: 3.4.0dev * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- test.rb ```ruby puts defined?(undefined_method(){}) ``` ``` ./ruby --parser=parse.y -I../ruby/lib -I. -I.ext/x86_64-linux -I.ext/common -r./x86_64-linux-fake ../ruby/test.rb expression ``` ``` ./ruby --parser=prism -I../ruby/lib -I. -I.ext/x86_64-linux -I.ext/common -r./x86_64-linux-fake ../ruby/test.rb # empty line ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119194] [Ruby master Bug#20741] RubyVM::InstructionSequence.compile emits a wrong warning (prism?)
by mame (Yusuke Endoh) 16 Sep '24

16 Sep '24

Issue #20741 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20741: RubyVM::InstructionSequence.compile emits a wrong warning (prism?) https://bugs.ruby-lang.org/issues/20741 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: kddnewton (Kevin Newton) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` $ ./local/bin/ruby -we 'p RubyVM::InstructionSequence.compile("42").eval' <compiled>:1: warning: possibly useless use of a literal in void context 42 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116581] [Ruby master Bug#20237] Unable to unshare(CLONE_NEWUSER) in Linux because of timer thread
by hanazuki (Kasumi Hanazuki) 16 Sep '24

16 Sep '24

Issue #20237 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #20237: Unable to unshare(CLONE_NEWUSER) in Linux because of timer thread https://bugs.ruby-lang.org/issues/20237 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2024-02-04T16:05:02Z master 8bc6fff322) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ## Backgrounds [unshare(2)](https://man7.org/linux/man-pages/man2/unshare.2.html) is a syscall in Linux to move the calling process into a fresh execution context. With `unshare(CLONE_NEWUSER)` you can move a process into a new [user_namespace(7)](https://man7.org/linux/man-pages/man7/user_namespaces.7.…, where the process gains the full capability on the resources within the namespace. This is fundamental for Linux containers to achieve privilege separation. `unshare(CLONE_NEWUSER)` requires the calling process to be single-threaded (or no background threads are running). So, it is often invoked after `fork(2)` as forking propagates only the calling thread to the child process. ## Problem It becomes a problem that Ruby 3.3 on Linux uses timer threads even for a single-`Thread`ed application. Because `Kernel#fork` spawns a thread in the child process before the control returns to the user code, there is no chance to call `unshare(CLONE_NEWUSER)` in Ruby. The following snippet is a reproducer of this problem. This program first forks and then shows the user namespace to which the process belongs before and after calling unshare(2). It also shows the threads of the child process after forking. ```ruby p(RUBY_DESCRIPTION:) require 'fiddle/import' module C extend Fiddle::Importer dlload 'libc.so.6' extern 'int unshare(int flags)' CLONE_NEWUSER = 0x10000000 def self.raise_system_call_error raise SystemCallError.new(Fiddle.last_error) end end pid = fork do system("ps -O tid -T -p #$$") system("ls -l /proc/self/ns/user") if C.unshare(C::CLONE_NEWUSER) != 0 C.raise_system_call_error # => EINVAL with Ruby 3.3 end system("ls -l /proc/self/ns/user") end p Process.wait2(pid) ``` The program successfully changes the user namespace with Ruby 3.2, but it raises EINVAL with Ruby 3.3. You can see Ruby 3.3 has two threads running after forking. ``` % rbenv shell 3.2 && ruby ./test.rb {:RUBY_DESCRIPTION=>"ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux]"} PID TID S TTY TIME COMMAND 1585787 1585787 S pts/12 00:00:00 ruby ./test.rb lrwxrwxrwx 1 kasumi kasumi 0 Feb 5 02:25 /proc/self/ns/user -> 'user:[4026531837]' lrwxrwxrwx 1 nobody nogroup 0 Feb 5 02:25 /proc/self/ns/user -> 'user:[4026532675]' [1585787, #<Process::Status: pid 1585787 exit 0>] % rbenv shell 3.3 && ruby ./test.rb {:RUBY_DESCRIPTION=>"ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux]"} PID TID S TTY TIME COMMAND 1585849 1585849 S pts/12 00:00:00 ruby ./test.rb 1585849 1585851 S pts/12 00:00:00 ruby ./test.rb lrwxrwxrwx 1 kasumi kasumi 0 Feb 5 02:25 /proc/self/ns/user -> 'user:[4026531837]' ./test.rb:10:in `raise_system_call_error': Invalid argument (Errno::EINVAL) from ./test.rb:24:in `block in <main>' from ./test.rb:19:in `fork' from ./test.rb:19:in `<main>' [1585849, #<Process::Status: pid 1585849 exit 1>] % rbenv shell master && ruby ./test.rb {:RUBY_DESCRIPTION=>"ruby 3.4.0dev (2024-02-04T16:05:02Z master 8bc6fff322) [x86_64-linux]"} PID TID S TTY TIME COMMAND 1585965 1585965 S pts/12 00:00:00 ruby ./test.rb 1585965 1585967 S pts/12 00:00:00 ruby ./test.rb lrwxrwxrwx 1 kasumi kasumi 0 Feb 5 02:25 /proc/self/ns/user -> 'user:[4026531837]' ./test.rb:10:in `raise_system_call_error': Invalid argument (Errno::EINVAL) from ./test.rb:24:in `block in <main>' from ./test.rb:19:in `fork' from ./test.rb:19:in `<main>' [1585965, #<Process::Status: pid 1585965 exit 1>] ``` ## Workaround My workaround is to rebuild ruby with `rb_thread_stop_timer_thread` and `rb_thread_start_timer_thread` exported, and use a C-ext that stops the timer thread before calling `unshare`. This seems not robust because the process cannot know when the terminated thread is reclaimed by the kernel, after which the process is considered single-threaded. ```c #define _GNU_SOURCE 1 #include <sched.h> #include <ruby/ruby.h> static VALUE Unshare_s_unshare(VALUE _self, VALUE rflags) { int const flags = NUM2INT(rflags); rb_thread_stop_timer_thread(); usleep(1000); // FIXME: It takes some time for the kernel to remove the stopped thread? int const ret = unshare(flags); rb_thread_start_timer_thread(); if(ret != 0) rb_sys_fail_str(rb_sprintf("unshare(%#x)", flags)); return Qnil; } RUBY_FUNC_EXPORTED void Init_unshare(void) { VALUE rb_mUnshare = rb_define_module("Unshare"); rb_define_singleton_method(rb_mUnshare, "unshare", Unshare_s_unshare, 1); rb_define_const(rb_mUnshare, "CLONE_NEWUSER", INT2FIX(CLONE_NEWUSER)); } ``` ## Questions - Is this a limitation of Ruby? - Is it safe (or even possible) to stop the timer thread during execution? - If so, can we export it as the public API? - But it may not so useful for this problem as explained in the workaround. - Is it guaranteed that no other threads are running after forks? - Are there any better ways to solve this issue? - Can we somehow delay the start of the timer thread after forking, or hook into `fork` to run some code in the child process immediately after it spawns. - Can they be Ruby API instead of C API? -- https://bugs.ruby-lang.org/

4 9

[ruby-core:119163] [Ruby master Bug#20736] prism emits wrong warnings in syntax-error code
by mame (Yusuke Endoh) 15 Sep '24

15 Sep '24

Issue #20736 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20736: prism emits wrong warnings in syntax-error code https://bugs.ruby-lang.org/issues/20736 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: kddnewton (Kevin Newton) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` $ ./local/bin/ruby -w --parser=parse.y -e 'begin eval("0a"); rescue SyntaxError; end' $ ./local/bin/ruby -w --parser=prism -e 'begin eval("0a"); rescue SyntaxError; end' (eval at -e:1):1: warning: possibly useless use of a literal in void context ``` ``` $ ./local/bin/ruby -w --parser=parse.y -e 'begin eval("+a.0"); rescue SyntaxError; end' $ ./local/bin/ruby -w --parser=prism -e 'begin eval("+a.0"); rescue SyntaxError; end' (eval at -e:1):1: warning: possibly useless use of +@ in void context ``` -- https://bugs.ruby-lang.org/

3 6