- ruby-core - ml.ruby-lang.org

[ruby-core:118254] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 08 Jun '24

08 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). mame (Yusuke Endoh) wrote in #note-23: > [...] instead of parsing the command line to a string array and passing it to `Kernel#exec`. Don't we use `execve()` (which takes `char**`) as well on Windows for `Kernel#exec`? It does seem to exist: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/execve-we… Did you mean `Kernel#spawn`/`Kernel#system` maybe? Of course, `RbConfig.ruby_args` would return an Array, not a String. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108751 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118253] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 08 Jun '24

08 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). shyouhei (Shyouhei Urabe) wrote in #note-29: > The current situation is that ruby is not the only valid executable that the method takes. Allowing untrusted inputs for it means it has to be secure for everything. This is too much a hustle. Better find a fine-grained alternative. There is no untrusted input involved here, because the user chooses what flags to pass to `ruby`. If ruby flags can be injected by an attacker, then all is lost regardless of this change (e.g. they can just inject `-r/backdoor.rb`). Regarding the Windows concern, it is the exact same problem for e.g. `spawn("dir", "*.mp3", "/s")`. From what I can see, it is completely separate from this ticket. The code for this on Windows must already escape as much as feasible, and if it fails it's a bug of that code which should be fixed to fix `spawn` etc in general, nothing to change in `RbConfig.ruby_args`. Or if the escaping fails maybe it's just considered a Windows limitation, independent of this ticket. For example, the user is running `ruby --yjit -rmytracing script.rb`, the only addition is the script can now find out the ruby flags it was called with (`["--yjit", "-rmytracing"]`). If the script spawn subprocesses, it already did before, so nothing changes there. It can choose to use `RbConfig.ruby_args`, and that's fine, the user running the script is responsible for whether it's safe to run the script, as always. Let's take the MSpec use-case (mentioned before in https://bugs.ruby-lang.org/issues/6648#note-11) what we want is to run Ruby subprocess with the same Ruby flags. So if e.g. `ruby --yjit -rmytracing path/to/mspec` is called, then if specs create subprocesses (via `ruby_exe()`), then those subprocesses (running some fixture) also have `--yjit -rmytracing`, as desired. You might argue `RUBYOPT` could be used instead, but that is problematic for various reasons: some flags are not allowed in RUBYOPT, RUBYOPT gets propagated arbitrarily far which is not necessarily desired (including to other Ruby implementations and executables written in Ruby, etc). A concrete example I often run into is running ruby/spec with TruffleRuby, I pass `--core-load-path=.../src/main/ruby/truffleruby` to use core library files from disk in development. It is critical that subprocesses in specs also use that (otherwise we'd get an inconsistent core library). The current workaround is to pass that flag both to the ruby process and as `-T`, which is quite ugly but it also slow, because it means mspec must create an extra subprocess just to apply these `-T` flags (it actually uses `exec` but that's just as slow): ``` $ mxbuild/truffleruby-jvm/bin/ruby \ --experimental-options --core-load-path=src/main/ruby/truffleruby \ spec/mspec/bin/mspec run \ --config spec/truffleruby.mspec \ -t .../mxbuild/truffleruby-jvm/bin/ruby \ --excl-tag fails --excl-tag slow \ -T--vm.ea -T--vm.esa \ -T--experimental-options -T--core-load-path=src/main/ruby/truffleruby ``` (as you can see there are already some bugs there because the -T and regular flags don't match exactly). (if you think this could be wrapped in some helper script, it already is, but it changes nothing because it must accept arbitrary ruby flags to be passed) With `RbConfig.ruby_args`, MSpec can know which ruby flags it was passed, which would avoid needing the extra subprocess, and it would be: ``` $ mxbuild/truffleruby-jvm/bin/ruby \ --vm.ea --vm.esa \ --experimental-options --core-load-path=src/main/ruby/truffleruby \ spec/mspec/bin/mspec run \ --config spec/truffleruby.mspec \ -t .../mxbuild/truffleruby-jvm/bin/ruby \ --excl-tag fails --excl-tag slow ``` This would be a killer feature when attaching a debugger, because then one could just `myruby -rmydebug spec/mspec/bin/mspec` and it would start running specs with the debugger (e.g. for TruffleRuby with the Java debugger). Instead of the current situation where the debugger is started on this mspec "wrapper" which just exec's to handle `-T` flags and is very annoying. This happens in CRuby just as much, for example https://github.com/ruby/ruby/blob/69c0b1438a45938e79e63407035f116de4634dcb/… is a workaround causing some duplication. And it makes much more messy to e.g. running a single spec under `gdb`/`lldb`. A very similar situation happens for `make test-all` I would imagine. I guess currently any Ruby subprocesses incorrectly omits Ruby flags, which means the test coverage is lower than intended (e.g. for --yjit, --rjit and all other flags). This would also be convenient for the way to run built-but-not-installed-ruby, which every CRuby developer uses. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108750 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118147] [Ruby master Misc#20519] Porting regexp to pure ruby?
by brightbits (Michael Baldry) 07 Jun '24

07 Jun '24

Issue #20519 has been reported by brightbits (Michael Baldry). ---------------------------------------- Misc #20519: Porting regexp to pure ruby? https://bugs.ruby-lang.org/issues/20519 * Author: brightbits (Michael Baldry) * Status: Open ---------------------------------------- Would there be any benefit in porting Regexp from Onigmo to a pure ruby implementation that could benefit from YJIT? Compiling a pattern could be translating to a ruby method which would be optimized by YJIT easily. Has this been explored or any work done around this kind of thing, before I take a look in to it more? Many thanks -- https://bugs.ruby-lang.org/

3 4

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:118242] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by shyouhei (Shyouhei Urabe) 07 Jun '24

07 Jun '24

Issue #6648 has been updated by shyouhei (Shyouhei Urabe). Please note that I'm not necessarily against a way to call the current ruby executable. I just say doing so using exec is a bad idea, because exec is not designed for that purpose. The current situation is that ruby is not the only valid executable that the method takes. Allowing untrusted inputs for it means it has to be secure for everything. This is too much a hustle. Better find a fine-grained alternative. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108741 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118241] [Ruby master Feature#18773] deconstruct to receive a range
by Eregon (Benoit Daloze) 07 Jun '24

07 Jun '24

Issue #18773 has been updated by Eregon (Benoit Daloze). FWIW I think CRuby currently support subclassing Array and redefining `[]` and pattern matching will call that on CRuby, maybe it can be used for this purpose. I don't think it's a good idea to rely on this behavior though, it might change. For example that doesn't work on TruffleRuby, which currently always uses standard `Array#[]` from pattern matching. ---------------------------------------- Feature #18773: deconstruct to receive a range https://bugs.ruby-lang.org/issues/18773#change-108740 * Author: kddnewton (Kevin Newton) * Status: Rejected * Assignee: ktsj (Kazuki Tsujimoto) ---------------------------------------- Currently when you're pattern matching against a hash pattern, `deconstruct_keys` receives the keys that are being matched. This is really useful for computing expensive hashes. However, when you're pattern matching against an array pattern, you don't receive any information. So if the array is expensive to compute (for instance loading an array of database records), you have no way to bail out. It would be useful to receive a range signifying how many records the pattern is specifying. It would be used like the following: ```ruby class ActiveRecord::Relation def deconstruct(range) (loaded? || range.cover?(count)) ? records : nil end end ``` It needs to be a range and not just a number to handle cases where `*` is used. You would use it like: ```ruby case Person.all in [] "No records" in [person] "Only #{person.name}" else "Multiple people" end ``` In this way, you wouldn't have to load the whole thing into memory to check if it pattern matched. The patch is here: https://github.com/ruby/ruby/pull/5905. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118240] [Ruby master Feature#18773] deconstruct to receive a range
by Eregon (Benoit Daloze) 07 Jun '24

07 Jun '24

Issue #18773 has been updated by Eregon (Benoit Daloze). The implementation of pattern matching needs to read at indices from the return value of `deconstruct`, but `Enumerable` doesn't have `[]` or `at` or anything like that. So I think it is difficult to support and could very well end up making pattern matching slower. There is a similar issue in the issue description, it would prevent caching the result of `deconstruct`, as already pointed out. I think pattern shouldn't do too crazy things (like calling `each` with a block behind the scenes), I think simple is good. ---------------------------------------- Feature #18773: deconstruct to receive a range https://bugs.ruby-lang.org/issues/18773#change-108739 * Author: kddnewton (Kevin Newton) * Status: Rejected * Assignee: ktsj (Kazuki Tsujimoto) ---------------------------------------- Currently when you're pattern matching against a hash pattern, `deconstruct_keys` receives the keys that are being matched. This is really useful for computing expensive hashes. However, when you're pattern matching against an array pattern, you don't receive any information. So if the array is expensive to compute (for instance loading an array of database records), you have no way to bail out. It would be useful to receive a range signifying how many records the pattern is specifying. It would be used like the following: ```ruby class ActiveRecord::Relation def deconstruct(range) (loaded? || range.cover?(count)) ? records : nil end end ``` It needs to be a range and not just a number to handle cases where `*` is used. You would use it like: ```ruby case Person.all in [] "No records" in [person] "Only #{person.name}" else "Multiple people" end ``` In this way, you wouldn't have to load the whole thing into memory to check if it pattern matched. The patch is here: https://github.com/ruby/ruby/pull/5905. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118238] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Dan0042 (Daniel DeLorme) 07 Jun '24

07 Jun '24

Issue #6648 has been updated by Dan0042 (Daniel DeLorme). > As you see there is no Windows API that takes `char**`. We cannot safely pass through what we have. You have to concatenate them into one argument string (`LPWSTR lpCommandLine`), with proper escaping of whitespace etc. But this issue is not specific to `RbConfig.ruby_args` is it? You have to do the concatenation in exec/system anyway; `RbConfig.ruby_args` will not change this situation either for better or worse. > Because process arguments come from out of the process itself by nature, there is no guarantee that they are written by good will. Can you explain that one? I don't understand how valid ruby options like `--enable=jit` could be "not written by good will". ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108737 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118236] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Dan0042 (Daniel DeLorme) 07 Jun '24

07 Jun '24

Issue #6648 has been updated by Dan0042 (Daniel DeLorme). shyouhei (Shyouhei Urabe) wrote in #note-26: > In short the problem we see is feeding strings from untrusted sources to generic `Kernel#exec`. Sounds ultra risky, no? It also sounds nothing like what this proposal is about. If the current script was executed with `ruby --enable=jit foo.rb` then it is, by definition, safe to run `exec("ruby", "--enable=jit", "foo.rb")` It's already possible to run `exec("ruby", "foo.rb")`; changing it to `exec("ruby", *RbConfig.ruby_args, "foo.rb")` does not reduce security, in fast it **increases** security. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108736 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116117] [Ruby master Misc#20170] Drop support for GCC < 11
by kddnewton (Kevin Newton) 07 Jun '24

07 Jun '24

Issue #20170 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20170: Drop support for GCC < 11 https://bugs.ruby-lang.org/issues/20170 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers. -- https://bugs.ruby-lang.org/

4 5