- ruby-core - ml.ruby-lang.org

[ruby-core:118261] [Ruby master Bug#18061] Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found
by kjtsanaktsidis (KJ Tsanaktsidis) 09 Jun '24

09 Jun '24

Issue #18061 has been updated by kjtsanaktsidis (KJ Tsanaktsidis). Assignee set to kjtsanaktsidis (KJ Tsanaktsidis) ---------------------------------------- Bug #18061: Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found https://bugs.ruby-lang.org/issues/18061#change-108759 * Author: jaruga (Jun Aruga) * Status: Open * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: REQUIRED ---------------------------------------- I found an issue in our company's internal test called "execshield" by a security tool annobin - annocheck command [1][2]. ``` Hardened: libruby.so.2.7.4: FAIL: property-note test because no .note.gnu.property section found ``` Here is the reproducer on the upstream latest master, commit is 5f2987d6c2ae9ace3178ac3e1bbb4ac7079101eb, ``` $ autoconf $ ./configure --enable-shared $ make $ ls libruby.so.3.1.0 libruby.so.3.1.0* ``` If you are using Red Hat based Linux distro, it's easy to install by the RPM package like this. ``` $ sudo dnf -y install annobin-annocheck ``` ``` $ sudo yum -y install annobin-annocheck ``` Then ``` $ annocheck libruby.so.3.1.0 ``` If you are using other Linux distros such as Ubuntu, you can use it by a container I prepared. Prepare the following `Dockerfile`. ``` $ cat Dockerfile FROM docker.io/fedora:34 RUN cat /etc/fedora-release RUN dnf -y install annobin-annocheck WORKDIR /work ``` Then build the container image with the `Dockerfile` and run the annocheck command for the `libruby.so.3.1.0` on your host environment. The `-v` is an option for bind mount between host and container environment. ``` $ docker build --rm -t fedora-annocheck . $ docker run --rm -t -v $(pwd):/work fedora-annocheck annocheck /work/libruby.so.3.1.0 annocheck: Version 9.79. Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found Hardened: Rerun annocheck with --verbose to see more information on the tests. ``` The message `Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found` is what I found in our internal test. For other FAIL messages, maybe it can be fixed by changing how to build. Asking a colleague, I was told that the `coroutine/*/Context.S` files such as [coroutine/x86/Context.S](https://github.com/ruby/ruby/blob/master/coroutine… cause the failure. Do you have any idea how to fix this? Thanks. * [1] https://sourceware.org/annobin/ * [2] You can see `man annocheck` or https://www.mankier.com/1/annocheck . ---Files-------------------------------- 0001-Add-.note.gnu.property-sections.patch (2.64 KB) 0001-Add-.note.gnu.property-sections.patch (3.69 KB) config-pie.log (11.4 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118259] [Ruby master Feature#20567] Net::HTTPResponse#detect_encoding should default to UTF-8 for application/javascript
by postmodern (Hal Brodigan) 09 Jun '24

09 Jun '24

Issue #20567 has been reported by postmodern (Hal Brodigan). ---------------------------------------- Feature #20567: Net::HTTPResponse#detect_encoding should default to UTF-8 for application/javascript https://bugs.ruby-lang.org/issues/20567 * Author: postmodern (Hal Brodigan) * Status: Open ---------------------------------------- It is fairly common now for JavaScript to include UTF-8 characters. If a misconfigured web server returns an `application/javascript` response for a JavaScript file, but *without* `charset=utf-8` in the `Content-Type`, `Net::HTTPResponse` will return a response body with encoding of ASCII-8BIT. This can cause lots of subtle String parsing/encoding issues when the response body is parsed or transformed. `Net::HTTPResponse#detect_encoding` should default the response encoding to UTF-8 if the `Content-Type` includes `application/javascript`. [RFC 4329](https://www.rfc-editor.org/rfc/rfc4329#section-4) also seems to encourage defaulting to UTF-8. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118257] [Ruby master Bug#20566] string << 0xC2 should raise a RangeError if the string encoding is Encoding::ASCII
by postmodern (Hal Brodigan) 09 Jun '24

09 Jun '24

Issue #20566 has been reported by postmodern (Hal Brodigan). ---------------------------------------- Bug #20566: string << 0xC2 should raise a RangeError if the string encoding is Encoding::ASCII https://bugs.ruby-lang.org/issues/20566 * Author: postmodern (Hal Brodigan) * Status: Open * ruby -v: ruby 3.4.0preview1 (2024-05-16 master 9d69619623) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I noticed that appending extended ASCII characters (0x80 - 0xff) works when the String's encoding is Encoding::ASCII, as well as Encoding::ASCII_8BIT. I was under the impression that Encoding::ASCII only covers the regular ASCII character set (0x00 - 0x7F). I believe appending 0xC2 to a String with Encoding::ASCII should raise a RangeError. ## Steps To Reproduce ```ruby str = String.new(encoding: Encoding::ASCII) str << 0xC2 ``` ### Expected Result ``` RangeError: character 194 is out of range ``` ### Actual Result ```ruby "\xC2" ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118250] [Ruby master Feature#20565] Relax formal argument at label
by nobu (Nobuyoshi Nakada) 08 Jun '24

08 Jun '24

Issue #20565 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #20565: Relax formal argument at label https://bugs.ruby-lang.org/issues/20565 * Author: nobu (Nobuyoshi Nakada) * Status: Open ---------------------------------------- Just a thought: if `def foo(if:)` is possible, shouldn't `def foo(C:)` and `def foo($g:)` also be possible...? -- https://bugs.ruby-lang.org/

4 3

[ruby-core:118255] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 08 Jun '24

08 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). And from a quick look there is also [`_spawnv`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/spawnv-wspawnv?view=msvc-170) which does take a `char**`, maybe we could use that on Windows? ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108752 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118254] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 08 Jun '24

08 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). mame (Yusuke Endoh) wrote in #note-23: > [...] instead of parsing the command line to a string array and passing it to `Kernel#exec`. Don't we use `execve()` (which takes `char**`) as well on Windows for `Kernel#exec`? It does seem to exist: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/execve-we… Did you mean `Kernel#spawn`/`Kernel#system` maybe? Of course, `RbConfig.ruby_args` would return an Array, not a String. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108751 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118253] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 08 Jun '24

08 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). shyouhei (Shyouhei Urabe) wrote in #note-29: > The current situation is that ruby is not the only valid executable that the method takes. Allowing untrusted inputs for it means it has to be secure for everything. This is too much a hustle. Better find a fine-grained alternative. There is no untrusted input involved here, because the user chooses what flags to pass to `ruby`. If ruby flags can be injected by an attacker, then all is lost regardless of this change (e.g. they can just inject `-r/backdoor.rb`). Regarding the Windows concern, it is the exact same problem for e.g. `spawn("dir", "*.mp3", "/s")`. From what I can see, it is completely separate from this ticket. The code for this on Windows must already escape as much as feasible, and if it fails it's a bug of that code which should be fixed to fix `spawn` etc in general, nothing to change in `RbConfig.ruby_args`. Or if the escaping fails maybe it's just considered a Windows limitation, independent of this ticket. For example, the user is running `ruby --yjit -rmytracing script.rb`, the only addition is the script can now find out the ruby flags it was called with (`["--yjit", "-rmytracing"]`). If the script spawn subprocesses, it already did before, so nothing changes there. It can choose to use `RbConfig.ruby_args`, and that's fine, the user running the script is responsible for whether it's safe to run the script, as always. Let's take the MSpec use-case (mentioned before in https://bugs.ruby-lang.org/issues/6648#note-11) what we want is to run Ruby subprocess with the same Ruby flags. So if e.g. `ruby --yjit -rmytracing path/to/mspec` is called, then if specs create subprocesses (via `ruby_exe()`), then those subprocesses (running some fixture) also have `--yjit -rmytracing`, as desired. You might argue `RUBYOPT` could be used instead, but that is problematic for various reasons: some flags are not allowed in RUBYOPT, RUBYOPT gets propagated arbitrarily far which is not necessarily desired (including to other Ruby implementations and executables written in Ruby, etc). A concrete example I often run into is running ruby/spec with TruffleRuby, I pass `--core-load-path=.../src/main/ruby/truffleruby` to use core library files from disk in development. It is critical that subprocesses in specs also use that (otherwise we'd get an inconsistent core library). The current workaround is to pass that flag both to the ruby process and as `-T`, which is quite ugly but it also slow, because it means mspec must create an extra subprocess just to apply these `-T` flags (it actually uses `exec` but that's just as slow): ``` $ mxbuild/truffleruby-jvm/bin/ruby \ --experimental-options --core-load-path=src/main/ruby/truffleruby \ spec/mspec/bin/mspec run \ --config spec/truffleruby.mspec \ -t .../mxbuild/truffleruby-jvm/bin/ruby \ --excl-tag fails --excl-tag slow \ -T--vm.ea -T--vm.esa \ -T--experimental-options -T--core-load-path=src/main/ruby/truffleruby ``` (as you can see there are already some bugs there because the -T and regular flags don't match exactly). (if you think this could be wrapped in some helper script, it already is, but it changes nothing because it must accept arbitrary ruby flags to be passed) With `RbConfig.ruby_args`, MSpec can know which ruby flags it was passed, which would avoid needing the extra subprocess, and it would be: ``` $ mxbuild/truffleruby-jvm/bin/ruby \ --vm.ea --vm.esa \ --experimental-options --core-load-path=src/main/ruby/truffleruby \ spec/mspec/bin/mspec run \ --config spec/truffleruby.mspec \ -t .../mxbuild/truffleruby-jvm/bin/ruby \ --excl-tag fails --excl-tag slow ``` This would be a killer feature when attaching a debugger, because then one could just `myruby -rmydebug spec/mspec/bin/mspec` and it would start running specs with the debugger (e.g. for TruffleRuby with the Java debugger). Instead of the current situation where the debugger is started on this mspec "wrapper" which just exec's to handle `-T` flags and is very annoying. This happens in CRuby just as much, for example https://github.com/ruby/ruby/blob/69c0b1438a45938e79e63407035f116de4634dcb/… is a workaround causing some duplication. And it makes much more messy to e.g. running a single spec under `gdb`/`lldb`. A very similar situation happens for `make test-all` I would imagine. I guess currently any Ruby subprocesses incorrectly omits Ruby flags, which means the test coverage is lower than intended (e.g. for --yjit, --rjit and all other flags). This would also be convenient for the way to run built-but-not-installed-ruby, which every CRuby developer uses. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108750 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118147] [Ruby master Misc#20519] Porting regexp to pure ruby?
by brightbits (Michael Baldry) 07 Jun '24

07 Jun '24

Issue #20519 has been reported by brightbits (Michael Baldry). ---------------------------------------- Misc #20519: Porting regexp to pure ruby? https://bugs.ruby-lang.org/issues/20519 * Author: brightbits (Michael Baldry) * Status: Open ---------------------------------------- Would there be any benefit in porting Regexp from Onigmo to a pure ruby implementation that could benefit from YJIT? Compiling a pattern could be translating to a ruby method which would be optimized by YJIT easily. Has this been explored or any work done around this kind of thing, before I take a look in to it more? Many thanks -- https://bugs.ruby-lang.org/

3 4

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:118242] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by shyouhei (Shyouhei Urabe) 07 Jun '24

07 Jun '24

Issue #6648 has been updated by shyouhei (Shyouhei Urabe). Please note that I'm not necessarily against a way to call the current ruby executable. I just say doing so using exec is a bad idea, because exec is not designed for that purpose. The current situation is that ruby is not the only valid executable that the method takes. Allowing untrusted inputs for it means it has to be secure for everything. This is too much a hustle. Better find a fine-grained alternative. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108741 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0