June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:115334] [Ruby master Feature#19998] Emit deprecation warnings when the old (non-Typed) Data_XXX API is used
by byroot (Jean Boussier) 06 Jun '24

06 Jun '24

Issue #19998 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19998: Emit deprecation warnings when the old (non-Typed) Data_XXX API is used https://bugs.ruby-lang.org/issues/19998 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- The replacement API was introduced in Ruby 1.9.2 (2010) [Feature #3064], and the old untyped data API was marked a deprecated in the documentation as of Ruby 2.3.0 (2015): https://github.com/ruby/ruby/commit/98544c372d948717de22afc86d162e411f1fb5f1 However, as of today, no deprecation warning of any sort is emitted when the API is used. Ultimately removing the old untyped API would allow to easily reclaim 8B on every `T_DATA` which could be used by the newly introduced embedded TypedData objects https://github.com/ruby/ruby/pull/7440 Was there any discussions about removing the old API at some point? Could we emit a warning when the API is used? Either verbose or non-verbose, to at least surface the issue to gem owners? This would help drive the effort to fix outdated gems, allowing to eventually remove this old API in the future. Looking at our big applications dependencies, there seem to be a small minority of gems still using the old API: - https://rubygems.org/gems/unf_ext - https://rubygems.org/gems/rotoscope - https://rubygems.org/gems/re2 - https://rubygems.org/gems/mysql2 Migrating them over doesn't seem too hard. -- https://bugs.ruby-lang.org/

6 11

[ruby-core:117027] [Ruby master Bug#20319] Singleton class is being frozen lazily in some cases
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20319 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20319: Singleton class is being frozen lazily in some cases https://bugs.ruby-lang.org/issues/20319 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've noticed suspicious behaviour (it doesn't affect anything in practice for me though) when an object becomes frozen only its own singleton class becomes frozen immediately. A singleton class of the immediate singleton class becomes frozen lazily after `#singleton_class` method call: ```ruby o = Object.new klass = o.singleton_class.singleton_class o.freeze puts klass.frozen? # false <== here we expect true puts o.singleton_class.singleton_class.frozen? # true puts klass.frozen? # true ``` I would expect all created singleton classes in an object singleton classes chain to become frozen immediately. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:114038] [Ruby master Bug#19749] Confirm correct behaviour when attaching private method with `#define_method`
by itarato (Peter Arato) 06 Jun '24

06 Jun '24

Issue #19749 has been reported by itarato (Peter Arato). ---------------------------------------- Bug #19749: Confirm correct behaviour when attaching private method with `#define_method` https://bugs.ruby-lang.org/issues/19749 * Author: itarato (Peter Arato) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This issue is a special case of https://bugs.ruby-lang.org/issues/19745: Should dynamically added private methods via `.singleton_class.send(:define_method,...` be accessible publicly? See the following example? ```ruby private def bar; end foo = Object.new foo.singleton_class.send(:define_method, :bar, method(:bar)) foo.bar # No error. ``` The script above runs fine on latest Ruby 3.3. Is this correct to ignore the fact that the added method (method(:bar)) is private? This came up during a TruffleRuby investigation (https://github.com/oracle/truffleruby/issues/3134) where the result for the same script is: `private method 'bar' called for #<Object:0xc8> (NoMethodError)` -- https://bugs.ruby-lang.org/

5 14

[ruby-core:117990] [Ruby master Misc#20504] Interpolated string literal in regexp encoding handling
by kddnewton (Kevin Newton) 06 Jun '24

06 Jun '24

Issue #20504 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20504: Interpolated string literal in regexp encoding handling https://bugs.ruby-lang.org/issues/20504 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- There is some very odd behavior that I'm not sure is intentional or not, so I'm looking for guidance. In here: ```ruby # encoding: us-ascii interp = "\x80" regexp = /#{interp}/ ``` the `regexp` variable is a ascii-8bit regular expression with the byte interpolated into the middle. However, if you inline that interpolation: ```ruby # encoding: us-ascii regexp = /#{"\x80"}/ ``` you get a syntax error, saying it's an invalid multi-byte character. I'm not sure what the rule is here, as it seems inconsistent. Is this the correct behavior? I would prefer if it would create an ascii-8bit regular expression like the first example, which would be consistent. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118194] [Ruby master Bug#18622] const_get still looks in Object, while lexical constant lookup no longer does
by matz (Yukihiro Matsumoto) 06 Jun '24

06 Jun '24

Issue #18622 has been updated by matz (Yukihiro Matsumoto). Status changed from Open to Closed Sorry, I forgot to reply. Due to the compatibility concern, I'd like to keep this behavior even though it's inconsistent with direct constant accesses. Matz. ---------------------------------------- Bug #18622: const_get still looks in Object, while lexical constant lookup no longer does https://bugs.ruby-lang.org/issues/18622#change-108677 * Author: Eregon (Benoit Daloze) * Status: Closed * ruby -v: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- There is some inconsistency here between literal constant lookup and the meta API (const_get). Lexical constant lookup no longer uses a special case for Object, and this is good as it avoids surprises: #11547 However, `const_get` still looks in Object, even though that's confusing, inconsistent and IMHO shouldn't really happen. ```ruby module ConstantSpecsTwo Foo = :cs_two_foo end module ConstantSpecs end p ConstantSpecs.const_get("ConstantSpecsTwo::Foo") # => :cs_two_foo p ConstantSpecs::ConstantSpecsTwo::Foo # => const_get.rb:9:in `<main>': uninitialized constant ConstantSpecs::ConstantSpecsTwo (NameError) ``` I think we should change it so both behave the same (i.e., NameError). It's like if `cd /foo/bar` would go to `/bar` if `/foo/bar` does not exist and `/bar` does. `const_get` is a meta API so it cannot know the surrounding `Module.nesting`, but so I think it should consider the receiver of `const_get` as the only nesting (so just `ConstantSpecs` in this case, not `Object`). Note this does not affect nested constants inside the `const_get` like `Foo` above (another way to look at the inconsistency that the first component is treated differently). From https://bugs.ruby-lang.org/issues/11547#note-19 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118193] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Dan0042 (Daniel DeLorme) 05 Jun '24

05 Jun '24

Issue #6648 has been updated by Dan0042 (Daniel DeLorme). > IMO it shouldn't exist, redundant with `$0` Keep in mind that `$0` can be set as process name, so `Process.argv0` is not redundant (despite the unfortunate naming). > I like your proposed naming in https://bugs.ruby-lang.org/issues/6648#note-10 but I think we should add `RbConfig.ruby_args` before the rest and file a new ticket for the rest. Agreed. This will also allow me to make a clearer point for the security risk of re-executing `$0` when it is equal to "-e" ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108676 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118192] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 05 Jun '24

05 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). Dan0042 (Daniel DeLorme) wrote in #note-20: > I tried, and `--enable-load-relative` doesn't appear to be a supported option in any version of ruby., It's a `./configure` option: `./configure --enable-load-relative`. Sorry I should have made that clear. > > (BTW there is `Process.argv0` which is about (Ruby) `ARGV[0]` and not (C) `argv[0]` > > I'm afraid not; `Process.argv0` is about ruby `$0` which is very different from `ARGV[0]` Ah right, the name of that method is so confusing (IMO it shouldn't exist, redundant with `$0`). It's mostly like `argv[0]` in C but it returns the Ruby script path being run (vs path of the current executable) and yet it's not `ARGV[0]`. So sort of related to this issue, but so awfully confusing I don't think we want to follow that unfortunate naming. I like your proposed naming in https://bugs.ruby-lang.org/issues/6648#note-10 but I think we should add `RbConfig.ruby_args` before the rest and file a new ticket for the rest. (re-executing the same script with the same arguments is a special case, there are more use cases for `RbConfig.ruby_args`) ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108674 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118172] [Ruby master Misc#20523] mswin and MSFT/vcpkg
by MSP-Greg (Greg L) 05 Jun '24

05 Jun '24

Issue #20523 has been reported by MSP-Greg (Greg L). ---------------------------------------- Misc #20523: mswin and MSFT/vcpkg https://bugs.ruby-lang.org/issues/20523 * Author: MSP-Greg (Greg L) * Status: Open ---------------------------------------- Currently the `windows.yml`workflow caches MSFT/vcpkg packages and the `vcpkg.json` file locks the vcpkg repo to an older commit. This results in the following package versions being used: ``` Package Ruby CI Current vcpkg libffi 3.4.4#1 3.4.6 libyaml 0.2.5#4 0.2.5#4 openssl 3.1.1 3.3.0#1 readline 0#4 0#5 readline-win32 na 5.0#8 zlib 1.2.13 1.3.1 ``` Note the updated versions of libffi, openssl, and zlib. There seems to be a concern about using packages from vcpkg’s master branch (see [PR #10907](https://github.com/ruby/ruby/pull/10907)). Ruby-loco’s mswin build has been running for almost 5 years, and many of the extension gems that are part of the std-lib use vcpkg master branch packages in their CI. I don’t recall any significant issues with vcpkg. Depending on the OS, current dependency package versions are often not available. Ruby’s package dependencies are not part of the Windows OS, and MSYS2 and vcpkg normally upgrade packages when they’ve been verified to work with all packages that depend on them. Hence, Windows CI in Ruby master is an opportunity to check master against the most recent versions of its dependencies. I would like to propose that either: 1. vcpkg’s commit is updated more frequently to use current package 2. vcpkg master branch is used -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118188] [Ruby master Bug#18061] Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found
by vo.x (Vit Ondruch) 05 Jun '24

05 Jun '24

Issue #18061 has been updated by vo.x (Vit Ondruch). vo.x (Vit Ondruch) wrote in #note-35: > Just FTR, the annocheck test is incomplete in case `--enable-shared` is used, because the test is checking just `ruby` executable, while it should also check `libruby.so`. The test should be likely executed above all executable ELF files and object files. I think that something like `annocheck . 2>/dev/null | grep FAIL` could do the job, because `annocheck` scans the whole directory structure. Here is the result on my system: ~~~ $ annocheck . 2>/dev/null | grep FAIL | less $ annocheck . 2>/dev/null | grep FAIL Hardened: Context.o: Overall: FAIL (due to MAYB results). Hardened: miniruby: FAIL: cf-protection test because .note.gnu.property section did not contain the necessary flags Hardened: miniruby: FAIL: property-note test because a property note was found but it shows that cf-protection is not enabled Hardened: miniruby: Overall: FAIL. Hardened: libruby-static.a:Context.o: Overall: FAIL (due to MAYB results). Hardened: libruby.so.3.3.1: FAIL: cf-protection test because .note.gnu.property section did not contain the necessary flags Hardened: libruby.so.3.3.1: FAIL: property-note test because a property note was found but it shows that cf-protection is not enabled Hardened: libruby.so.3.3.1: Overall: FAIL. ~~~ ---------------------------------------- Bug #18061: Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found https://bugs.ruby-lang.org/issues/18061#change-108639 * Author: jaruga (Jun Aruga) * Status: Open * Backport: 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: REQUIRED ---------------------------------------- I found an issue in our company's internal test called "execshield" by a security tool annobin - annocheck command [1][2]. ``` Hardened: libruby.so.2.7.4: FAIL: property-note test because no .note.gnu.property section found ``` Here is the reproducer on the upstream latest master, commit is 5f2987d6c2ae9ace3178ac3e1bbb4ac7079101eb, ``` $ autoconf $ ./configure --enable-shared $ make $ ls libruby.so.3.1.0 libruby.so.3.1.0* ``` If you are using Red Hat based Linux distro, it's easy to install by the RPM package like this. ``` $ sudo dnf -y install annobin-annocheck ``` ``` $ sudo yum -y install annobin-annocheck ``` Then ``` $ annocheck libruby.so.3.1.0 ``` If you are using other Linux distros such as Ubuntu, you can use it by a container I prepared. Prepare the following `Dockerfile`. ``` $ cat Dockerfile FROM docker.io/fedora:34 RUN cat /etc/fedora-release RUN dnf -y install annobin-annocheck WORKDIR /work ``` Then build the container image with the `Dockerfile` and run the annocheck command for the `libruby.so.3.1.0` on your host environment. The `-v` is an option for bind mount between host and container environment. ``` $ docker build --rm -t fedora-annocheck . $ docker run --rm -t -v $(pwd):/work fedora-annocheck annocheck /work/libruby.so.3.1.0 annocheck: Version 9.79. Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found Hardened: Rerun annocheck with --verbose to see more information on the tests. ``` The message `Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found` is what I found in our internal test. For other FAIL messages, maybe it can be fixed by changing how to build. Asking a colleague, I was told that the `coroutine/*/Context.S` files such as [coroutine/x86/Context.S](https://github.com/ruby/ruby/blob/master/coroutine… cause the failure. Do you have any idea how to fix this? Thanks. * [1] https://sourceware.org/annobin/ * [2] You can see `man annocheck` or https://www.mankier.com/1/annocheck . ---Files-------------------------------- 0001-Add-.note.gnu.property-sections.patch (2.64 KB) 0001-Add-.note.gnu.property-sections.patch (3.69 KB) config-pie.log (11.4 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118187] [Ruby master Bug#18061] Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found
by vo.x (Vit Ondruch) 05 Jun '24

05 Jun '24

Issue #18061 has been updated by vo.x (Vit Ondruch). Just FTR, the annocheck test is incomplete in case `--enable-shared` is used, because the test is checking just `ruby` executable, while it should also check `libruby.so`. ---------------------------------------- Bug #18061: Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found https://bugs.ruby-lang.org/issues/18061#change-108638 * Author: jaruga (Jun Aruga) * Status: Open * Backport: 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: REQUIRED ---------------------------------------- I found an issue in our company's internal test called "execshield" by a security tool annobin - annocheck command [1][2]. ``` Hardened: libruby.so.2.7.4: FAIL: property-note test because no .note.gnu.property section found ``` Here is the reproducer on the upstream latest master, commit is 5f2987d6c2ae9ace3178ac3e1bbb4ac7079101eb, ``` $ autoconf $ ./configure --enable-shared $ make $ ls libruby.so.3.1.0 libruby.so.3.1.0* ``` If you are using Red Hat based Linux distro, it's easy to install by the RPM package like this. ``` $ sudo dnf -y install annobin-annocheck ``` ``` $ sudo yum -y install annobin-annocheck ``` Then ``` $ annocheck libruby.so.3.1.0 ``` If you are using other Linux distros such as Ubuntu, you can use it by a container I prepared. Prepare the following `Dockerfile`. ``` $ cat Dockerfile FROM docker.io/fedora:34 RUN cat /etc/fedora-release RUN dnf -y install annobin-annocheck WORKDIR /work ``` Then build the container image with the `Dockerfile` and run the annocheck command for the `libruby.so.3.1.0` on your host environment. The `-v` is an option for bind mount between host and container environment. ``` $ docker build --rm -t fedora-annocheck . $ docker run --rm -t -v $(pwd):/work fedora-annocheck annocheck /work/libruby.so.3.1.0 annocheck: Version 9.79. Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found Hardened: Rerun annocheck with --verbose to see more information on the tests. ``` The message `Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found` is what I found in our internal test. For other FAIL messages, maybe it can be fixed by changing how to build. Asking a colleague, I was told that the `coroutine/*/Context.S` files such as [coroutine/x86/Context.S](https://github.com/ruby/ruby/blob/master/coroutine… cause the failure. Do you have any idea how to fix this? Thanks. * [1] https://sourceware.org/annobin/ * [2] You can see `man annocheck` or https://www.mankier.com/1/annocheck . ---Files-------------------------------- 0001-Add-.note.gnu.property-sections.patch (2.64 KB) 0001-Add-.note.gnu.property-sections.patch (3.69 KB) config-pie.log (11.4 KB) -- https://bugs.ruby-lang.org/

1 0