December 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:120297] [Ruby master Bug#20962] Change in behaviour in 3.4.0rc1 of `Hash#inspect`
by aharan (Aidan Haran) 18 Dec '24

18 Dec '24

Issue #20962 has been reported by aharan (Aidan Haran). ---------------------------------------- Bug #20962: Change in behaviour in 3.4.0rc1 of `Hash#inspect` https://bugs.ruby-lang.org/issues/20962 * Author: aharan (Aidan Haran) * Status: Open * ruby -v: ruby 3.4.0rc1 (2024-12-12 master 29caae9991) +PRISM [aarch64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The output from `Hash#inspect` in v3.4.0rc1 does not match the documentation and is different from v3.3.6. <pre> # ruby -v ruby 3.4.0rc1 (2024-12-12 master 29caae9991) +PRISM [aarch64-linux] # irb irb(main):001> h = {foo: 0, bar: 1, baz: 2} => {foo: 0, bar: 1, baz: 2} irb(main):002> h.inspect => "{foo: 0, bar: 1, baz: 2}" irb(main):003> </pre> The documentation says that the output should be @"{:foo=>0, :bar=>1, :baz=>2}"@ https://github.com/ruby/ruby/blob/707c6420b1ea599c223a17781185c0b6fb0f3518/… In Ruby v3.3.6 the output is: <pre> # ruby -v ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [aarch64-linux] # irb irb(main):001> h = {foo: 0, bar: 1, baz: 2} => {:foo=>0, :bar=>1, :baz=>2} irb(main):002> h.inspect => "{:foo=>0, :bar=>1, :baz=>2}" </pre> -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118957] [Ruby master Bug#20699] On Windows, the `__dir__` keyword is garbled in paths containing Japanese characters, and `require_relative` fails as well
by flatland001 (Hayato Arai) 18 Dec '24

18 Dec '24

Issue #20699 has been reported by flatland001 (Hayato Arai). ---------------------------------------- Bug #20699: On Windows, the `__dir__` keyword is garbled in paths containing Japanese characters, and `require_relative` fails as well https://bugs.ruby-lang.org/issues/20699 * Author: flatland001 (Hayato Arai) * Status: Open * ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x64-mingw-ucrt] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- In paths containing Japanese characters, such as: C:\テスト_test\test.rb Code: ``` ruby # this code -> C:\テスト_test\test.rb # external file -> C:\テスト_test\lib\foo.rb p __dir__ p __dir__.encoding p File.dirname(File.expand_path(__FILE__)) p File.dirname(File.expand_path(__FILE__)).encoding print "__dir__ == File.dirname(File.expand_path(__FILE__)): " p __dir__ == File.dirname(File.expand_path(__FILE__)) begin require_relative "lib/foo" puts "foo.rb loaded" rescue LoadError puts "LoadError: #{$!}" end ``` Results: ``` "C:/?e?X?g_test" #<Encoding:Windows-31J> "C:/\x{8365}\x{8358}\x{8367}_test" #<Encoding:Windows-31J> __dir__ == File.dirname(File.expand_path(__FILE__)): false LoadError: cannot load such file -- C:/?e?X?g_test/lib/foo ``` However, in ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x64-mingw-ucrt], the same code produces the following expected results: ``` "C:/\x{8365}\x{8358}\x{8367}_test" #<Encoding:Windows-31J> "C:/\x{8365}\x{8358}\x{8367}_test" #<Encoding:Windows-31J> __dir__ == File.dirname(File.expand_path(__FILE__)): true foo.rb loaded ``` Is this behavior expected, or could this be a bug in the Ruby interpreter on Windows? If this is my misunderstanding or a known issue, I apologize. Thank you for your consideration. Environment Information: - OS: Windows 11 Pro 23H2 (OS build 22631.4037) - Installed via: RubyInstaller - Shell: Command Prompt - File System: NTFS - System Locale: Japanese (Japan), Language: Japanese -- https://bugs.ruby-lang.org/

3 2

[ruby-core:119380] [Ruby master Misc#20774] Remove remaining locale dependent code from Windows port
by larskanis (Lars Kanis) 18 Dec '24

18 Dec '24

Issue #20774 has been reported by larskanis (Lars Kanis). ---------------------------------------- Misc #20774: Remove remaining locale dependent code from Windows port https://bugs.ruby-lang.org/issues/20774 * Author: larskanis (Lars Kanis) * Status: Open ---------------------------------------- The external_encoding of files, file names and ENV on Windows were changed from locale codepage to UTF-8 in ruby-3.0. But there are still several remaining points where locale encoding is used although there is no need to do so. The Windows port is already fully UTF-16/UTF-8 based and locale encoding is only used for historical and not for technical reasons. My proposal is to remove (most of) the locale dependent conversions from the ruby code for Windows. Before I open pull requests in this regard, I would like to confirm this direction with the ruby core team. Let me show what I mean: ``` # täst-locale-enc.rb def pr(*strs) strs.each do |str| p [str, IO===str ? str.external_encoding&.name : str.encoding.name] end end if $0==__FILE__ pr STDIN # => [#<IO:<STDIN>>, "CP850"] pr $0 # => ["ruby/t\x84st-locale-enc.rb", "CP850"] pr __FILE__ # => ["ruby/t\x84st-locale-enc.rb", "CP850"] pr __dir__ # => ["C:/Users/kanis/ruby", "CP850"] pr 'ä' # => ["ä", "UTF-8"] pr '€' # => ["€", "UTF-8"] pr $:.first # => ["C:/Users/kanis/t\xE2\x82\xACst", "ASCII-8BIT"] pr $:.last # => ["C:/Ruby33-x64/lib/ruby/3.3.0/x64-mingw-ucrt", "CP850"] require "win32/registry" pr Win32::Registry::HKEY_CURRENT_USER.open("Environment")['TMP'] # => ["C:\\Users\\kanis\\AppData\\Local\\Temp", "UTF-8"] pr Win32::Registry::HKEY_CURRENT_USER.open("\\").each_key{ break _1 } # => ["AppEvents", "CP850"] end # execute with: ruby -It€st ruby\täst-locale-enc.rb ``` I wrote the results on `ruby-3.3 x64-mingw-ucrt` right into the code. The situation is even worse when called with `-e` script: ``` $ ruby -It€st -r .\ruby\täst-locale-enc.rb -e "pr STDIN, $0, __FILE__, __dir__, 'ä', '€', $:.first, $:.last" [#<IO:<STDIN>>, "CP850"] ["-e", "CP850"] ["-e", "UTF-8"] [".", "US-ASCII"] ["\x84", "CP850"] ["?", "CP850"] ["C:/Users/kanis/t\xE2\x82\xACst", "ASCII-8BIT"] ["C:/Ruby33-x64/lib/ruby/3.3.0/x64-mingw-ucrt", "CP850"] ``` There are also some inconsistencies like that it's possible to `require` script names with characters outside of the codepage, but it fails to execute a script directly or by using `require_relative` : ``` $ ruby -r .\t€st-locale-enc.rb -e "pr STDIN" [#<IO:<STDIN>>, "CP850"] $ ruby .\t€st-locale-enc.rb ruby: Invalid argument -- ./t?st-locale-enc.rb (LoadError) ``` Maybe there are more places which are working with locale codepage - these are only the few that I remember from memory. I would like to change all the above results to be UTF-8 encoded, like it is the case on Ubuntu. Compatibility ------------- Changing the encoding of returned strings is of course an API change. IMHO it is still something we should change in a minor release of ruby. The reason is that I don't remember about only a single issue cased by the change to UTF-8 in ruby-3.0 in the company I work for. To the contrary many issues are caused by using locale codepage where some non-ASCII characters work and other characters don't. Most issue with ruby-3.0 were cased by the keyword argument changes. -- https://bugs.ruby-lang.org/

2 4

[ruby-core:120280] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by deivid 17 Dec '24

17 Dec '24

Issue #6648 has been updated by deivid (David Rodríguez). > But anyway, I was talking specifically about your Bundler example; I didn't see the point of -C in that specific example. Bundler executes right from the start; there's no opportunity for "user code" to change to a different directory, is there? The entrypoint to Bundler is not only the `bundle` CLI, but also a `require` of `bundler/setup`, for example. And sometimes stuff happens before that. But you're right that my example was not directly related to `-C`. Overall I don't think whether `-C` is included or not will affect Bundler, because of how little usage `-C` seems to have but it still feels better to me to include it since after all, it's part of the original argv. If `Process.argv` does not need it or wants to ignore it, that can be done manually. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-111046 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120277] [Ruby master Bug#20958] fix ENV.keys encoding on windows
by YO4 (Yoshinao Muramatsu) 17 Dec '24

17 Dec '24

Issue #20958 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Bug #20958: fix ENV.keys encoding on windows https://bugs.ruby-lang.org/issues/20958 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- [github PR#12368](https://github.com/ruby/ruby/pull/12368) ENV.keys elements contain UTF-8 byte sequences with locale encoding on Windows. ``` > ruby -v -e "p ENV.keys.last; p ENV.keys.last.encoding; p ENV.keys.last.dup.force_encoding('utf-8')" ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x64-mingw-ucrt] "\x{E383}\x{86E3}\x{82B9}\x{E383}\x88" #<Encoding:Windows-31J> "テスト" ``` This patch changes the encoding of ENV.keys to utf-8 on Windows. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120272] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by headius (Charles Nutter) 16 Dec '24

16 Dec '24

Issue #6648 has been updated by headius (Charles Nutter). I am in favor of including -C in the argv API along with every other flag that was passed at the command line. The purpose of getting the list of arguments is not solely for relaunching, it is also to be able to reprocess that list and act on it within the same process. The relaunching api, which I agree would make sense as a separate API call, would have more smarts for launching a new process with equivalent flags to this process. That may mean that global state change flags like -C must be handled differently. Someone here also mentioned having access to the original state in which the runtime was launched, such as the original working directory before -C or chdir calls have happened. Original environment probably should be included as well. My goal with these APIs is to make it transparent to Ruby code exactly how the current instance of Ruby was launched, along with providing a standard way to relaunch the same runtime in the same way. The argv API should come first. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-111039 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120253] [Ruby master Bug#20956] Nested alternation pattern matching bug in 3.4.0-rc1
by nevans (Nicholas Evans) 16 Dec '24

16 Dec '24

Issue #20956 has been reported by nevans (Nicholas Evans). ---------------------------------------- Bug #20956: Nested alternation pattern matching bug in 3.4.0-rc1 https://bugs.ruby-lang.org/issues/20956 * Author: nevans (Nicholas Evans) * Status: Open * ruby -v: 3.4.0-rc1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I think I've found a pattern matching bug in ruby 3.4.0-rc1. ```ruby [123, ["foo"]] in [_, /\Afoo\b/i | [/\Afoo\z/i, *]] ``` I also tested this in ruby 2.7.8, with: ```ruby case [123, ["foo"]]; in [_, /\Afoo\b/i | [/\Afoo\z/i, *]] then true; else false end ``` These return `true` in ruby 2.7.8, 3.0.7, 3.1.6, 3.2.6, and 3.3.6. But they return false in 3.4.0-rc1. I'm not sure if this is the best example, but it is a simplification of how I came across the bug. Here are some other similar cases, which give the same result in 3.4.0-rc1 and 3.3.6: ```irb irb(main):001> ["foo"] in /\Afoo\b/i | [/\Afoo\z/i, *] => true irb(main):002> [123, ["foo"]] in [_, [/\Afoo\z/i, *]] => true irb(main):003> [["foo"]] in [[/\Afoo\z/i, *]] => true ``` But this case exhibits the bug again: ```irb irb(main):004> [["foo"]] in [/\Afoo\b/i | [/\Afoo\z/i, *]] => false ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:120270] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Dan0042 (Daniel DeLorme) 16 Dec '24

16 Dec '24

Issue #6648 has been updated by Dan0042 (Daniel DeLorme). deivid (David Rodríguez) wrote in #note-50: > What if "user code" changed to a different directory before we restart, then `-C` will have a different effect if it's relative. That's why I don't want `-C` in ruby_args at all. What if user code changed to a different directory **and intended** for that to be the CWD upon re-exec? Then `-C` would break that intent. The way I see it, the CWD is part of the environment; in the same way that a re-exec inherits any changes to the ENV variables, a re-exec should inherit any changes to the CWD. But anyway, I was talking specifically about your Bundler example; I didn't see the point of `-C` in that specific example. Bundler executes right from the start; there's no opportunity for "user code" to change to a different directory, is there? ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-111037 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120269] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by deivid 16 Dec '24

16 Dec '24

Issue #6648 has been updated by deivid (David Rodríguez). But to be honest, I get your point about hearing from realworld use cases of `-C`. I've never seen it used. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-111036 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120268] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by deivid 16 Dec '24

16 Dec '24

Issue #6648 has been updated by deivid (David Rodríguez). > I don't follow the logic here. You would re-run the original command in the original directory, just so that it can change back to the directory specified by -C. What for? Why not just stay in the CWD if you're going to wind up back there anyway? What's the point of using Process.original_working_directory and -C to cancel each other? Well, that's the only safe way, right? What if "user code" changed to a different directory before we restart, then `-C` will have a different effect if it's relative. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-111035 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0