February 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:120951] [Ruby master Feature#15663] Documenting autoload semantics
by fxn (Xavier Noria) 12 Feb '25

12 Feb '25

Issue #15663 has been updated by fxn (Xavier Noria). A classic way to exploit that behavior I've seen out there (in the past): ```ruby module MyGem autoload :Date, 'date' end ``` That meant, make `Date` available in case client code uses my gem in a way that hits an internal reference to `Date`. But do it with `autoload` so that it is not eager loaded if not needed. ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111846 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec… of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_r…. One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements…. Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `<main>' 1: from main.rb:17:in `block in <main>' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * #<Thread:0x000055929cc5b348 sleep_forever> rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in <main>' main.rb:20:in `<main>' * #<Thread:0x000055929ce2b380@main.rb:17 sleep_forever> rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in <main>' foo.rb:1:in `<top (required)>' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in <main>' * #<Thread:0x000055929ce29c38@main.rb:17 sleep_forever> rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in <main>' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021…, or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120950] [Ruby master Feature#15663] Documenting autoload semantics
by fxn (Xavier Noria) 12 Feb '25

12 Feb '25

Issue #15663 has been updated by fxn (Xavier Noria). Hi @eregon! > From that POV any autoload which when triggered does not define the constant should be an error, specifically a LoadError Not quite! Autoload is just a trigger for `Kernel#require`, there is no expectation on its side-effects. Once triggered, constant lookup continues and it is constant lookup what drives possible errors. For example, consider ``` ruby # x.rb M::X = 1 # m.rb module M module N autoload :X, 'x' p X # prints 1! end end ``` That works, because `autoload` does not care about what happened. It triggered `Kernel#require`, which loaded the file, control back. `N::X` does not exist, up the nesting `M::X` exist, so the constant is resolved. (Zeitwerk does require the constant to be defined in the receiver, but that is a contract enforced by the library to match file name conventions and have more strict rules.) ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111845 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec… of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_r…. One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements…. Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `<main>' 1: from main.rb:17:in `block in <main>' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * #<Thread:0x000055929cc5b348 sleep_forever> rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in <main>' main.rb:20:in `<main>' * #<Thread:0x000055929ce2b380@main.rb:17 sleep_forever> rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in <main>' foo.rb:1:in `<top (required)>' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in <main>' * #<Thread:0x000055929ce29c38@main.rb:17 sleep_forever> rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in <main>' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021…, or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120948] [Ruby master Feature#15663] Documenting autoload semantics
by Eregon (Benoit Daloze) 12 Feb '25

12 Feb '25

Issue #15663 has been updated by Eregon (Benoit Daloze). @fxn Regarding the comment above, I think always treating autoload constants as if they already existed seems one way to be more consistent. From that POV any autoload which when triggered does not define the constant should be an error, specifically a LoadError or so, which the program shouldn't catch and so should be considered a "fatal" error of that program. Similar to e.g. a constant is defined but some bad code does `remove_const` of it later, those kind of broken code edge cases should be considered bugs of the program, and in fact anyone writing Ruby code relies on gems not doing such nasty things. ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111844 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec… of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_r…. One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements…. Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `<main>' 1: from main.rb:17:in `block in <main>' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * #<Thread:0x000055929cc5b348 sleep_forever> rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in <main>' main.rb:20:in `<main>' * #<Thread:0x000055929ce2b380@main.rb:17 sleep_forever> rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in <main>' foo.rb:1:in `<top (required)>' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in <main>' * #<Thread:0x000055929ce29c38@main.rb:17 sleep_forever> rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in <main>' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021…, or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120940] [Ruby master Bug#21128] dir.c defines O_CLOEXEC before it is defined by the system.
by collinfunk (Collin Funk) 12 Feb '25

12 Feb '25

Issue #21128 has been reported by collinfunk (Collin Funk). ---------------------------------------- Bug #21128: dir.c defines O_CLOEXEC before it is defined by the system. https://bugs.ruby-lang.org/issues/21128 * Author: collinfunk (Collin Funk) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Building from master commit hash bfcf36fe2243d34a01b5707754be9164849150c9 on GNU/Linux, I see: ``` compiling dir.c In file included from /usr/include/bits/fcntl.h:61, from /usr/include/fcntl.h:35, from dir.c:39: /usr/include/bits/fcntl-linux.h:144:10: warning: "O_CLOEXEC" redefined 144 | # define O_CLOEXEC __O_CLOEXEC /* Set close_on_exec. */ | ^~~~~~~~~ dir.c:26:11: note: this is the location of the previous definition 26 | # define O_CLOEXEC 0 | ^~~~~~~~~ ``` This is because the file includes unistd.h before falling back to 0 for O_CLOEXEC. But glibc requires inclusion of fcntl.h for O_CLOEXEC. Since that file is included after, it is redefined. The C standard leaves the behavior of redefining a variable undefined and it is a simple fix, so I will create a pull request. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:120939] [Ruby master Bug#18605] Fails to run on (newer) 32bit Windows with ucrt
by luke-gru (Luke Gruber) 11 Feb '25

11 Feb '25

Issue #18605 has been updated by luke-gru (Luke Gruber). I'm trying to get ARM64 build of Windows to work so I can debug an issue I was having with a branch I'm working on that's failing in CI with Visual Studio 2022. It would be great if ARM64 Windows was working. Any news on this? ---------------------------------------- Bug #18605: Fails to run on (newer) 32bit Windows with ucrt https://bugs.ruby-lang.org/issues/18605#change-111830 * Author: lazka (Christoph Reiter) * Status: Open * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- 32bit ruby using ucrt has started to fail on newer Windows with "unexpected ucrtbase.dll" -> https://github.com/ruby/ruby/blob/3fb7d2cadc18472ec107b14234933b017a33c14d/… The problem is that ruby depends on ucrt internals and those have apparently changed with newer versions. See https://github.com/msys2/MINGW-packages/pull/10878 and https://github.com/msys2/MINGW-packages/issues/10896 for some background and a potential fix. But ideally ruby wouldn't depend on Windows internals like this. thanks! -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120938] [Ruby master Bug#16145] regexp match error if mixing /i, character classes, and utf8
by mjrzasa 11 Feb '25

11 Feb '25

Issue #16145 has been updated by mjrzasa (Maciek Rząsa). I've tested it for Polish letters, the bug appears only for `ó`, all other work OK: ``` pry(main)> ['ą', 'ę', 'ó', 'ś', 'ł', 'ć', 'ź', 'ż', 'ń'].map { [_1, _1.bytes, /[x#{_1}]/i.match?("qwer#{_1.capitalize}")] } => [["ą", [196, 133], true], ["ę", [196, 153], true], ["ó", [195, 179], false], ["ś", [197, 155], true], ["ł", [197, 130], true], ["ć", [196, 135], true], ["ź", [197, 186], true], ["ż", [197, 188], true], ["ń", [197, 132], true]] ``` `ó`, like `é` starts with a byte of `195` ``` pry(main)> 'é'.bytes => [195, 169] ``` ---------------------------------------- Bug #16145: regexp match error if mixing /i, character classes, and utf8 https://bugs.ruby-lang.org/issues/16145#change-111829 * Author: zenspider (Ryan Davis) * Status: Open * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- (reported on behalf of mage(a)mage.gold -- there appears to be an error in registration or login): See: ruby-talk @ X-Mail-Count: 440336 2.6.3 :049 > 'SHOP' =~ /[xo]/i => 2 2.6.3 :050 > 'CAFÉ' =~ /[é]/i => 3 2.6.3 :051 > 'CAFÉ' =~ /[xé]/i => nil 2.6.3 :052 > 'CAFÉ' =~ /[xÉ]/i => 3 Expected result: 2.6.3 :051 > 'CAFÉ' =~ /[xé]/i => 3 I tested it on random regex online pages. It does not match on https://regex101.com/ It matches on: https://regexr.com/ https://www.regextester.com/ https://www.freeformatter.com/regex-tester.html (Ignore case turned on). The reason I suppose it’s more like a bug than a feature is the fact that /[é]/i matches 'CAFÉ'. If the //i didn’t work for UTF-8 characters then the /[é]/i wouldn’t match it either. For example, [é] does not match 'CAFÉ' on https://regex101.com/ I could not find a page or a system that behaves the same way as Ruby does. For example, it matches in PostgreSQL 10 (under FreeBSD 12) too: # select 'CAFÉ'~ '[xé]'; ?column? ---------- f (1 row) # select 'CAFÉ' ~* '[xé]'; ?column? ---------- t (1 row) Tested it in IRB on macOS and FreeBSD. $ uname -a && ruby -v && locale Darwin xxx 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64 ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin18] LANG="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_ALL="en_US.UTF-8" $ uname -a && ruby -v && locale FreeBSD xxx 12.0-RELEASE-p9 FreeBSD 12.0-RELEASE-p9 GENERIC amd64 ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-freebsd12.0] LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_ALL=en_US.UTF-8 I installed Ruby with RVM. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120914] [Ruby master Bug#21124] Enumerable#find called without block returns Enumerator without size
by andrykonchin (Andrew Konchin) 11 Feb '25

11 Feb '25

Issue #21124 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #21124: Enumerable#find called without block returns Enumerator without size https://bugs.ruby-lang.org/issues/21124 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.4.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- When a collection's size is known then enumerator that iterates it usually has size (that's `#size` returns a collection size). But an enumerator returned by `Enumerable#find` called without a block doesn't have size: ```ruby [1, 2, 3].find.size # => nil ``` Expected result here is `3`. Usually similar methods in the Enumerable module return enumerators with size (when these enumerators for instance iterate the collection itself): ```ruby [1, 2, 3].each_entry.size # => 3 ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:120922] [Ruby master Feature#21126] Drop default_proc when Hash#freeze is called for better Ractor support
by osyoyu (Daisuke Aritomo) 10 Feb '25

10 Feb '25

Issue #21126 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Feature #21126: Drop default_proc when Hash#freeze is called for better Ractor support https://bugs.ruby-lang.org/issues/21126 * Author: osyoyu (Daisuke Aritomo) * Status: Open ---------------------------------------- Hash instances with default_proc set cannot be sent/moved across Ractors, even if they are frozen. Consider the following code. Using a default proc to set an empty Array is a very common pattern, even introduced in the docs. ```ruby h = Hash.new {|h, k| h[k] = [] } h[:foo] << 1 h.freeze Ractor.new(h) {|h| p h }.take # <internal:ractor>:282:in 'Ractor.new': allocator undefined for Proc (TypeError) ``` https://docs.ruby-lang.org/en/3.4/Hash.html#class-Hash-label-Default+Proc One must explicitly call `h.default_proc = nil` before sending the hash to another Ractor. This isn't the most friendly way for the programmer since (1) it is not easy to spot that the default hash is rendering the Hash unsendable, and (2) `Hash#default_proc=` isn't a widely known API anyway (at least from my perspective). ## Proposal Automatically drop the `default` `default_proc` when Hash#freeze is called. They have little use after the Hash gets frozen. Nevertheless, it should be pointed out that this is an incompatibility for `Hash#default` `Hash#default_proc` -- they currently return the original value, but they will return `nil`. Patch: https://github.com/ruby/ruby/pull/12717 Dropping the default_proc on `Hash#freeze` will also be nicer for `Ractor.make_shareable` users, since it does not require users to find the particular Hash with the default_proc buried somewhere. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:120930] [Ruby master Bug#21127] Thread deadlock does not display backtraces
by pocke (Masataka Kuwabara) 10 Feb '25

10 Feb '25

Issue #21127 has been reported by pocke (Masataka Kuwabara). ---------------------------------------- Bug #21127: Thread deadlock does not display backtraces https://bugs.ruby-lang.org/issues/21127 * Author: pocke (Masataka Kuwabara) * Status: Open * ruby -v: ruby 3.5.0dev (2025-02-10T00:27:18Z master 135479a58d) +PRISM [arm64-darwin24] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Previously, Ruby displayed backtraces for each thread on deadlock. However, it has not been shown since Ruby 3.0. It should display the backtrace for debugging. (related: https://bugs.ruby-lang.org/issues/8214) ## Step to reproduce ```ruby def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join ``` Ruby 2.7.8: It displays the trace. ``` ruby-2.7.8 ruby 2.7.8p225 (2023-03-30 revision 1f4d455848) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000004000653e80 main thread:0x000000400035da50 * #<Thread:0x000000400038ad70 sleep_forever> rb_thread_t:0x000000400035da50 native:0x0000004002fb7740 int:0 -e:9:in `join' -e:9:in `<main>' * #<Thread:0x00000040004fe760 -e:6 sleep_forever> rb_thread_t:0x00000040004518a0 native:0x00000040067ae700 int:0 depended by: tb_thread_id:0x000000400035da50 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:6:in `block in <main>' * #<Thread:0x00000040004fe620 -e:7 sleep_forever> rb_thread_t:0x0000004000653e80 native:0x000000400e9af700 int:0 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:7:in `block in <main>' from -e:9:in `<main>' ``` Ruby 3.0.0: There is no trace. ``` ruby-3.0.0 ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00000040006e88c0 main thread:0x00000040003ffa10 * #<Thread:0x000000400046f448 sleep_forever> rb_thread_t:0x00000040003ffa10 native:0x0000004003059740 int:0 * #<Thread:0x000000400073c2e8 -e:6 sleep_forever> rb_thread_t:0x00000040006e15a0 native:0x0000004006850700 int:0 depended by: tb_thread_id:0x00000040003ffa10 * #<Thread:0x000000400073c1f8 -e:7 sleep_forever> rb_thread_t:0x00000040006e88c0 native:0x000000400ea51700 int:0 from -e:9:in `<main>' ``` Ruby 3.5.0.dev (latest): There is no trace. ``` ruby 3.5.0dev (2025-02-10T00:27:18Z master 135479a58d) +PRISM [arm64-darwin24] -e:9:in 'Thread#join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x000000012ae3b530 main thread:0x000000012af04080 * #<Thread:0x0000000104aaaf78 sleep_forever> rb_thread_t:0x000000012af04080 native:0x00000001e9d93240 int:0 * #<Thread:0x0000000104e5e048 -e:6 sleep_forever> rb_thread_t:0x000000012ae23a90 native:0x000000016b5f3000 int:0 depended by: tb_thread_id:0x000000012af04080 * #<Thread:0x0000000104e5deb8 -e:7 sleep_forever> rb_thread_t:0x000000012ae3b530 native:0x000000016b6ff000 int:0 from -e:9:in '<main>' ``` Note: The results of 2.7 and 3.0 are indented because I executed them with all-ruby. ## Cause This commit has introduced this change. https://github.com/ruby/ruby/commit/3b24b7914c16930bfadc89d6aff6326a51c54295 The ticket is https://bugs.ruby-lang.org/issues/17031 It changes `rb_ec_backtrace_str_ary` function in `vm_backtrace.c`. Previously, `rb_ec_backtrace_str_ary(ec, 0, 0)` returns all traces, but it returns an empty array with this change. It should pass `ALL_BACKTRACE_LINES` (`-1`) to get all traces. I used rubyfarm also to confirm that this commit was the cause. With the commit, it does not display the traces. ``` docker run --rm -ti rubylang/rubyfarm:3b24b7914c16930bfadc89d6aff6326a51c54295 ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 2.8.0dev (2020-08-27T22:17:36Z :detached: 3b24b7914c) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000004000566a80 main thread:0x0000004000202a60 * #<Thread:0x000000400026faf8 sleep_forever> rb_thread_t:0x0000004000202a60 native:0x000000400440d740 int:0 * #<Thread:0x00000040004f1768 -e:6 sleep_forever> rb_thread_t:0x0000004000566710 native:0x00000040081fe700 int:0 depended by: tb_thread_id:0x0000004000202a60 * #<Thread:0x00000040004f1600 -e:7 sleep_forever> rb_thread_t:0x0000004000566a80 native:0x00000040103ff700 int:0 from -e:9:in `<main>' ``` With the parent commit, it displays the traces. ``` docker run --rm -ti rubylang/rubyfarm:8095114f1715070fcdc2b29303dcf55a7fcc32a3 ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 2.8.0dev (2020-08-27T19:54:36Z :detached: 8095114f17) [x86_64-linux] -e:9:in `join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x000000400055b3a0 main thread:0x0000004000202a60 * #<Thread:0x000000400026faf8 sleep_forever> rb_thread_t:0x0000004000202a60 native:0x000000400440d740 int:0 -e:9:in `join' -e:9:in `<main>' * #<Thread:0x00000040004f1768 -e:6 sleep_forever> rb_thread_t:0x000000400055afe0 native:0x00000040081fe700 int:0 depended by: tb_thread_id:0x0000004000202a60 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:6:in `block in <main>' * #<Thread:0x00000040004f1600 -e:7 sleep_forever> rb_thread_t:0x000000400055b3a0 native:0x00000040103ff700 int:0 -e:3:in `pop' -e:3:in `h' -e:2:in `g' -e:1:in `f' -e:7:in `block in <main>' from -e:9:in `<main>' ``` I think it's not an intentional change. The ticket and NEWS do not mention the backtrace on a deadlock. (And the backtrace is really helpful!) ## Solution The following patch fixes the problem. I'll make a PR on GitHub soon. ```diff diff --git a/eval_intern.h b/eval_intern.h index 9a551120ff..f783c93aff 100644 --- a/eval_intern.h +++ b/eval_intern.h @@ -300,6 +300,8 @@ void rb_vm_set_progname(VALUE filename); VALUE rb_vm_cbase(void); /* vm_backtrace.c */ +#define BACKTRACE_START 0 +#define ALL_BACKTRACE_LINES -1 VALUE rb_ec_backtrace_object(const rb_execution_context_t *ec); VALUE rb_ec_backtrace_str_ary(const rb_execution_context_t *ec, long lev, long n); VALUE rb_ec_backtrace_location_ary(const rb_execution_context_t *ec, long lev, long n, bool skip_internal); diff --git a/thread.c b/thread.c index d40d5140b7..5280c667af 100644 --- a/thread.c +++ b/thread.c @@ -5596,7 +5596,7 @@ debug_deadlock_check(rb_ractor_t *r, VALUE msg) } } rb_str_catf(msg, "\n "); - rb_str_concat(msg, rb_ary_join(rb_ec_backtrace_str_ary(th->ec, 0, 0), sep)); + rb_str_concat(msg, rb_ary_join(rb_ec_backtrace_str_ary(th->ec, BACKTRACE_START, ALL_BACKTRACE_LINES), sep)); rb_str_catf(msg, "\n"); } } diff --git a/vm_backtrace.c b/vm_backtrace.c index 56a0d861d6..6f2cf9e169 100644 --- a/vm_backtrace.c +++ b/vm_backtrace.c @@ -31,9 +31,6 @@ id2str(ID id) } #define rb_id2str(id) id2str(id) -#define BACKTRACE_START 0 -#define ALL_BACKTRACE_LINES -1 - inline static int calc_pos(const rb_iseq_t *iseq, const VALUE *pc, int *lineno, int *node_id) { ``` The result is here: ``` $ ruby -ve 'def f(q) g(q) end def g(q) h(q) end def h(q) q.pop end q = Thread::Queue.new th1 = Thread.new { f q } th2 = Thread.new { f q } th1.join th2.join' ruby 3.5.0dev (2025-02-10T00:27:18Z origin/master 135479a58d) +PRISM [arm64-darwin24] -e:9:in 'Thread#join': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x0000000157a053a0 main thread:0x0000000156704080 * #<Thread:0x000000011bcfaf78 sleep_forever> rb_thread_t:0x0000000156704080 native:0x00000001e9d93240 int:0 -e:9:in 'Thread#join' -e:9:in '<main>' * #<Thread:0x000000011d2d0bb8 -e:6 sleep_forever> rb_thread_t:0x0000000157a04d40 native:0x000000016dcb7000 int:0 depended by: tb_thread_id:0x0000000156704080 <internal:thread_sync>:18:in 'Thread::Queue#pop' -e:3:in 'Object#h' -e:2:in 'Object#g' -e:1:in 'Object#f' -e:6:in 'block in <main>' * #<Thread:0x000000011d2d07a8 -e:7 sleep_forever> rb_thread_t:0x0000000157a053a0 native:0x000000016ddc3000 int:0 <internal:thread_sync>:18:in 'Thread::Queue#pop' -e:3:in 'Object#h' -e:2:in 'Object#g' -e:1:in 'Object#f' -e:7:in 'block in <main>' from -e:9:in '<main>' ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120908] [Ruby master Bug#21122] library/socket/basicsocket/recv_nonblock_spec.rb fails with IO::EAGAINWaitReadable sometimes.
by hsbt (Hiroshi SHIBATA) 09 Feb '25

09 Feb '25

Issue #21122 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #21122: library/socket/basicsocket/recv_nonblock_spec.rb fails with IO::EAGAINWaitReadable sometimes. https://bugs.ruby-lang.org/issues/21122 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- After https://github.com/ruby/ruby/commit/d7a5ad2a21f7d2c45e3fea674ff077bb0e2cadae, I got test failure with `library/socket/basicsocket/recv_nonblock_spec.rb` sometimes. * https://github.com/ruby/ruby/actions/runs/13108171270/job/36566403804#step:… * https://github.com/ruby/ruby/actions/runs/13195573860/job/36836170533?pr=12… * my local macOS. @andrykonchin @Eregon Can you look this? -- https://bugs.ruby-lang.org/

2 2