- ruby-core - ml.ruby-lang.org

[ruby-core:120879] [Ruby master Bug#21113] net-smtp.gemspec is broken at Ruby 3.3/3.4
by hsbt (Hiroshi SHIBATA) 13 Feb '25

13 Feb '25

Issue #21113 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #21113: net-smtp.gemspec is broken at Ruby 3.3/3.4 https://bugs.ruby-lang.org/issues/21113 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- The bundled `net-smtp-0.4.0.1` at Ruby 3.3 and `net-smtp-0.5.0` at Ruby 3.4 are broken. ``` ❯ diff -u 3.2.7/lib/ruby/gems/3.2.0/specifications/net-smtp-0.3.4.gemspec 3.3.7/lib/ruby/gems/3.3.0/specifications/net-smtp-0.4.0.1.gemspec --- 3.2.7/lib/ruby/gems/3.2.0/specifications/net-smtp-0.3.4.gemspec 2025-02-05 14:31:27 +++ 3.3.7/lib/ruby/gems/3.3.0/specifications/net-smtp-0.4.0.1.gemspec 2025-02-05 14:33:37 @@ -1,26 +1,22 @@ # -*- encoding: utf-8 -*- -# stub: net-smtp 0.3.4 ruby lib +# stub: net-smtp 0.4.0.1 ruby lib Gem::Specification.new do |s| s.name = "net-smtp".freeze - s.version = "0.3.4" + s.version = "0.4.0.1".freeze s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version= s.metadata = { "homepage_uri" => "https://github.com/ruby/net-smtp", "source_code_uri" => "https://github.com/ruby/net-smtp" } if s.respond_to? :metadata= s.require_paths = ["lib".freeze] s.authors = ["Yukihiro Matsumoto".freeze] - s.date = "2025-02-04" + s.date = "2025-01-15" s.description = "Simple Mail Transfer Protocol client library for Ruby.".freeze s.email = ["matz(a)ruby-lang.org".freeze] s.homepage = "https://github.com/ruby/net-smtp".freeze s.licenses = ["Ruby".freeze, "BSD-2-Clause".freeze] s.required_ruby_version = Gem::Requirement.new(">= 2.6.0".freeze) - s.rubygems_version = "3.4.19".freeze + s.rubygems_version = "3.4.20".freeze s.summary = "Simple Mail Transfer Protocol client library for Ruby.".freeze - s.installed_by_version = "3.4.19" if s.respond_to? :installed_by_version - - s.specification_version = 4 - - s.add_runtime_dependency(%q<net-protocol>.freeze, [">= 0"]) + s.installed_by_version = "3.5.22".freeze end ``` ``` ❯ diff -u 3.2.7/lib/ruby/gems/3.2.0/specifications/net-smtp-0.3.4.gemspec 3.4.1/lib/ruby/gems/3.4.0/specifications/net-smtp-0.5.0.gemspec --- 3.2.7/lib/ruby/gems/3.2.0/specifications/net-smtp-0.3.4.gemspec 2025-02-05 14:31:27 +++ 3.4.1/lib/ruby/gems/3.4.0/specifications/net-smtp-0.5.0.gemspec 2025-02-05 14:36:00 @@ -1,26 +1,22 @@ # -*- encoding: utf-8 -*- -# stub: net-smtp 0.3.4 ruby lib +# stub: net-smtp 0.5.0 ruby lib Gem::Specification.new do |s| s.name = "net-smtp".freeze - s.version = "0.3.4" + s.version = "0.5.0".freeze s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version= s.metadata = { "homepage_uri" => "https://github.com/ruby/net-smtp", "source_code_uri" => "https://github.com/ruby/net-smtp" } if s.respond_to? :metadata= s.require_paths = ["lib".freeze] s.authors = ["Yukihiro Matsumoto".freeze] - s.date = "2025-02-04" + s.date = "2024-12-25" s.description = "Simple Mail Transfer Protocol client library for Ruby.".freeze s.email = ["matz(a)ruby-lang.org".freeze] s.homepage = "https://github.com/ruby/net-smtp".freeze s.licenses = ["Ruby".freeze, "BSD-2-Clause".freeze] s.required_ruby_version = Gem::Requirement.new(">= 2.6.0".freeze) - s.rubygems_version = "3.4.19".freeze + s.rubygems_version = "3.4.20".freeze s.summary = "Simple Mail Transfer Protocol client library for Ruby.".freeze - s.installed_by_version = "3.4.19" if s.respond_to? :installed_by_version - - s.specification_version = 4 - - s.add_runtime_dependency(%q<net-protocol>.freeze, [">= 0"]) + s.installed_by_version = "3.6.2".freeze end ``` The broken gemspecs lost `net-protocol` dependency at fresh installation from Ruby package. This is similar issue with #20581 The workaround is `gem install net-smtp` and overwrite `net-smtp` gemspec after Ruby installation. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:120537] [Ruby master Bug#21013] Backport request for YJIT fix for `yield` with keyword arguments
by alanwu (Alan Wu) 13 Feb '25

13 Feb '25

Issue #21013 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #21013: Backport request for YJIT fix for `yield` with keyword arguments https://bugs.ruby-lang.org/issues/21013 * Author: alanwu (Alan Wu) * Status: Closed * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONTNEED, 3.4: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/12525 The patch applies cleanly. Programs like the following used to crash when run with YJIT: ```ruby def a = yield(x: 0) 1000.times { a { |x:| x } } ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:120962] [Ruby master Bug#21132] Changed postposition `resque` and `if` behavior since Ruby 3.4
by watson1978 (Shizuo Fujita) 13 Feb '25

13 Feb '25

Issue #21132 has been reported by watson1978 (Shizuo Fujita). ---------------------------------------- Bug #21132: Changed postposition `resque` and `if` behavior since Ruby 3.4 https://bugs.ruby-lang.org/issues/21132 * Author: watson1978 (Shizuo Fujita) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- When I use postposition `resque` and `if` at same line, the behavior is changed since Ruby 3.4. ## Reproduction ```ruby def foo return "hogehoge" rescue nil if false return "fugafuga" end foo ``` ## Result with Ruby 3.3 ``` irb(main):001> RUBY_VERSION => "3.3.6" irb(main):002* def foo irb(main):003* return "hogehoge" rescue nil if false irb(main):004* return "fugafuga" irb(main):005> end => :foo irb(main):006> foo => "fugafuga" ``` YARV bytecode ``` ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [x86_64-linux] == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(6,3)> 0000 definemethod :foo, foo ( 1)[Li] 0003 putself ( 6)[Li] 0004 opt_send_without_block <calldata!mid:foo, argc:0, FCALL|VCALL|ARGS_SIMPLE> 0006 leave == disasm: #<ISeq:foo@<compiled>:1 (1,0)-(4,3)> 0000 putstring "fugafuga" ( 3)[LiCa] 0002 leave ( 4)[Re] ``` ## Result with Ruby 3.4 ``` irb(main):001> RUBY_VERSION => "3.4.1" irb(main):002* def foo irb(main):003* return "hogehoge" rescue nil if false irb(main):004* return "fugafuga" irb(main):005> end => :foo irb(main):006> foo => "hogehoge" ``` YARV bytecode ``` ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux] == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(1,54)> 0000 putself ( 1)[Li] 0001 opt_getconstant_path <ic:0 RubyVM::InstructionSequence> 0003 opt_getconstant_path <ic:1 DATA> 0005 opt_send_without_block <calldata!mid:new, argc:1, ARGS_SIMPLE> 0007 opt_send_without_block <calldata!mid:disassemble, argc:0, ARGS_SIMPLE> 0009 opt_send_without_block <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE> ``` -- https://bugs.ruby-lang.org/

2 3

[ruby-core:120047] [Ruby master Bug#20920] When loading a file, __FILE__ gets relative paths expanded only when they start with "./"
by deivid 13 Feb '25

13 Feb '25

Issue #20920 has been reported by deivid (David Rodríguez). ---------------------------------------- Bug #20920: When loading a file, __FILE__ gets relative paths expanded only when they start with "./" https://bugs.ruby-lang.org/issues/20920 * Author: deivid (David Rodríguez) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- $ cat foo.rb puts __FILE__ $ ruby foo.rb foo.rb $ ruby ./foo.rb ./foo.rb $ ruby -e 'load "foo.rb"' foo.rb $ ruby -e 'load "./foo.rb"' /full/path/to/foo.rb ``` More than an issue, this is mainly a question. In principle, it seems more consistent to me to either expand or not expand, but this is not causing real issues for me. I just want to figure out what to do with one pending spec in Bundler (https://github.com/rubygems/rubygems/blob/cd65092deb3168759820c613ecbc54cd9…) -- https://bugs.ruby-lang.org/

5 5

[ruby-core:120897] [Ruby master Bug#21119] Programs containing `Dir.glob` with a thread executing a CPU-heavy task run very slowly.
by genya0407 (Yusuke Sangenya) 13 Feb '25

13 Feb '25

Issue #21119 has been reported by genya0407 (Yusuke Sangenya). ---------------------------------------- Bug #21119: Programs containing `Dir.glob` with a thread executing a CPU-heavy task run very slowly. https://bugs.ruby-lang.org/issues/21119 * Author: genya0407 (Yusuke Sangenya) * Status: Open * ruby -v: ruby 3.5.0dev (2025-02-06T14:10:34Z master adbf9c5b36) +PRISM [arm64-darwin24] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- Executing the following code in Ruby 3.4.1 takes a very long time, especially when there are many files $100~$ in the current directory. This delay does not occur in Ruby 3.3.6. ## Reproducible script ```ruby # hoge.rb # Launch a thread to execute CPU-heavy task Thread.new do loop do arr = [] 100.times do arr << rand(1...100) end end end # Execute a program containing `Dir.glob` in the main thread. 10.times do Dir.glob('*') puts "aaaa" end ``` ## Execution Results Executiong the above code in Ruby 3.4.1 takes **119.43s**. ```shell $ ruby -v ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24] $ time ruby hoge.rb aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa ruby hoge.rb 119.43s user 0.30s system 99% cpu 1:59.89 total ``` Executing it in Ruby master also takes **118.87s**. ```shell $ ~/opt-ruby/bin/ruby -v ruby 3.5.0dev (2025-02-06T14:10:34Z master adbf9c5b36) +PRISM [arm64-darwin24] $ time ~/opt-ruby/bin/ruby hoge.rb aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa ~/opt-ruby/bin/ruby hoge.rb 118.87s user 0.46s system 99% cpu 2:00.45 total ``` Executing it in Ruby 3.3.6 takes only **2.22s**. ```shell $ ruby -v ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [arm64-darwin24] $ time ruby hoge.rb aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa ruby hoge.rb 2.22s user 0.03s system 98% cpu 2.286 total ``` So, there are roughly **50x** delays. ## Possible Cause From Ruby 3.4.0, `Dir.glob` releases the GVL frequently. * https://bugs.ruby-lang.org/issues/20587 * https://github.com/ruby/ruby/pull/11147 Due to this change, when a CPU-heavy thread releases the GVL, `Dir.glob` also releases the GVL immediately. As a result, `Dir.glob` gets significantly delayed because it has to continuously regain the GVL causing a major slowdown in execution. ## Note about Execution Results I measured the execution results under a stress condition, with 100 files in the current directory. If there are fewer files, the slowdown may be less pronounced. -- https://bugs.ruby-lang.org/

5 5

[ruby-core:120850] [Ruby master Bug#21102] Unexpected encoding when concatenating ASCII string with ASCII compatible string with non ASCII encoding
by toy (Ivan Kuchin) 13 Feb '25

13 Feb '25

Issue #21102 has been reported by toy (Ivan Kuchin). ---------------------------------------- Bug #21102: Unexpected encoding when concatenating ASCII string with ASCII compatible string with non ASCII encoding https://bugs.ruby-lang.org/issues/21102 * Author: toy (Ivan Kuchin) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- The problem was noticed in code that is boiled down to: ```ruby # encoding: UTF-8 str = "something" p str.encoding # => #<Encoding:UTF-8> p [nil, str].join.encoding # => #<Encoding:US-ASCII> ``` As `nil.to_s` is an empty string with encoding `ASCII` and `"something"` is `ASCII` compatible string, the result is a string with `ASCII` encoding. Event simpler example is `p (nil.to_s + "something").encoding`. Confusing is that resulting encoding depends on order and on compatibility of string encodings: ```ruby # encoding: UTF-8 str1 = "something" # ASCII compatible str2 = "söméthíng" # not ASCII compatible p (nil.to_s + str1).encoding # => #<Encoding:US-ASCII> p (nil.to_s + str2).encoding # => #<Encoding:UTF-8> p (str1 + nil.to_s).encoding # => #<Encoding:UTF-8> p (str2 + nil.to_s).encoding # => #<Encoding:UTF-8> ``` I would expect it to behave akin to summing integers and floats or rationals: ```ruby p 1 + 1.0 # => 2.0 p 1.0 + 1 # => 2.0 p 1 + 1r # => (2/1) p 1r + 1 # => (2/1) ``` So it is at least surprising to me. #18579 is probably the most related, but also #14975 and #20594 -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113033] [Ruby master Feature#19555] Allow passing default options to `Data.define`
by p8 (Petrik de Heus) 13 Feb '25

13 Feb '25

Issue #19555 has been reported by p8 (Petrik de Heus). ---------------------------------------- Feature #19555: Allow passing default options to `Data.define` https://bugs.ruby-lang.org/issues/19555 * Author: p8 (Petrik de Heus) * Status: Open * Priority: Normal ---------------------------------------- Defining a subclass of `Data` with default attributes can currently be implemented by overriding `intialize`: ```ruby class Point < Data.define(:x, :y, :z) def initialize(x:, y:, z: 0) = super end p Point.new(1, 2) #=> #<data Point x=1, y=2, z=0> ``` It would be nice if we could do it in `define` as well: ```ruby Point = Data.define(:x, :y, z: 0) ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:120747] [Ruby master Feature#21082] Alias it to its
by northeastprince (Matt Almeida) 13 Feb '25

13 Feb '25

Issue #21082 has been reported by northeastprince (Matt Almeida). ---------------------------------------- Feature #21082: Alias it to its https://bugs.ruby-lang.org/issues/21082 * Author: northeastprince (Matt Almeida) * Status: Open ---------------------------------------- One of Ruby's core tenants is beautiful code - and a key part of that is sweating names. Adding `it` as a pointer to the first block parameter was a huge step forward in that regard, but we're not done yet. If `[1, 2, 3].each { puts it }` works, `posts.each { puts its.title }` should work too. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:120958] [Ruby master Feature#15663] Documenting autoload semantics
by Eregon (Benoit Daloze) 12 Feb '25

12 Feb '25

Issue #15663 has been updated by Eregon (Benoit Daloze). Right, that's a good summary. It's natural for the constant lookup to use the constant scope as usual, and that part is fine. The part that's messy is when doing the `require` for an `autoload`, and it doesn't define the constant it "promised" to. In such a case I think an exception would be warranted, with of course a migration path of first warning such cases. We actually already have these warnings: ``` $ cat a.rb A = 42 $ ruby -w -I. -e 'autoload :A, "a"; p A' 42 $ ruby -w -I. -e 'autoload :B, "a"; p B' -e:1: warning: Expected a to define B but it didn't -e:1:in `<main>': uninitialized constant B (NameError) autoload :B, "a"; p B ^ $ ruby -w -I. -e 'module M; autoload :A, "a"; p A; end' -e:1: warning: Expected a to define M::A but it didn't 42 ``` So I guess I should make a ticket about turning those warnings into errors now :) At least it would be one complex aspect of autoload removed. ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111853 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec… of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_r…. One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements…. Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `<main>' 1: from main.rb:17:in `block in <main>' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * #<Thread:0x000055929cc5b348 sleep_forever> rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in <main>' main.rb:20:in `<main>' * #<Thread:0x000055929ce2b380@main.rb:17 sleep_forever> rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in <main>' foo.rb:1:in `<top (required)>' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in <main>' * #<Thread:0x000055929ce29c38@main.rb:17 sleep_forever> rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in <main>' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021…, or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:120955] [Ruby master Feature#15663] Documenting autoload semantics
by fxn (Xavier Noria) 12 Feb '25

12 Feb '25

Issue #15663 has been updated by fxn (Xavier Noria). I see what you mean, but let me rephrase it a bit. The autoload does not depend on the nesting (depending on your definition of "autoload"). It is triggered by a missing constant reference when the receiver is the cref, and the execution of the required file is not influence by the nesting in the caller (you of course know this, only describing). Success/failure of the `require` call does not depend on the nesting. Constant lookup does depend on the nesting, but constant lookup and autoload are independent of each other (except that one may trigger the other one). I agree with you in those rules, I would also prefer this to be simpler and more strict. Either you define the constant in the receiver or error. I'd go further for the sake of the exercise (but impossible to change). An autoload is a promise, but it may fail. In that sense, if this was day 0 and I was the author of the language, autoloads would not appear in the constants API. Only "actual" constants would. ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111850 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec… of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_r…. One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements…. Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `<main>' 1: from main.rb:17:in `block in <main>' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * #<Thread:0x000055929cc5b348 sleep_forever> rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in <main>' main.rb:20:in `<main>' * #<Thread:0x000055929ce2b380@main.rb:17 sleep_forever> rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in <main>' foo.rb:1:in `<top (required)>' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in <main>' * #<Thread:0x000055929ce29c38@main.rb:17 sleep_forever> rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in <main>' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021…, or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/

1 0