[ruby-core:123484] [Ruby Bug#21640] Core Pathname is missing 3 methods / is partially-defined
Issue #21640 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Bug #21640: Core Pathname is missing 3 methods / is partially-defined https://bugs.ruby-lang.org/issues/21640 * Author: Eregon (Benoit Daloze) * Status: Open * ruby -v: ruby 3.5.0dev (2025-10-09T08:06:20Z master a29c90c3b0) +PRISM [x86_64-linux] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- ``` $ ruby -e 'puts Pathname.instance_methods(false).sort; puts Pathname.singleton_methods.sort' > core_pathname_methods.txt $ ruby -rpathname -e 'puts Pathname.instance_methods(false).sort; puts Pathname.singleton_methods.sort' > require_pathname_methods.txt $ diff core_pathname_methods.txt require_pathname_methods.txt 36a37
find 72a74 rmtree 98a101 mktmpdir
So `#find`, `#rmtree` and `.mktmpdir` are missing from core Pathname.
And indeed, they give NoMethodError, e.g.
$ ruby -ve 'Pathname.new("doesnotexist").rmtree' ruby 3.5.0dev (2025-10-09T08:06:20Z master a29c90c3b0) +PRISM [x86_64-linux] -e:1:in '<main>': undefined method 'rmtree' for an instance of Pathname (NoMethodError) ``` I think this is confusing and unexpected for most users. Either Pathname should be fully defined or not at all, having a partially-defined Pathname seems particularly confusing (a bit like an autoload that failed in the middle or so). AFAIK the only core class which used to have this was Fiber with `alive?` and `transfer`, that certainly caused its share of confusion, and Fiber has since been fixed, now even without `require "fiber"` Fiber has all methods. Furthermore these 3 methods are not documented as needing `require 'pathname'` in their documentation: https://github.com/ruby/ruby/blob/master/lib/pathname.rb The reason these 3 methods are not in core pathname seems to be (from [this comment](https://github.com/ruby/pathname/issues/64#issuecomment-3388907046)):
That's consensus with akr and me. We should avoid loading other libraries by simply calling methods from the embedded core classes.
I understand that concern and I share it, but I think the situation of having a partially-defined Pathname in core is quite problematic. I think there are 2 solutions: * Define all Pathname methods in core. The `require` are done lazily inside the methods so AFAIK there is no technical issue blocking that, but it's not ideal design-wise that a core class can load default gems (i.e. give up that concern for this particular case). * Do not define Pathname in core and let it be a default gem as it was in Ruby 3.4. I discuss the second in more details, because I found more problems than I would have imagined with core Pathname: ## Should Pathname be in core? #17473 proposed to make Pathname core. That sounded great to me at first thought, but as I realize the practical problems with it I'm thinking it's actually better to keep Pathname non-core and as a default gems. Here are concerns with having Pathname in core: 1. Pathname inherently depends on other default gems/stdlib like `find`, `fileutils` and `tmpdir`. This is important to make Pathname useful. The docs even phrase it like this: `ri Pathname`
... All functionality from File, FileTest, and some from Dir and FileUtils is included, in an unsurprising way. It is essentially a facade for all of these, and more.
2. core Pathname is missing 3 methods compared to gem Pathname (shown above) 3. the pathname gem is likely to gain extra features, methods and bug fixes. For example I see [4 PRs](https://github.com/ruby/pathname/pulls) adding new methods. It means core Pathname will always be outdated and potentially missing some new methods. 4. One can still use the pathname gem even if it's in core, but surprisingly just `require 'pathname'` is not enough: ``` $ gem install pathname Building native extensions. This could take a while... Successfully installed pathname-0.4.0 1 gem installed $ ruby -e 'require "pathname"; puts $".grep(/pathname/)' pathname.so /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/pathname.rb ^ wrong that's the stdlib pathname $ ruby -e 'gem "pathname"; require "pathname"; puts $".grep(/pathname/)' # some warnings, reported in https://github.com/ruby/pathname/issues/66 pathname.so /home/eregon/prefix/ruby-master/lib/ruby/gems/3.5.0+4/gems/pathname-0.4.0/lib/pathname.rb ^ correct ``` This is another source of confusion caused by core pathname. It even happens with Bundler! ``` $ cat Gemfile source "https://rubygems.org" gem "pathname" $ bundle install $ ruby -e 'require "bundler/setup"; require "pathname"; puts $".grep(/pathname/); puts; puts $:' pathname.so /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/pathname.rb /home/eregon/tmp/bundler-pathname/vendor/bundle/ruby/3.5.0+4/gems/pathname-0.4.0/lib /home/eregon/tmp/bundler-pathname/vendor/bundle/ruby/3.5.0+4/extensions/x86_64-linux/3.5.0+4-static/pathname-0.4.0 /home/eregon/prefix/ruby-master/lib/ruby/gems/3.5.0+4/gems/bundler-2.8.0.dev/lib /home/eregon/prefix/ruby-master/lib/ruby/site_ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/site_ruby/3.5.0+4/x86_64-linux /home/eregon/prefix/ruby-master/lib/ruby/site_ruby /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby/3.5.0+4/x86_64-linux /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/x86_64-linux $ bundle exec ruby -e 'require "pathname"; puts $".grep(/pathname/); puts; puts $:' same as above ``` I'm not sure why, but that seems a serious bug that the `pathname` gem apparently can't be used at all with Bundler. 4. due to this difference in terms of methods for core & gem pathname it seems complicated to keep core Pathname and gem Pathname in sync, since e.g. everything is naturally in one `.rb` file in the gem, but in two `.rb` files in core 5. the pathname gem has to `remove_const :Pathname` to avoid conflicts and warnings in https://github.com/ruby/pathname/blob/4689b0b78d081ae855f325e086d95803fa5bd5.... This is bad for Ruby JITs, especially if core Pathname methods might have been used, as it will invalidate JITs compilations, invalidate some inline caches (also bad for the interpreter), caused the JIT to do more compilations which means slower warmup, makes it much harder to persist JITed code across process executions, etc. Based on all of these I think it would actually be better to not have Pathname in core, and let it be a default gem. Having to `require "pathname"` as one always needed to do so far seems far better than a partially-defined core Pathname with rough edge cases (those concerns). -- https://bugs.ruby-lang.org/
Issue #21640 has been updated by Eregon (Benoit Daloze). I'm happy to help making the changes in ruby/ruby to make Pathname "just a default gem" again like it was in 3.4, if that is the decided outcome of the dev meeting (I added this ticket to https://bugs.ruby-lang.org/issues/21606). ---------------------------------------- Bug #21640: Core Pathname is missing 3 methods / is partially-defined https://bugs.ruby-lang.org/issues/21640#change-114859 * Author: Eregon (Benoit Daloze) * Status: Open * ruby -v: ruby 3.5.0dev (2025-10-09T08:06:20Z master a29c90c3b0) +PRISM [x86_64-linux] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- ``` $ ruby -e 'puts Pathname.instance_methods(false).sort; puts Pathname.singleton_methods.sort' > core_pathname_methods.txt $ ruby -rpathname -e 'puts Pathname.instance_methods(false).sort; puts Pathname.singleton_methods.sort' > require_pathname_methods.txt $ diff core_pathname_methods.txt require_pathname_methods.txt 36a37
find 72a74 rmtree 98a101 mktmpdir
So `#find`, `#rmtree` and `.mktmpdir` are missing from core Pathname.
And indeed, they give NoMethodError, e.g.
$ ruby -ve 'Pathname.new("doesnotexist").rmtree' ruby 3.5.0dev (2025-10-09T08:06:20Z master a29c90c3b0) +PRISM [x86_64-linux] -e:1:in '<main>': undefined method 'rmtree' for an instance of Pathname (NoMethodError) ``` I think this is confusing and unexpected for most users. Either Pathname should be fully defined or not at all, having a partially-defined Pathname seems particularly confusing (a bit like a `require` that failed in the middle or so). AFAIK the only core class which used to have this was Fiber with `alive?` and `transfer`, that certainly caused its share of confusion, and Fiber has since been fixed, now even without `require "fiber"` Fiber has all methods. Furthermore these 3 methods are not documented as needing `require 'pathname'` in their documentation: https://github.com/ruby/ruby/blob/master/lib/pathname.rb The reason these 3 methods are not in core pathname seems to be (from [this comment](https://github.com/ruby/pathname/issues/64#issuecomment-3388907046)):
That's consensus with akr and me. We should avoid loading other libraries by simply calling methods from the embedded core classes.
I understand that concern and I share it, but I think the situation of having a partially-defined Pathname in core is quite problematic. I think there are 2 solutions: * Define all Pathname methods in core. The `require` are done lazily inside the methods so AFAIK there is no technical issue blocking that, but it's not ideal design-wise that a core class can load default gems (i.e. give up that concern for this particular case). * Do not define Pathname in core and let it be a default gem as it was in Ruby 3.4. I discuss the second in more details, because I found more problems than I would have imagined with core Pathname: ## Should Pathname be in core? #17473 proposed to make Pathname core. That sounded great to me at first thought, but as I realized the practical problems with it I'm thinking it's actually better to keep Pathname non-core and as a default gem. Here are 6 problems/concerns with having Pathname in core: 1. Pathname inherently depends on other default gems/stdlib like `find`, `fileutils` and `tmpdir`. This is important to make Pathname useful. The docs even phrase it like this: `ri Pathname`
... All functionality from File, FileTest, and some from Dir and FileUtils is included, in an unsurprising way. It is essentially a facade for all of these, and more.
2. core Pathname is missing 3 methods compared to gem Pathname (shown above) 3. the pathname gem is likely to gain extra features, methods and bug fixes. For example I see [4 PRs](https://github.com/ruby/pathname/pulls) adding new methods. It means core Pathname will always be outdated and potentially missing some new methods. 4. One can still use the pathname gem even if it's in core, but surprisingly just `require 'pathname'` is not enough: ``` $ gem install pathname Building native extensions. This could take a while... Successfully installed pathname-0.4.0 1 gem installed $ ruby -e 'require "pathname"; puts $".grep(/pathname/)' pathname.so /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/pathname.rb ^ wrong that's the stdlib pathname $ ruby -e 'gem "pathname"; require "pathname"; puts $".grep(/pathname/)' # some warnings, reported in https://github.com/ruby/pathname/issues/66 pathname.so /home/eregon/prefix/ruby-master/lib/ruby/gems/3.5.0+4/gems/pathname-0.4.0/lib/pathname.rb ^ correct ``` This is another source of confusion caused by core pathname. It even happens with Bundler! ``` $ cat Gemfile source "https://rubygems.org" gem "pathname" $ bundle install $ ruby -e 'require "bundler/setup"; require "pathname"; puts $".grep(/pathname/); puts; puts $:' pathname.so /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/pathname.rb /home/eregon/tmp/bundler-pathname/vendor/bundle/ruby/3.5.0+4/gems/pathname-0.4.0/lib /home/eregon/tmp/bundler-pathname/vendor/bundle/ruby/3.5.0+4/extensions/x86_64-linux/3.5.0+4-static/pathname-0.4.0 /home/eregon/prefix/ruby-master/lib/ruby/gems/3.5.0+4/gems/bundler-2.8.0.dev/lib /home/eregon/prefix/ruby-master/lib/ruby/site_ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/site_ruby/3.5.0+4/x86_64-linux /home/eregon/prefix/ruby-master/lib/ruby/site_ruby /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby/3.5.0+4/x86_64-linux /home/eregon/prefix/ruby-master/lib/ruby/vendor_ruby /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4 /home/eregon/prefix/ruby-master/lib/ruby/3.5.0+4/x86_64-linux $ bundle exec ruby -e 'require "pathname"; puts $".grep(/pathname/); puts; puts $:' same as above ``` I'm not sure why, but that seems a serious bug that the `pathname` gem apparently can't be used at all with Bundler. 5. due to this difference in terms of methods for core & gem pathname it seems complicated to keep core Pathname and gem Pathname in sync, since e.g. everything is naturally in one `.rb` file in the gem, but in two `.rb` files in core 6. the pathname gem has to `remove_const :Pathname` to avoid conflicts and warnings in https://github.com/ruby/pathname/blob/4689b0b78d081ae855f325e086d95803fa5bd5.... This is bad for Ruby JITs, especially if core Pathname methods might have been used, as it will invalidate JITs compilations, invalidate some inline caches (also bad for the interpreter), caused the JIT to do more compilations which means slower warmup, makes it much harder to persist JITed code across process executions, etc. Based on all of these I think it would actually be better to not have Pathname in core, and let it be a default gem (as it was in 3.4). Having to `require "pathname"` as one always needed to do so far seems far better than a partially-defined core Pathname with rough edge cases (those concerns). -- https://bugs.ruby-lang.org/
participants (1)
-
Eregon (Benoit Daloze)