Issue #22275 has been updated by Eregon (Benoit Daloze). (for some reason this didn't get sent to the `ruby-core` mailing list. Other recent `Misc` tickets didn't have the issue, so unclear what happened) ---------------------------------------- Misc #22275: Ruby::Box support plan for RubyGems and Bundler https://bugs.ruby-lang.org/issues/22275#change-118745 * Author: hsbt (Hiroshi SHIBATA) * Status: Open ---------------------------------------- This is a tracking issue for making RubyGems and Bundler work with Ruby::Box, so that reviewers can see the whole picture and what to look at next in one place. ## Goal The goal is to realize [Feature #13847](https://bugs.ruby-lang.org/issues/13847) with Ruby::Box: loading a specific version of a gem, including default gems and C extensions, isolated from the copy that RubyGems/Bundler themselves use. Vendoring with namespace rewriting has been our workaround for pure-Ruby gems for years; Box removes that limitation. As a further step, we want RubyGems/Bundler to activate different gem versions per box, which enables the use cases already proposed around Box: gradual dependency upgrades, plugin systems with conflicting dependencies, and multiple applications in one process. ## First milestone Before designing such APIs, Box needs to leave experimental status. Our first milestone is that Rails and RubyGems/Bundler themselves work under `RUBY_BOX=1`, and that the ruby/rubygems repository runs its test suites with Box enabled continuously so we do not regress. ## Current status A `rails new` application now boots, serves scaffold CRUD requests, and reloads code in development mode once [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) is applied. The rubygems test suite runs with 0 failures under `RUBY_BOX=1` ([ruby/rubygems#9826](https://github.com/ruby/rubygems/pull/9826), open), with pending marks referencing the unfixed core bugs below. ## Fixed ### ruby/ruby * [ruby/ruby#18218](https://github.com/ruby/ruby/pull/18218) / [ruby/ruby#18219](https://github.com/ruby/ruby/pull/18219) — `BUNDLER_SETUP` was consumed outside the main box ([Bug #22123](https://bugs.ruby-lang.org/issues/22123)). Shipped in 4.0.7. * [ruby/ruby#18509](https://github.com/ruby/ruby/pull/18509) — box-local extension DLLs on Windows were unloaded too early; now deferred. ### ruby/rubygems * [ruby/rubygems#9809](https://github.com/ruby/rubygems/pull/9809) — Bundler evaluated gemspecs through `TOPLEVEL_BINDING`, which always belongs to the main box. It now uses a binding in the box Bundler is loaded in, so gemspecs resolve the right `Gem::Specification`. * [ruby/rubygems#9810](https://github.com/ruby/rubygems/pull/9810) — the `gem` CLI died under `RUBY_BOX=1`. `Marshal`-based deep copies could not resolve `Gem::` constants across boxes and were replaced with a plain deep dup, and `RUBY_BOX` is now stripped from extension build subprocesses to avoid an `RbConfig.expand` infinite recursion. Includes a CLI canary test that runs under `RUBY_BOX=1`. ## Open pull requests All review-requested to @tagomoris: * [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) — autoload-triggered require bypasses the box's `Kernel#require` ([Bug #21830](https://bugs.ruby-lang.org/issues/21830)). This alone takes Rails from 500 on every request to fully working. * [ruby/ruby#18546](https://github.com/ruby/ruby/pull/18546) — `Symbol#to_proc` ignores box-local method definitions ([Bug #22015](https://bugs.ruby-lang.org/issues/22015), revives [ruby/ruby#16865](https://github.com/ruby/ruby/pull/16865)). This alone unblocks Bundler's spec harness, which dies in rspec-core's `&:shellsplit` before running a single spec. * [ruby/ruby#18544](https://github.com/ruby/ruby/pull/18544) — `Marshal.load` resolves classes in the root box ([Bug #22090](https://bugs.ruby-lang.org/issues/22090)); an in-process round-trip of `Gem::Version` fails today. * [ruby/ruby#18536](https://github.com/ruby/ruby/pull/18536) — the box extension copy embeds the full path in the temporary filename, exceeding NAME_MAX on deep paths, and the name is predictable ([Bug #22110](https://bugs.ruby-lang.org/issues/22110)). * [ruby/ruby#18535](https://github.com/ruby/ruby/pull/18535) — with `--disable=gems`, modules prepended to `Kernel` in a user box end up behind `Kernel` in the ancestry, so the prepended method is never dispatched. ## Remaining problem without a fix [Bug #21867](https://bugs.ruby-lang.org/issues/21867) turned out to be broader than `warn`: reassigning `$stdout`/`$stderr` is invisible to builtin writers under Box, which breaks output-capturing test helpers everywhere. We could not find a harness-side workaround. ## Critical path The most critical items for the milestone are [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) (everything Zeitwerk-based, i.e. Rails, depends on it) and [ruby/ruby#18546](https://github.com/ruby/ruby/pull/18546) (Bundler's entire spec suite is gated behind it). After those, [ruby/ruby#18544](https://github.com/ruby/ruby/pull/18544) and [Bug #21867](https://bugs.ruby-lang.org/issues/21867) let us drop most of the pending marks in the rubygems suite. -- https://bugs.ruby-lang.org/