Issue #21458 has been updated by Eregon (Benoit Daloze). hsbt (Hiroshi SHIBATA) wrote in #note-9:
But I'm against to add cli test like `rdoc --version` because we explicitly ignored that with development version.
I don't think such regressions were ever intentional. They are real bugs, and if ruby-dev-builder wouldn't test such things it would have broken gems CI testing against ruby-head, and maybe some gems would have dropped testing against ruby-head or started to just always ignore the result. So I think it's a clear win to test an installed Ruby in CRuby CI. Also it seems a good time to repeat that every production use of Ruby uses an installed Ruby. None of them uses a Ruby from just `make`. So it seems of utmost important to test something close to what people use in production (or simply what every CRuby user uses). There has also been numerous bugs related to `make install` and `tool/rbinstall.rb`. Some of those would have likely been prevented with tests running on an installed Ruby. ---------------------------------------- Misc #21458: Test 'make install'? https://bugs.ruby-lang.org/issues/21458#change-114376 * Author: MSP-Greg (Greg L) * Status: Open ---------------------------------------- Can `make install` be tested? Two items come to mind: 1. Test that all gems can be required? 2. Check that all gem binstubs are installed and work. Some of the above are being done in ruby-loco and https://github.com/ruby/ruby-dev-builder/blob/master/cli_test.rb. Both of these are cron jobs, they don't run on every commit. A long time ago, I think this was discussed, but it was back when CI was much more limited. It was also mixed in with the idea of running tests from the 'install' folder, rather than the 'build' folder. Maybe a few jobs could be added to check `make install` on the three main OS's (Ubuntu, macOS, Windows)? Lastly, with minimal changes, tests can be run from the install folder, which is what ruby-loco has done for years. Recently, two tests that I believe are skipped here started failing: ``` TestGemRequire#test_realworld_default_gem Line: 471 Expected "" to not be empty. TestGemRequire#test_realworld_upgraded_default_gem Line: 492 <"999.99.9"> expected but was <"/ruby/tmp/test_rubygems_20250628-8920-sifyoe/test_realworld_upgraded_default_gem.rb:3:in '<main>': undefined method 'version' for nil (NoMethodError)">. ``` -- https://bugs.ruby-lang.org/