Issue #19378 has been updated by aidog (Andi Idogawa). File MINIMAL-stat-fastpath.patch added File recommendations.txt added ruby -v changed from 3.2.0 to 4.0.5 3 years and a few RubyKaigis later, I've run some tests. Using GetFileInformationByName (Windows 11 24H2+) in winnt_stat I'm seeing speedups of 2.32x faster loading of active_support/all 2.77x for loading nokogiri and File.stat of an existing file 112 us -> 21 us (5.4x). Benchmark results — minimal one-file patch vs unpatched, Ruby 4.0.5 (best of 5 cold-process runs) ┌──────────────────────┬───────────┬──────────┬─────────┬──────────┐ │ Gem │ Unpatched │ Minimal │ Speedup │ Features │ │ │ │ patch │ │ │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ rubocop │ 3186 ms │ 1651 ms │ 1.93× │ 1221 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ sinatra │ 546 ms │ 296 ms │ 1.85× │ 137 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ faraday │ 340 ms │ 204 ms │ 1.67× │ 112 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ sequel │ 254 ms │ 140 ms │ 1.81× │ 60 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ rspec │ 213 ms │ 100 ms │ 2.13× │ 62 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ bigdecimal │ 52 ms │ 18 ms │ 2.95× │ 2 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ nokogiri (earlier) │ 224 ms │ 81 ms │ 2.77× │ 84 │ ├──────────────────────┼───────────┼──────────┼─────────┼──────────┤ │ active_support/all │ 1005 ms │ 433 ms │ 2.32× │ 433 │ │ (earlier) │ │ │ │ │ └──────────────────────┴───────────┴──────────┴─────────┴──────────┘ I've attached a Claude patch about 70 lines of code, which i'm sure a lot of it is wrong, but it might be interesting regardless. It is not a big change. Some other improvements were found, but nothing else made such a huge difference. ---------------------------------------- Bug #19378: Windows: Use less syscalls for faster require of big gems https://bugs.ruby-lang.org/issues/19378#change-117694 * Author: aidog (Andi Idogawa) * Status: Assigned * Assignee: windows * ruby -v: 4.0.5 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello 🙂 ## Problem require is slow on windows for big gems. (example: require 'gtk3'=> 3 seconds+). This is a problem for people who want to make cross platform GUI apps with ruby. ## Possible Reason As touched on in [#15797](https://bugs.ruby-lang.org/issues/15797) it seems like require uses realpath, which is emulated on windows. It checks every parent directory. The same syscalls run many times. ## Testfile C:\tmp\speedtest\testrequire.rb: ``` ruby require __dir__ + "/helloworld1.rb" require __dir__ + "/helloworld2.rb" ``` ``` shell ruby --disable-gems C:\tmp\speedtest\testrequire.rb ``` ### Syscalls per File/Directory: 1. CreateFile 2. QueryInformationVolume 3. QueryIdInformation 4. QueryAllInformationFile 5. QueryNameInformationFile 6. QueryNameInformationFile 7. QueryNormalizedNameInformationFile 8. CloseFile ### Files/Directories checked 1. C:\tmp 2. C:\tmp\speedtest 3. C:\tmp\speedtest\helloworld1.rb 4. C:\tmp 5. C:\tmp\speedtest 6. C:\tmp\speedtest\helloworld2.rb For two required files Ruby had to do 8*6 = **48** syscalls. The syscalls orginate from rb_w32_reparse_symlink_p / lstat Rubygems live in subfolders with 9+ parts: "C:\Ruby32-x64\lib\ruby\gems\3.2.0\gems\glib2-4.0.8\lib\glib2\variant.rb" Each file takes 8 * 9 = **72**+ calls. For variant.rb it is **80** calls. The result for the syscalls don't change in such a short time, so it should be possible to cache it. With require_relative it's twice as many calls. ## Other testcases Same result: ``` ruby File.realpath __dir__ + "/helloworld1.rb" File.realpath __dir__ + "/helloworld2.rb" ``` ``` ruby File.stat __dir__ + "/helloworld1.rb" File.stat __dir__ + "/helloworld2.rb" ``` It does not happen in $LOAD_PATH.resolve_feature_path(__dir__ + "/helloworld1.rb") ## Request Would it be possible to cache the stat calls when using require? I tried to implement a cache inside the ruby source code, but failed. If not, is there now a way to combine ruby files into one? I previously talked about require here: [YJIT: Windows support lacking.](https://bugs.ruby-lang.org/issues/19325#note-11) ## How to reproduce Ruby versions: At least 3.0+, most likely older ones too. Tested using Ruby Installer 3.1 and 3.2. [Procmon Software by Sysinternals](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) ---Files-------------------------------- windows-no-realpath-require.patch (992 Bytes) windows-revert-79a4484a.patch (5.42 KB) MINIMAL-stat-fastpath.patch (6.41 KB) recommendations.txt (4.01 KB) -- https://bugs.ruby-lang.org/