[ruby-core:124616] [Ruby Bug#21848] GC compaction freezes during code reload with 100% CPU in rb_vm_ci_free -> st_general_delete
Issue #21848 has been reported by henrik (Henrik A). ---------------------------------------- Bug #21848: GC compaction freezes during code reload with 100% CPU in rb_vm_ci_free -> st_general_delete https://bugs.ruby-lang.org/issues/21848 * Author: henrik (Henrik A) * Status: Open * ruby -v: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [arm64-darwin24] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- Ruby 4.0.1 freezes with 100% CPU usage when garbage collection runs during Rails code reloading (hot reload in development). The process becomes unresponsive and requires kill -9 to terminate. Environment: - Ruby: 4.0.1 (Homebrew, ARM64) - Platform: macOS 15.6.1 (Darwin 24.6.0, Apple Silicon) - Rails: 8.x with Puma 7.2.0 - Mode: Development with code reloading enabled Steps to reproduce: 1. Run a Rails application with Puma in development mode (bundle exec rails server -e development -b 0.0.0.0) 2. Make a several code change to trigger hot reload, reload the site in the browser in between 3. Process freezes with ~100% CPU Expected behavior: Code reloads successfully, GC completes normally. Actual behavior: Process freezes indefinitely at 100% CPU. Stack trace from sample shows GC stuck in hash table lookup while freeing VM call info: ``` gc_sweep (in libruby.4.0.dylib) gc_compact_plane gc_sweep_page gc_sweep_plane rb_gc_obj_free_vm_weak_references rb_vm_ci_free st_general_delete find_table_bin_ind vm_ci_hash_cmp <-- stuck here, spinning ``` Full relevant stack (Puma worker thread): ``` Thread: puma srv tp 005 rb_ary_each -> rb_yield -> invoke_block_from_c_bh -> rb_vm_exec -> int_upto -> rb_yield (nested loops) -> vm_search_method_slowpath0 -> rb_vm_search_method_slowpath -> newobj_of -> newobj_cache_miss -> gc_continue -> gc_sweep -> gc_compact_plane -> gc_sweep_page -> gc_sweep_plane -> rb_gc_obj_free_vm_weak_references -> rb_vm_ci_free -> st_general_delete -> find_table_bin_ind -> vm_ci_hash_cmp ``` Workaround: Disabling auto-compaction prevents the freeze: GC.auto_compact = false Analysis: The freeze occurs during GC sweep/compaction when freeing cached VM call info entries (rb_vm_ci_free). This is triggered by code reloading which invalidates method caches. The GC appears to get stuck in an infinite loop or extremely slow operation in find_table_bin_ind while looking up entries in the call info hash table. ---Files-------------------------------- rails_stuck.txt (159 KB) -- https://bugs.ruby-lang.org/
Issue #21848 has been updated by luke-gru (Luke Gruber). This bug may be related to #22104. ---------------------------------------- Bug #21848: GC compaction freezes during code reload with 100% CPU in rb_vm_ci_free -> st_general_delete https://bugs.ruby-lang.org/issues/21848#change-117730 * Author: henrik (Henrik A) * Status: Assigned * Assignee: tenderlovemaking (Aaron Patterson) * ruby -v: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [arm64-darwin24] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- Ruby 4.0.1 freezes with 100% CPU usage when garbage collection runs during Rails code reloading (hot reload in development). The process becomes unresponsive and requires kill -9 to terminate. Environment: - Ruby: 4.0.1 (Homebrew, ARM64) - Platform: macOS 15.6.1 (Darwin 24.6.0, Apple Silicon) - Rails: 8.x with Puma 7.2.0 - Mode: Development with code reloading enabled Steps to reproduce: 1. Run a Rails application with Puma in development mode (bundle exec rails server -e development -b 0.0.0.0) 2. Make a several code change to trigger hot reload, reload the site in the browser in between 3. Process freezes with ~100% CPU Expected behavior: Code reloads successfully, GC completes normally. Actual behavior: Process freezes indefinitely at 100% CPU. Stack trace from sample shows GC stuck in hash table lookup while freeing VM call info: ``` gc_sweep (in libruby.4.0.dylib) gc_compact_plane gc_sweep_page gc_sweep_plane rb_gc_obj_free_vm_weak_references rb_vm_ci_free st_general_delete find_table_bin_ind vm_ci_hash_cmp <-- stuck here, spinning ``` Full relevant stack (Puma worker thread): ``` Thread: puma srv tp 005 rb_ary_each -> rb_yield -> invoke_block_from_c_bh -> rb_vm_exec -> int_upto -> rb_yield (nested loops) -> vm_search_method_slowpath0 -> rb_vm_search_method_slowpath -> newobj_of -> newobj_cache_miss -> gc_continue -> gc_sweep -> gc_compact_plane -> gc_sweep_page -> gc_sweep_plane -> rb_gc_obj_free_vm_weak_references -> rb_vm_ci_free -> st_general_delete -> find_table_bin_ind -> vm_ci_hash_cmp ``` Workaround: Disabling auto-compaction prevents the freeze: GC.auto_compact = false Analysis: The freeze occurs during GC sweep/compaction when freeing cached VM call info entries (rb_vm_ci_free). This is triggered by code reloading which invalidates method caches. The GC appears to get stuck in an infinite loop or extremely slow operation in find_table_bin_ind while looking up entries in the call info hash table. ---Files-------------------------------- rails_stuck.txt (159 KB) -- https://bugs.ruby-lang.org/
Issue #21848 has been updated by mame (Yusuke Endoh). Status changed from Assigned to Feedback Assignee deleted (tenderlovemaking (Aaron Patterson)) Can we close this, assuming it was fixed by #22104? The fix will be included in 4.0.6. Could you try it once it is out? Setting this to Feedback for now. ---------------------------------------- Bug #21848: GC compaction freezes during code reload with 100% CPU in rb_vm_ci_free -> st_general_delete https://bugs.ruby-lang.org/issues/21848#change-117918 * Author: henrik (Henrik A) * Status: Feedback * ruby -v: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [arm64-darwin24] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- Ruby 4.0.1 freezes with 100% CPU usage when garbage collection runs during Rails code reloading (hot reload in development). The process becomes unresponsive and requires kill -9 to terminate. Environment: - Ruby: 4.0.1 (Homebrew, ARM64) - Platform: macOS 15.6.1 (Darwin 24.6.0, Apple Silicon) - Rails: 8.x with Puma 7.2.0 - Mode: Development with code reloading enabled Steps to reproduce: 1. Run a Rails application with Puma in development mode (bundle exec rails server -e development -b 0.0.0.0) 2. Make a several code change to trigger hot reload, reload the site in the browser in between 3. Process freezes with ~100% CPU Expected behavior: Code reloads successfully, GC completes normally. Actual behavior: Process freezes indefinitely at 100% CPU. Stack trace from sample shows GC stuck in hash table lookup while freeing VM call info: ``` gc_sweep (in libruby.4.0.dylib) gc_compact_plane gc_sweep_page gc_sweep_plane rb_gc_obj_free_vm_weak_references rb_vm_ci_free st_general_delete find_table_bin_ind vm_ci_hash_cmp <-- stuck here, spinning ``` Full relevant stack (Puma worker thread): ``` Thread: puma srv tp 005 rb_ary_each -> rb_yield -> invoke_block_from_c_bh -> rb_vm_exec -> int_upto -> rb_yield (nested loops) -> vm_search_method_slowpath0 -> rb_vm_search_method_slowpath -> newobj_of -> newobj_cache_miss -> gc_continue -> gc_sweep -> gc_compact_plane -> gc_sweep_page -> gc_sweep_plane -> rb_gc_obj_free_vm_weak_references -> rb_vm_ci_free -> st_general_delete -> find_table_bin_ind -> vm_ci_hash_cmp ``` Workaround: Disabling auto-compaction prevents the freeze: GC.auto_compact = false Analysis: The freeze occurs during GC sweep/compaction when freeing cached VM call info entries (rb_vm_ci_free). This is triggered by code reloading which invalidates method caches. The GC appears to get stuck in an infinite loop or extremely slow operation in find_table_bin_ind while looking up entries in the call info hash table. ---Files-------------------------------- rails_stuck.txt (159 KB) -- https://bugs.ruby-lang.org/
Issue #21848 has been updated by henrik (Henrik A). ruby -v changed from ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [arm64-darwin24] to ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin25] Still reproducible on `ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin25]` (Homebrew, macOS Darwin 25.5.0). Rails 8.1.2 dev server, Puma 8.0.2. Same symptom: during code reloading a thread pegs one core at 100% CPU and never recovers. Only with `GC.auto_compact = true`. #22104 did change something: the old leaf (`rb_vm_ci_free` -> `st_general_delete`) is gone. The same compacting-sweep path now dies one frame over. ``` #0 rb_class_classext_free + 92 #1 rb_class_classext_foreach + 104 #2 rb_gc_obj_free + 148 #3 gc_sweep_plane + 312 #4 gc_sweep_page + 260 #5 gc_compact_plane + 424 #6 gc_sweep + 1144 #7 gc_continue + 304 #8 newobj_cache_miss + 324 #9 newobj_of + 140 #10 rb_str_buf_new + 92 #11 rb_str_concat_literals + 76 #12 vm_exec_core + 16120 #13 rb_vm_exec + 508 ``` Above that, three nested rounds of `rb_vm_opt_getconstant_path` -> `rb_const_search_from` -> `rb_autoload_load` -> `rb_require_string` -> `load_iseq_eval` (autoload triggering autoload). It is a use-after-free, not a loop. The thread re-faults on one instruction forever: ``` stop reason = EXC_BAD_ACCESS (code=2, address=0x127dbfe38) rb_class_classext_free+92: ldr x10, [x9, #0x68] x9 = 0x0000000127dbfdd0 ``` `x9` itself is already in an unmapped page, so the struct is gone before it is walked: ``` 0x127dbfdd0 -> [0x127db0000-0x127dc0000] readable=False writable=False mapped=False ``` The fault lands inside GC mid-sweep, so the SEGV handler cannot progress and the instruction retries indefinitely: hang at 100% CPU, no crash dump. Two `sample(1)` runs 10s apart put 100% of samples in that same leaf. Reproduces reliably here, happy to test a patch. ---------------------------------------- Bug #21848: GC compaction freezes during code reload with 100% CPU in rb_vm_ci_free -> st_general_delete https://bugs.ruby-lang.org/issues/21848#change-118247 * Author: henrik (Henrik A) * Status: Feedback * ruby -v: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin25] * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- Ruby 4.0.1 freezes with 100% CPU usage when garbage collection runs during Rails code reloading (hot reload in development). The process becomes unresponsive and requires kill -9 to terminate. Environment: - Ruby: 4.0.1 (Homebrew, ARM64) - Platform: macOS 15.6.1 (Darwin 24.6.0, Apple Silicon) - Rails: 8.x with Puma 7.2.0 - Mode: Development with code reloading enabled Steps to reproduce: 1. Run a Rails application with Puma in development mode (bundle exec rails server -e development -b 0.0.0.0) 2. Make a several code change to trigger hot reload, reload the site in the browser in between 3. Process freezes with ~100% CPU Expected behavior: Code reloads successfully, GC completes normally. Actual behavior: Process freezes indefinitely at 100% CPU. Stack trace from sample shows GC stuck in hash table lookup while freeing VM call info: ``` gc_sweep (in libruby.4.0.dylib) gc_compact_plane gc_sweep_page gc_sweep_plane rb_gc_obj_free_vm_weak_references rb_vm_ci_free st_general_delete find_table_bin_ind vm_ci_hash_cmp <-- stuck here, spinning ``` Full relevant stack (Puma worker thread): ``` Thread: puma srv tp 005 rb_ary_each -> rb_yield -> invoke_block_from_c_bh -> rb_vm_exec -> int_upto -> rb_yield (nested loops) -> vm_search_method_slowpath0 -> rb_vm_search_method_slowpath -> newobj_of -> newobj_cache_miss -> gc_continue -> gc_sweep -> gc_compact_plane -> gc_sweep_page -> gc_sweep_plane -> rb_gc_obj_free_vm_weak_references -> rb_vm_ci_free -> st_general_delete -> find_table_bin_ind -> vm_ci_hash_cmp ``` Workaround: Disabling auto-compaction prevents the freeze: GC.auto_compact = false Analysis: The freeze occurs during GC sweep/compaction when freeing cached VM call info entries (rb_vm_ci_free). This is triggered by code reloading which invalidates method caches. The GC appears to get stuck in an infinite loop or extremely slow operation in find_table_bin_ind while looking up entries in the call info hash table. ---Files-------------------------------- rails_stuck.txt (159 KB) -- https://bugs.ruby-lang.org/
participants (3)
-
henrik (Henrik A) -
luke-gru (Luke Gruber) -
mame (Yusuke Endoh)