[ruby-core:116201] [Ruby master Bug#20184] Ruby segfaults on Fly.io with 256 MB RAM

Issue #20184 has been reported by aalin (Andreas Alin). ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184 * Author: aalin (Andreas Alin) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by nobu (Nobuyoshi Nakada). In shape.c:Init_default_shapes(), the result of `mmap` for `rb_shape_tree_ptr->shape_cache` seems unchecked. ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-106219 * Author: aalin (Andreas Alin) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by nobu (Nobuyoshi Nakada). Status changed from Open to Closed Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED commit:8b8dcc7af174def5216044019c1d3e42edfdc7cf should have fixed this. ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-106220 * Author: aalin (Andreas Alin) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by ioquatix (Samuel Williams). I was also investigating this issue. On low-memory systems (< ~400MB RAM+Swap), `Init_default_shapes` will fail and cause Ruby to segfault later. Even something as simple as ruby -e "puts 123" will fail. The 2nd `mmap` in `shape.c` tries to allocate 384MB as a cache. This fails with `MAP_FAILED` which is stored into a `shape_cache` pointer and later this pointer is derefenced causing a segfault. The current merged fix skips the cache if the `mmap` fails, but might I suggest we explore the following: - Divide and conquer strategy for finding an allocation size that works (e.g. divide by half and try again a few times). - Expose an environment variable to control the size, so that those on memory constrained systems can at least enable a smaller cache. ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-106228 * Author: aalin (Andreas Alin) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by ioquatix (Samuel Williams). I also wonder if we should add some CI for running ruby on a low memory/virtual memory constrained system. ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-106229 * Author: aalin (Andreas Alin) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by anton-stepanof (Anton Stepanov). Backport PR for 3.3: https://github.com/ruby/ruby/pull/9805 ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-106636 * Author: aalin (Andreas Alin) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/

Issue #20184 has been updated by naruse (Yui NARUSE). Backport changed from 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE ruby_3_3 commit:53f0c5a4e8834f11af0f903d2c59754d9be2a7f2. ---------------------------------------- Bug #20184: Ruby segfaults on Fly.io with 256 MB RAM https://bugs.ruby-lang.org/issues/20184#change-107397 * Author: aalin (Andreas Alin) * Status: Closed * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE ---------------------------------------- I've been running a Ruby 3.2 app on Fly.io with 256 MB RAM and it has been working fine. When I upgraded to Ruby 3.3.0, any Ruby script would crash immediately with a segfault. I haven't been able to reproduce it outside of fly.io. I created a repository with a basic rack app which crashes on fly.io. The readme includes the full output and a GDB backtrace. https://github.com/aalin/ruby-3-3-0-fly-crash -- https://bugs.ruby-lang.org/
participants (5)
-
aalin (Andreas Alin)
-
anton-stepanof (Anton Stepanov)
-
ioquatix (Samuel Williams)
-
naruse (Yui NARUSE)
-
nobu (Nobuyoshi Nakada)