[ruby-core:112865] [Ruby master Bug#19527] Object allocation during garbage collection phase
Issue #19527 has been reported by hjimenez89rb (Hugo Alberto Jiménez Santos). ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Open * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
Issue #19527 has been updated by ufuk (Ufuk Kayserilioglu). Isn't this a duplicate of #19524? I don't think you will get a different answer to this ticket from the one that was given in that ticket. The bug is indicating that a C-extension (probably, the IBM DB Adapter) is allocating during garbage collection phase. ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527#change-102381 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Open * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
Issue #19527 has been updated by rubyFeedback (robert heiler). Seems to be identical indeed. ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527#change-102382 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Open * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
Issue #19527 has been updated by alanwu (Alan Wu). Status changed from Open to Third Party's Issue Unfortunately, the traces are not very helpful. But I agree, it's likely an issue with the [IBM gem](https://github.com/ibmdb/ruby-ibmdb) and not a Ruby issue. Here is a guess as to what's happening: the traces indicate that it's crashing soon after a new Ruby thread starts up, and for "Object allocation during garbage collection phase" to happen at that timing, maybe someone is running Ruby code without holding the global VM lock. Any code paths in the gem that terminates in `rb_thread_call_without_gvl()`, such as ones involving `_ruby_ibm_db_check_sql_errors()`, can have this class of bugs. You can try asking IBM to check this. ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527#change-102383 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Third Party's Issue * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
Issue #19527 has been updated by hjimenez89rb (Hugo Alberto Jiménez Santos). Hi everyone. As I mentioned the first thing I did was check the issue with IBM, if you check the stack message, our application (PEC) is started, then it establish the connection to the database using IBM_GEM and this error is triggered by ruby after the ibm gem connects to the database. [INFO] Process pid 1002200 [INFO] Initializing PEC Engine ... [INFO] Establishing connection to control database ... [INFO] Engine started /ruby-3.2.1/lib/ruby/3.2.0/rubygems/specification.rb:1048: [BUG] object allocation during garbage collection phase ruby 3.2.1. Our application is something like a process dispatcher for a few moments it runs very well but after that suddenly ruby crashes. about your guess that someone else is running Ruby code without holding the global VM lock, is not possible because in our Centos 8 server is the only application running and we have enough physical resources to run. ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527#change-102384 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Third Party's Issue * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
Issue #19527 has been updated by alanwu (Alan Wu). Please understand that while Ruby giving you the crash report saying `[BUG] object allocation during garbage collection phase` is the proximate cause of the problem, I'm telling you that the ultimate cause is likely a bug in the IBM gem. C extensions have contracts they must uphold, and if they don't, they crash the whole process. Ruby is the messenger in all crashes, but it is incorrect to always blame the messenger. Crashes involving threads like the one you are facing are by nature non-local; the code initiating the crash is not always problematic.
about your guess that someone else is running Ruby code without holding the global VM lock, is not possible because in our Centos 8 server is the only application running and we have enough physical resources to run.
It is absolutely possible. The VM lock arbitrates concurrency of threads within the same process. From the stack trace, it's clear that you have multiple threads in your Ruby process. The lock has nothing to do with how many applications you run on your server. If you have multiple threads in your Ruby process, it's in-play. Here is something concrete you can try: 1. Clone https://github.com/ibmdb/ruby-ibmdb to get the code for the version you're running. 5.4.0 seems to be at https://github.com/ibmdb/ruby-ibmdb/commit/2eec1bfe637e3320721daa5a92d3aa800... 2. Adjust the version, `gem build` and `gem install`, then point your `Gemfile` to that local version 3. Make sure you can still reproduce the crash with this local version of the gem 3. Apply the following patch (untested, you might have to fix build errors): ```patch diff --git a/IBM_DB_Adapter/ibm_db/ext/ibm_db.c b/IBM_DB_Adapter/ibm_db/ext/ibm_db.c index 200a527..0b139ee 100644 --- a/IBM_DB_Adapter/ibm_db/ext/ibm_db.c +++ b/IBM_DB_Adapter/ibm_db/ext/ibm_db.c @@ -686,6 +686,7 @@ static void _ruby_ibm_db_mark_stmt_struct(stmt_handle *handle) static inline VALUE ibm_Ruby_Thread_Call(rb_blocking_function_t *func, void *data1, rb_unblock_function_t *ubf, void *data2) { + return func(data1); void *(*f)(void*) = (void *(*)(void*))func; return (VALUE)rb_thread_call_without_gvl(f, data1, ubf, data2); } ``` 5. Repeat (2) to rebuild and reinstall the gem now that it's changed 6. See if the crash still reproduces. If this patch makes the crash go away, we can say with high confidence that the `ibm_db` gem is misusing `rb_thread_call_without_gvl()`. Send this to IBM as a bug report. If the crash still happens, maybe you can try reproducing the bug without any third-party C extensions. If you can do that, that'd be a more actionable bug report for us. There is not much we can do on our end with the information you have posted. ---------------------------------------- Bug #19527: Object allocation during garbage collection phase https://bugs.ruby-lang.org/issues/19527#change-102393 * Author: hjimenez89rb (Hugo Alberto Jiménez Santos) * Status: Third Party's Issue * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We are currently developing a Ruby based web application which connects to a DB2 Database and we have been using ibm_db-5.4.0 to establish a connection, suddenly we got a error related to RUBY garbage collector PHASE. We have checked the issue with IBM_team to make sure that It was not a IBM_GEM problem but as a result of their tests, IBM_GEM is working in different cases but for us we face up with those errors even with those versions (2.7.6, 3.1.2, 3.2.1): *0x0/usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760: [BUG] object allocation during garbage collection phase ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] *Exception occurred on Step thread ID #SID:34117;RSEQ:911723; wrong instance allocation; backtrace: /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in server_info' (RuntimeError) /usr/local/rvm/gems/ruby-3.1.2/gems/ibm_db-5.4.0/lib/active_record/connection_adapters/ibm_db_adapter.rb:760:in initialize'. (all trace is attached in this ticket) OS name: "CentOS" version: "8" architecture: "x86_64" rvm: version: "1.29.12 (latest)" ---Files-------------------------------- LOGS3.txt (12.5 KB) LOGS4.txt (127 KB) LOGS2.txt (9.62 KB) LOGS1.txt (123 KB) -- https://bugs.ruby-lang.org/
participants (4)
-
alanwu (Alan Wu) -
hjimenez89rb
-
rubyFeedback (robert heiler)
-
ufuk (Ufuk Kayserilioglu)