[ruby-core:113209] [Ruby master Bug#19596] Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2

Issue #19596 has been reported by silva96 (Benjamín Silva). ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596 * Author: silva96 (Benjamín Silva) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by k0kubun (Takashi Kokubun). Can you try turning on GC metrics? https://docs.newrelic.com/docs/apm/agents/ruby-agent/features/garbage-collec... ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102752 * Author: silva96 (Benjamín Silva) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by byroot (Jean Boussier). With the information provided it's impossible to tell what is wrong. As @k0kubun suggested, you should profile your application on both version to see if some code is disproportionately slow on 3.2 vs 2.7. ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102760 * Author: silva96 (Benjamín Silva) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by byroot (Jean Boussier).
But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic
Yes, extremely likely. Can you show the whole breakdown and not just the "ruby" part? If response time stayed similar but "ruby" went up, it means other parts went up. I suspect it's simply that you version of New Relic is no longer hooking properly in one of the libraries you use for IOs (e.g. redis/memcached/db). ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102761 * Author: silva96 (Benjamín Silva) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by silva96 (Benjamín Silva). File clipboard-202304130930-c24f0.png added File clipboard-202304130936-vtuwy.png added File 4511491.json added File clipboard-202304130938-rj5ny.png added byroot (Jean Boussier) wrote in #note-3:
But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic
Yes, extremely likely. Can you show the whole breakdown and not just the "ruby" part? If response time stayed similar but "ruby" went up, it means other parts went up.
I suspect it's simply that you version of New Relic is no longer hooking properly in one of the libraries you use for IOs (e.g. redis/memcached/db).
Here's the whole breakdown for the past 3 days.  Also, here's a profiling session:  (Also attache the whole JSON) And this is the GC Execution I just enabled:  ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102767 * Author: silva96 (Benjamín Silva) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) clipboard-202304130930-c24f0.png (253 KB) clipboard-202304130936-vtuwy.png (618 KB) 4511491.json (1010 KB) clipboard-202304130938-rj5ny.png (100 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by byroot (Jean Boussier). Status changed from Open to Closed Ok, so as expected, where Ruby is up Postgres is down. This is very likely be to be caused by how NewRelic attribute time to either. I suspect it's when we changed the profiling API in 3.0 to show the C frames. Either way, I don't think there is any performance degradation here, just different (likely more accurate) reporting. ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102770 * Author: silva96 (Benjamín Silva) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) clipboard-202304130930-c24f0.png (253 KB) clipboard-202304130936-vtuwy.png (618 KB) 4511491.json (1010 KB) clipboard-202304130938-rj5ny.png (100 KB) -- https://bugs.ruby-lang.org/

Issue #19596 has been updated by Eregon (Benoit Daloze). The Response time no longer being the total of everything except Web external seems to make it clear this is a NewRelic issue. Maybe something is getting counted twice or so. ---------------------------------------- Bug #19596: Decreased performance after upgrading from ruby 2.7.2 to ruby 3.2.2 https://bugs.ruby-lang.org/issues/19596#change-102776 * Author: silva96 (Benjamín Silva) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I just upgraded to ruby 3.2.2 from 2.7.2 in one Rails app in production. Tried with and without YJIT, but no matter what, I'm having worst performance. Time spent in ruby went from 20ms to more than 50ms and very inconsistent (peaks) - I'm using RVM - I've set MALLOC_ARENAS_MAX=2 I may be wrong, but I suspect I have worst performance because I see this chart in newrelic increased right after the upgrade  In the picture, the chart has the "ruby" toggle, to see how much time is spent in ruby operations. But I don't see a noticeable impact on the overall response times. So that makes me think that maybe, ruby 3.2 reports differently than 2.7 in newrelic  Right now, I'm using YJIT, and I was expecting to see some kind of improvements in app performance upgrading from 2.7 to 3.2 YJIT, but there's no performance gain, is that even possible? The website is https://profe.social, it's a very standard content site, no super expensive ruby computations, most operations are rendering erb views pulling data from the database. Any Ideas how can I get performance improvements? Best regards. ---Files-------------------------------- clipboard-202304130005-opyyk.png (120 KB) clipboard-202304130014-vnb0h.png (134 KB) clipboard-202304130930-c24f0.png (253 KB) clipboard-202304130936-vtuwy.png (618 KB) 4511491.json (1010 KB) clipboard-202304130938-rj5ny.png (100 KB) -- https://bugs.ruby-lang.org/
participants (4)
-
byroot (Jean Boussier)
-
Eregon (Benoit Daloze)
-
k0kubun (Takashi Kokubun)
-
silva96