[ruby-core:113052] [Ruby master Bug#19562] Setting default_(in|ex)ternal encoding after running an Ractor freezes ruby

Issue #19562 has been reported by larskanis (Lars Kanis). ---------------------------------------- Bug #19562: Setting default_(in|ex)ternal encoding after running an Ractor freezes ruby https://bugs.ruby-lang.org/issues/19562 * Author: larskanis (Lars Kanis) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following command blocks infinitely and must be killed with "kill -9" ``` ruby -e "Ractor.new{}.take; Encoding.default_external = Encoding::ISO8859_2" ``` Backtrace is: ``` #0 futex_wait (private=0, expected=2, futex_word=0x55555555abd0) at ../sysdeps/nptl/futex-internal.h:146 #1 __GI___lll_lock_wait (futex=futex@entry=0x55555555abd0, private=0) at ./nptl/lowlevellock.c:49 #2 0x00007ffff760a082 in lll_mutex_lock_optimized (mutex=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:48 #3 ___pthread_mutex_lock (mutex=mutex@entry=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:93 #4 0x00007ffff7c5a40d in rb_native_mutex_lock (lock=lock@entry=0x55555555abd0) at ruby-3.2.1/thread_pthread.c:556 #5 0x00007ffff7cc5376 in vm_lock_enter (cr=0x55555555deb0, vm=0x55555555aba0, locked=locked@entry=false, no_barrier=no_barrier@entry=false, lev=0x55555555ac00) at vm_sync.c:56 #6 0x00007ffff7cc5747 in rb_vm_lock_body () at vm_sync.c:175 #7 0x00007ffff7bc02c2 in rb_vm_lock (line=1829, file=0x7ffff7e209cb "ractor.c") at ruby-3.2.1/vm_sync.h:57 #8 ractor_check_blocking (remained_thread_cnt=<optimized out>, file=0x7ffff7e209cb "ractor.c", line=1865, cr=0x55555555deb0) at ractor.c:1829 #9 ractor_check_blocking (line=1865, file=0x7ffff7e209cb "ractor.c", remained_thread_cnt=<optimized out>, cr=0x55555555deb0) at ractor.c:1811 #10 rb_ractor_blocking_threads_inc (cr=0x55555555deb0, file=file@entry=0x7ffff7e29b04 "thread.c", line=line@entry=1488) at ractor.c:1865 #11 0x00007ffff7c60977 in blocking_region_begin (th=0x55555555d070, region=0x7fffffffd220, ubf=0x7ffff7c5b300 <ubf_select>, arg=0x55555555d070, fail_if_interrupted=<optimized out>) at thread.c:1488 #12 0x00007ffff7c61edf in rb_nogvl (func=0x7ffff7ac66e0 <no_gvl_stat>, data1=0x7fffffffd2a0, ubf=<optimized out>, data2=0x55555555d070, flags=<optimized out>) at thread.c:1548 #13 0x00007ffff7acdfbc in stat_without_gvl (st=0x7fffffffd2b0, path=0x7ffff27dfe88 "ruby-3.2.1/lib/ruby/3.2.0/x86_64-linux/enc/iso_8859_2.so") at file.c:1171 #14 rb_check_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, origenc=0x555555565030, mode=mode@entry=RB_REALPATH_STRICT) at file.c:4491 #15 0x00007ffff7acebaa in rb_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, strict=strict@entry=1) at file.c:4536 #16 0x00007ffff7b2cdd4 in require_internal (ec=0x55555555e1f0, fname=<optimized out>, exception=1, warn=<optimized out>) at load.c:1195 #17 0x00007ffff7aa0a17 in load_encoding (name=0x5555556049b0 "ISO-8859-2") at encoding.c:779 #18 0x00007ffff7aa2ef8 in check_encoding (enc=0x5555556049d0) at encoding.c:191 #19 enc_check_encoding (obj=140737335533800) at encoding.c:202 #20 rb_to_encoding (enc=enc@entry=140737335533800) at encoding.c:307 #21 0x00007ffff7aa4049 in enc_set_default_encoding (name=0x7ffff7de6287 "external", encoding=140737335533800, def=0x7ffff7fb00c0 <default_external>) at encoding.c:1563 #22 rb_enc_set_default_external (encoding=140737335533800) at encoding.c:1637 #23 0x00007ffff7aa411f in set_default_external (klass=<optimized out>, encoding=140737335533800) at encoding.c:1658 #24 0x00007ffff7c99037 in vm_call_cfunc_with_frame (ec=0x55555555e1f0, reg_cfp=0x7ffff6f7df90, calling=<optimized out>) at ruby-3.2.1/vm_insnhelper.c:3268 #25 0x00007ffff7ca9564 in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at ruby-3.2.1/vm_callinfo.h:367 #26 vm_exec_core (ec=0x55555555e1f0, initial=128) at ruby-3.2.1/insns.def:820 #27 0x00007ffff7caee93 in rb_vm_exec (ec=0x55555555e1f0, jit_enable_p=true) at vm.c:2374 #28 0x00007ffff7abfbf1 in rb_ec_exec_node (ec=ec@entry=0x55555555e1f0, n=n@entry=0x7ffff796c2c0) at eval.c:289 #29 0x00007ffff7ac60b3 in ruby_run_node (n=0x7ffff796c2c0) at eval.c:330 #30 0x0000555555555187 in rb_main (argv=0x7fffffffdb98, argc=3) at ./main.c:38 #31 main (argc=<optimized out>, argv=<optimized out>) at ./main.c:57 ``` When the encoding was already loaded before running `Ractor.new`, then it works and doesn't block: ``` ruby -e "''.encode(Encoding::ISO8859_2); Ractor.new{}.take; Encoding.default_internal = Encoding::ISO8859_2" ``` When the encoding is not assigned as default encoding, but used otherwise, it works too: ``` ruby -e "Ractor.new{}.take; ''.encode(Encoding::ISO8859_2)" ``` -- https://bugs.ruby-lang.org/

Issue #19562 has been updated by luke-gru (Luke Gruber). I don't know if this fix is acceptable, but I have a PR here: https://github.com/ruby/ruby/pull/7656 ---------------------------------------- Bug #19562: Setting default_(in|ex)ternal encoding after running an Ractor freezes ruby https://bugs.ruby-lang.org/issues/19562#change-102648 * Author: larskanis (Lars Kanis) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following command blocks infinitely and must be killed with "kill -9" ``` ruby -e "Ractor.new{}.take; Encoding.default_external = Encoding::ISO8859_2" ``` Backtrace is: ``` #0 futex_wait (private=0, expected=2, futex_word=0x55555555abd0) at ../sysdeps/nptl/futex-internal.h:146 #1 __GI___lll_lock_wait (futex=futex@entry=0x55555555abd0, private=0) at ./nptl/lowlevellock.c:49 #2 0x00007ffff760a082 in lll_mutex_lock_optimized (mutex=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:48 #3 ___pthread_mutex_lock (mutex=mutex@entry=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:93 #4 0x00007ffff7c5a40d in rb_native_mutex_lock (lock=lock@entry=0x55555555abd0) at ruby-3.2.1/thread_pthread.c:556 #5 0x00007ffff7cc5376 in vm_lock_enter (cr=0x55555555deb0, vm=0x55555555aba0, locked=locked@entry=false, no_barrier=no_barrier@entry=false, lev=0x55555555ac00) at vm_sync.c:56 #6 0x00007ffff7cc5747 in rb_vm_lock_body () at vm_sync.c:175 #7 0x00007ffff7bc02c2 in rb_vm_lock (line=1829, file=0x7ffff7e209cb "ractor.c") at ruby-3.2.1/vm_sync.h:57 #8 ractor_check_blocking (remained_thread_cnt=<optimized out>, file=0x7ffff7e209cb "ractor.c", line=1865, cr=0x55555555deb0) at ractor.c:1829 #9 ractor_check_blocking (line=1865, file=0x7ffff7e209cb "ractor.c", remained_thread_cnt=<optimized out>, cr=0x55555555deb0) at ractor.c:1811 #10 rb_ractor_blocking_threads_inc (cr=0x55555555deb0, file=file@entry=0x7ffff7e29b04 "thread.c", line=line@entry=1488) at ractor.c:1865 #11 0x00007ffff7c60977 in blocking_region_begin (th=0x55555555d070, region=0x7fffffffd220, ubf=0x7ffff7c5b300 <ubf_select>, arg=0x55555555d070, fail_if_interrupted=<optimized out>) at thread.c:1488 #12 0x00007ffff7c61edf in rb_nogvl (func=0x7ffff7ac66e0 <no_gvl_stat>, data1=0x7fffffffd2a0, ubf=<optimized out>, data2=0x55555555d070, flags=<optimized out>) at thread.c:1548 #13 0x00007ffff7acdfbc in stat_without_gvl (st=0x7fffffffd2b0, path=0x7ffff27dfe88 "ruby-3.2.1/lib/ruby/3.2.0/x86_64-linux/enc/iso_8859_2.so") at file.c:1171 #14 rb_check_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, origenc=0x555555565030, mode=mode@entry=RB_REALPATH_STRICT) at file.c:4491 #15 0x00007ffff7acebaa in rb_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, strict=strict@entry=1) at file.c:4536 #16 0x00007ffff7b2cdd4 in require_internal (ec=0x55555555e1f0, fname=<optimized out>, exception=1, warn=<optimized out>) at load.c:1195 #17 0x00007ffff7aa0a17 in load_encoding (name=0x5555556049b0 "ISO-8859-2") at encoding.c:779 #18 0x00007ffff7aa2ef8 in check_encoding (enc=0x5555556049d0) at encoding.c:191 #19 enc_check_encoding (obj=140737335533800) at encoding.c:202 #20 rb_to_encoding (enc=enc@entry=140737335533800) at encoding.c:307 #21 0x00007ffff7aa4049 in enc_set_default_encoding (name=0x7ffff7de6287 "external", encoding=140737335533800, def=0x7ffff7fb00c0 <default_external>) at encoding.c:1563 #22 rb_enc_set_default_external (encoding=140737335533800) at encoding.c:1637 #23 0x00007ffff7aa411f in set_default_external (klass=<optimized out>, encoding=140737335533800) at encoding.c:1658 #24 0x00007ffff7c99037 in vm_call_cfunc_with_frame (ec=0x55555555e1f0, reg_cfp=0x7ffff6f7df90, calling=<optimized out>) at ruby-3.2.1/vm_insnhelper.c:3268 #25 0x00007ffff7ca9564 in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at ruby-3.2.1/vm_callinfo.h:367 #26 vm_exec_core (ec=0x55555555e1f0, initial=128) at ruby-3.2.1/insns.def:820 #27 0x00007ffff7caee93 in rb_vm_exec (ec=0x55555555e1f0, jit_enable_p=true) at vm.c:2374 #28 0x00007ffff7abfbf1 in rb_ec_exec_node (ec=ec@entry=0x55555555e1f0, n=n@entry=0x7ffff796c2c0) at eval.c:289 #29 0x00007ffff7ac60b3 in ruby_run_node (n=0x7ffff796c2c0) at eval.c:330 #30 0x0000555555555187 in rb_main (argv=0x7fffffffdb98, argc=3) at ./main.c:38 #31 main (argc=<optimized out>, argv=<optimized out>) at ./main.c:57 ``` When the encoding was already loaded before running `Ractor.new`, then it works and doesn't block: ``` ruby -e "''.encode(Encoding::ISO8859_2); Ractor.new{}.take; Encoding.default_internal = Encoding::ISO8859_2" ``` When the encoding is not assigned as default encoding, but used otherwise, it works too: ``` ruby -e "Ractor.new{}.take; ''.encode(Encoding::ISO8859_2)" ``` -- https://bugs.ruby-lang.org/

Issue #19562 has been updated by hsbt (Hiroshi SHIBATA). Status changed from Open to Assigned Assignee set to ko1 (Koichi Sasada) ---------------------------------------- Bug #19562: Setting default_(in|ex)ternal encoding after running an Ractor freezes ruby https://bugs.ruby-lang.org/issues/19562#change-109875 * Author: larskanis (Lars Kanis) * Status: Assigned * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following command blocks infinitely and must be killed with "kill -9" ``` ruby -e "Ractor.new{}.take; Encoding.default_external = Encoding::ISO8859_2" ``` Backtrace is: ``` #0 futex_wait (private=0, expected=2, futex_word=0x55555555abd0) at ../sysdeps/nptl/futex-internal.h:146 #1 __GI___lll_lock_wait (futex=futex@entry=0x55555555abd0, private=0) at ./nptl/lowlevellock.c:49 #2 0x00007ffff760a082 in lll_mutex_lock_optimized (mutex=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:48 #3 ___pthread_mutex_lock (mutex=mutex@entry=0x55555555abd0) at ./nptl/pthread_mutex_lock.c:93 #4 0x00007ffff7c5a40d in rb_native_mutex_lock (lock=lock@entry=0x55555555abd0) at ruby-3.2.1/thread_pthread.c:556 #5 0x00007ffff7cc5376 in vm_lock_enter (cr=0x55555555deb0, vm=0x55555555aba0, locked=locked@entry=false, no_barrier=no_barrier@entry=false, lev=0x55555555ac00) at vm_sync.c:56 #6 0x00007ffff7cc5747 in rb_vm_lock_body () at vm_sync.c:175 #7 0x00007ffff7bc02c2 in rb_vm_lock (line=1829, file=0x7ffff7e209cb "ractor.c") at ruby-3.2.1/vm_sync.h:57 #8 ractor_check_blocking (remained_thread_cnt=<optimized out>, file=0x7ffff7e209cb "ractor.c", line=1865, cr=0x55555555deb0) at ractor.c:1829 #9 ractor_check_blocking (line=1865, file=0x7ffff7e209cb "ractor.c", remained_thread_cnt=<optimized out>, cr=0x55555555deb0) at ractor.c:1811 #10 rb_ractor_blocking_threads_inc (cr=0x55555555deb0, file=file@entry=0x7ffff7e29b04 "thread.c", line=line@entry=1488) at ractor.c:1865 #11 0x00007ffff7c60977 in blocking_region_begin (th=0x55555555d070, region=0x7fffffffd220, ubf=0x7ffff7c5b300 <ubf_select>, arg=0x55555555d070, fail_if_interrupted=<optimized out>) at thread.c:1488 #12 0x00007ffff7c61edf in rb_nogvl (func=0x7ffff7ac66e0 <no_gvl_stat>, data1=0x7fffffffd2a0, ubf=<optimized out>, data2=0x55555555d070, flags=<optimized out>) at thread.c:1548 #13 0x00007ffff7acdfbc in stat_without_gvl (st=0x7fffffffd2b0, path=0x7ffff27dfe88 "ruby-3.2.1/lib/ruby/3.2.0/x86_64-linux/enc/iso_8859_2.so") at file.c:1171 #14 rb_check_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, origenc=0x555555565030, mode=mode@entry=RB_REALPATH_STRICT) at file.c:4491 #15 0x00007ffff7acebaa in rb_realpath_internal (basedir=basedir@entry=4, path=path@entry=140737261984200, strict=strict@entry=1) at file.c:4536 #16 0x00007ffff7b2cdd4 in require_internal (ec=0x55555555e1f0, fname=<optimized out>, exception=1, warn=<optimized out>) at load.c:1195 #17 0x00007ffff7aa0a17 in load_encoding (name=0x5555556049b0 "ISO-8859-2") at encoding.c:779 #18 0x00007ffff7aa2ef8 in check_encoding (enc=0x5555556049d0) at encoding.c:191 #19 enc_check_encoding (obj=140737335533800) at encoding.c:202 #20 rb_to_encoding (enc=enc@entry=140737335533800) at encoding.c:307 #21 0x00007ffff7aa4049 in enc_set_default_encoding (name=0x7ffff7de6287 "external", encoding=140737335533800, def=0x7ffff7fb00c0 <default_external>) at encoding.c:1563 #22 rb_enc_set_default_external (encoding=140737335533800) at encoding.c:1637 #23 0x00007ffff7aa411f in set_default_external (klass=<optimized out>, encoding=140737335533800) at encoding.c:1658 #24 0x00007ffff7c99037 in vm_call_cfunc_with_frame (ec=0x55555555e1f0, reg_cfp=0x7ffff6f7df90, calling=<optimized out>) at ruby-3.2.1/vm_insnhelper.c:3268 #25 0x00007ffff7ca9564 in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at ruby-3.2.1/vm_callinfo.h:367 #26 vm_exec_core (ec=0x55555555e1f0, initial=128) at ruby-3.2.1/insns.def:820 #27 0x00007ffff7caee93 in rb_vm_exec (ec=0x55555555e1f0, jit_enable_p=true) at vm.c:2374 #28 0x00007ffff7abfbf1 in rb_ec_exec_node (ec=ec@entry=0x55555555e1f0, n=n@entry=0x7ffff796c2c0) at eval.c:289 #29 0x00007ffff7ac60b3 in ruby_run_node (n=0x7ffff796c2c0) at eval.c:330 #30 0x0000555555555187 in rb_main (argv=0x7fffffffdb98, argc=3) at ./main.c:38 #31 main (argc=<optimized out>, argv=<optimized out>) at ./main.c:57 ``` When the encoding was already loaded before running `Ractor.new`, then it works and doesn't block: ``` ruby -e "''.encode(Encoding::ISO8859_2); Ractor.new{}.take; Encoding.default_internal = Encoding::ISO8859_2" ``` When the encoding is not assigned as default encoding, but used otherwise, it works too: ``` ruby -e "Ractor.new{}.take; ''.encode(Encoding::ISO8859_2)" ``` -- https://bugs.ruby-lang.org/
participants (3)
-
hsbt (Hiroshi SHIBATA)
-
larskanis (Lars Kanis)
-
luke-gru (Luke Gruber)