March 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:113024] [Ruby master Bug#19549] Inconsistency in backref parsing

by hurricup (Alexandr Evstigneev)

Issue #19549 has been reported by hurricup (Alexandr Evstigneev). ---------------------------------------- Bug #19549: Inconsistency in backref parsing https://bugs.ruby-lang.org/issues/19549 * Author: hurricup (Alexandr Evstigneev) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This is fine: ``` puts "test #$01234 value" ``` This is not: ``` puts $01234 ``` -- https://bugs.ruby-lang.org/

1 year, 1 month

2
1
0 0

[ruby-core:113039] [Ruby master Bug#19557] Deadlock on STDOUT(ERR) lock on signal handler

by ko1 (Koichi Sasada)

Issue #19557 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #19557: Deadlock on STDOUT(ERR) lock on signal handler https://bugs.ruby-lang.org/issues/19557 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following Ruby code produces `deadlock; recursive locking (ThreadError)`. It means some IO operations (`puts`, ...) is not used on trap handlers safely. ```ruby trap(:USR1){puts 'world'} Thread.new{loop{Process.kill(:USR1, $$); sleep 0.5}} loop{puts 'hello'} ``` ``` ... hello hello hello hello hello ../../src/clean/test.rb:2:in `write': deadlock; recursive locking (ThreadError) from ../../src/clean/test.rb:2:in `puts' from ../../src/clean/test.rb:2:in `puts' from ../../src/clean/test.rb:2:in `block in <main>' from ../../src/clean/test.rb:4:in `write' from ../../src/clean/test.rb:4:in `puts' from ../../src/clean/test.rb:4:in `puts' from ../../src/clean/test.rb:4:in `block in <main>' from <internal:kernel>:187:in `loop' from ../../src/clean/test.rb:4:in `<main>' ``` ## Reason * `puts()` calls `rb_io_writev()` * it calls `IO#write` * it calls `io_write_m()` * it calls `io_writev()` * it calls `io_fwritev()` * it calls `io_binwritev()` * it calls `rb_mutex_synchronize()` with `io_binwritev_internal()` * STDOUT's `fptr->write_lock` is acquired here * `io_binwritev_internal()` calls `rb_writev_internal()` * it calls `rb_thread_io_blocking_region()` with `internal_writev_func()` Here, `internal_writev_func()` can be interrupted by signals and if a trap handler is registered, call the trap handler (written in Ruby, `puts 'world'` in this case) and call `Kernel#puts()` and `fptr->write_lock` is already acquired -> `deadlock; recursive locking`. ## Ideas I'm not sure why `fptr->write_lock` is needed, but if there is no internal consistency issue, we can make `fptr->write_lock` `nil` at least for STDOUT/ERR. Another idea is, calling trap handlers (or other interruptible Ruby code such as finalizers and so on) after releasing the lock. But I'm not sure it is feasible. -- https://bugs.ruby-lang.org/

1 year, 1 month

1
0
0 0

[ruby-core:113035] [Ruby master Bug#19556] Backport latest versions of URI

by hsbt (Hiroshi SHIBATA)

Issue #19556 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19556: Backport latest versions of URI https://bugs.ruby-lang.org/issues/19556 * Author: hsbt (Hiroshi SHIBATA) * Status: Closed * Priority: Normal * Backport: 2.7: DONE, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONE ---------------------------------------- https://www.ruby-lang.org/en/news/2023/03/28/redos-in-uri-cve-2023-28755/ has been published. We should upgrade URI version each stable branch. * [for Ruby 3.0](https://github.com/ruby/ruby/pull/7607) * [for Ruby 3.1](https://github.com/ruby/ruby/pull/7605) -- https://bugs.ruby-lang.org/

1 year, 1 month

2
1
0 0

[ruby-core:113031] [Ruby master Feature#19553] Document thread-safety of OpenSSL related classes such as OpenSSL::PKey::RSA or OpenSSL::Cipher etc

by rupert (Robert Pankowecki)

Issue #19553 has been reported by rupert (Robert Pankowecki). ---------------------------------------- Feature #19553: Document thread-safety of OpenSSL related classes such as OpenSSL::PKey::RSA or OpenSSL::Cipher etc https://bugs.ruby-lang.org/issues/19553 * Author: rupert (Robert Pankowecki) * Status: Open * Priority: Normal ---------------------------------------- From the ruby documentation it is not clear if these objects can be used (i.e. when assigned to a constant) by multiple threads. Especially for the respective encrypting/decrypting method after those objects are fully initialized with keys etc. If you look at what OpenSSL::PKey::RSA#private_encrypt does, it uses RSA_private_encrypt method from openssl: https://www.openssl.org/docs/man1.1.1/man3/RSA_private_encrypt.html . Which also is not documented in relation to thread-safety. The only thing that I found so far about this subject is: > OpenSSL can generally be used safely in multi-threaded applications provided that at least two callback functions are set, the locking_function and threadid_func. Note that OpenSSL is not completely thread-safe, and unfortunately not all global resources have the necessary locks. Further, the thread-safety does not extend to things like multiple threads using the same SSL object at the same time. from https://www.openssl.org/docs/man1.0.2/man3/threads.html which on itself does not help much because: * it is vague (not clear what's the meaning of both "SSL object" and "at the same time") * I do not know if Ruby provides locking_function and threadid_func, perhaps it does https://github.com/ruby/ruby/blob/99d0a257af54aa819c6ced5f1ff8ff37e3d5342b/… ? * I am not sure what's the impact of GIL on all this Looking at how Rails uses OpenSSL related objects, it seems that creating a new instance every time is the recommended approach. Can you confirm? -- https://bugs.ruby-lang.org/

1 year, 1 month

1
0
0 0

[ruby-core:112652] [Ruby master Bug#19471] Regexp::compile does not handle :timeout argument

by os (Shigeki OHARA)

Issue #19471 has been reported by os (Shigeki OHARA). ---------------------------------------- Bug #19471: Regexp::compile does not handle :timeout argument https://bugs.ruby-lang.org/issues/19471 * Author: os (Shigeki OHARA) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-freebsd13.1] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I think Regexp::compile is just an alias for Regexp::new, so I assume it handles the :timeout argument. However, Regexp::compile does not seem to handle the :timeout argument properly. ``` $ irb irb(main):001:0> RUBY_VERSION => "3.2.1" irb(main):002:0> re_new = Regexp.new('hoge', timeout: 1.0) => /hoge/ irb(main):003:0> re_new.timeout => 1.0 irb(main):004:0> re_compile = Regexp.compile('hoge', timeout: 1.0) => /hoge/i irb(main):005:0> re_compile.timeout => nil irb(main):006:0> ``` -- https://bugs.ruby-lang.org/

1 year, 1 month

2
1
0 0

[ruby-core:113021] [Ruby master Bug#18464] RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors

by ivoanjo (Ivo Anjo)

Issue #18464 has been updated by ivoanjo (Ivo Anjo). Thank you @nagachika! :) :) :) ---------------------------------------- Bug #18464: RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors https://bugs.ruby-lang.org/issues/18464#change-102557 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] * Backport: 2.6: DONTNEED, 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: REQUIRED ---------------------------------------- When a Ractor is created whilst a tracepoint for `RUBY_INTERNAL_EVENT_NEWOBJ` is active (registered with `rb_tracepoint_new`/`rb_tracepoint_enabled`), the interpreter crashes with a null pointer dereference with the following backtrace: ``` [BUG] Segmentation fault at 0x0000000000000000 ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] ... -- C level backtrace information ------------------------------------------- /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_print_backtrace+0xf) [0x10a15fadd] vm_dump.c:759 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) vm_dump.c:1045 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) (null):0 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(bug_report_end+0x0) [0x109f96b81] error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_bug_for_fatal_signal) error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(sigsegv+0x52) [0x10a0be3a2] signal.c:964 /usr/lib/system/libsystem_platform.dylib(_sigtramp+0x1d) [0x7fff20934d7d] /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(gc_event_hook_body+0x4) [0x109fb9d21] gc.c:2214 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath) gc.c:2486 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath_wb_unprotected) gc.c:2507 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_fill+0x0) [0x109fac92e] gc.c:2543 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of0) gc.c:2553 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of) gc.c:2552 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_wb_unprotected_newobj_of) gc.c:2567 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(io_alloc+0x12) [0x109fd341c] io.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_io) io.c:8483 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_stdio) io.c:8514 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_io_prep_stdin) io.c:8532 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_2+0xf7) [0x10a1058a7] thread.c:802 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_native_cond_initialize+0x0) [0x10a1055fb] ./thread_pthread.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(register_cached_thread_and_wait) ./thread_pthread.c:1099 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_1) ./thread_pthread.c:1054 /usr/lib/system/libsystem_pthread.dylib(_pthread_start+0xe0) [0x7fff208ef8fc] ``` (full output is attached). This seems to be because the new Ractor sets up stdio objects (`rb_io_prep_stdin` et. al.), which in turn allocate Ruby objects, before `rb_ec_initialize_vm_stack` is called to set up the initial stack frame. I've attached a patch which works around this by not firing GC event hooks if there is no control frame on the execution context. The patch also includes a test which reproduces the issue using the `objspace` extension; creating a Ractor within an `ObjectSpace.trace_object_allocations` block is enough to trigger the crash. The patch seems to fix things, but if you folk prefer I can also try swapping around the order of `prep_stdio` and `rb_ec_initialize_vm_stack`. ---Files-------------------------------- 0001-Fix-interpreter-crash-caused-by-RUBY_INTERNAL_EVENT_.patch (1.91 KB) crash.log (26.1 KB) ruby_2022-01-08-151326_8927-ktsanaktsidis.crash (18.8 KB) -- https://bugs.ruby-lang.org/

1 year, 1 month

1
0
0 0

[ruby-core:112218] [Ruby master Bug#19414] uninitialized constant URI::WSS in 3.0.X and 3.1.X

by noraj (Alexandre ZANNI)

Issue #19414 has been reported by noraj (Alexandre ZANNI). ---------------------------------------- Bug #19414: uninitialized constant URI::WSS in 3.0.X and 3.1.X https://bugs.ruby-lang.org/issues/19414 * Author: noraj (Alexandre ZANNI) * Status: Open * Priority: Normal * ruby -v: 3.1.3 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I have a library called [ctf-party](https://github.com/noraj/ctf-party), [making use of URI:WSS](https://github.com/noraj/ctf-party/blob/ef3b82cc1668325b782246a9bb…, and it's making my [CI pipeline fails](https://github.com/noraj/ctf-party/actions/runs/4059895670). I'm able to reproduce it locally (`bundle exec rake test`), working with ruby 3.2.0 and not with 3.1.3. The error is: ``` irb(main):001:0> require 'uri' => true irb(main):002:0> URI::WS => URI::WS irb(main):003:0> URI::WSS (irb):3:in `<main>': uninitialized constant URI::WSS (NameError) from /home/noraj/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>' from /home/noraj/.asdf/installs/ruby/3.1.3/bin/irb:25:in `load' from /home/noraj/.asdf/installs/ruby/3.1.3/bin/irb:25:in `<main>' ``` I don't understand why is that happening since [it exists in the official documentation](https://ruby-doc.org/3.1.3/stdlibs/uri/URI/WSS.html) and the [code looks ok to me on tag 3.1.3](https://github.com/ruby/ruby/blob/v3_1_3/lib/uri/wss.rb) which is [strictly identical to the code on tag 3.2.0](https://github.com/ruby/ruby/blob/v3_2_0/lib/uri/wss.rb). It works if I require the file explicitly: ``` irb(main):004:0> require 'uri/wss' => true irb(main):005:0> URI::WSS => URI::WSS ``` The issue come from the fact that [in 3.2.0 `'uri/wss'` is required](https://github.com/ruby/ruby/blob/v3_2_0/lib/uri.rb#L104) but [not in 3.1.3](https://github.com/ruby/ruby/blob/v3_1_3/lib/uri.rb#L103). In [3.0.5](https://github.com/ruby/ruby/blob/v3_0_5/lib/uri.rb) it is even worse as neither ws and wss are required. -- https://bugs.ruby-lang.org/

1 year, 1 month

5
6
0 0

[ruby-core:112433] [Ruby master Bug#19439] Marshal.load doesn't load Regexp instance variables

by andrykonchin (Andrew Konchin)

Issue #19439 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19439: Marshal.load doesn't load Regexp instance variables https://bugs.ruby-lang.org/issues/19439 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * ruby -v: 3.1.3 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello, I've noticed this strange behaviour: ```ruby source_object = Regexp.new("a") source_object.instance_variable_set(:@foo, :bar) regexp = Marshal.load(Marshal.dump(source_object)) regexp.instance_variables # => [] ``` I would expect instance variables to be loaded -- https://bugs.ruby-lang.org/

1 year, 1 month

5
7
0 0

[ruby-core:112744] [Ruby master Bug#19485] Unexpected behavior in squiggly heredocs

by jemmai (Jemma Issroff)

Issue #19485 has been reported by jemmai (Jemma Issroff). ---------------------------------------- Bug #19485: Unexpected behavior in squiggly heredocs https://bugs.ruby-lang.org/issues/19485 * Author: jemmai (Jemma Issroff) * Status: Open * Priority: Normal * ruby -v: 3.2.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Based on [the squiggly heredoc documentation](https://ruby-doc.org/3.2.1/syntax/literals_rdoc.html), I found the following to be unexpected behavior. Explicitly, the documentation specifies, "The indentation of the least-indented line will be removed from each line of the content." After running: ```ruby File.write("test.rb", "p <<~EOF\n\ta\n b\nEOF\n") ``` and then `ruby test.rb`, I get the following output: ``` "\ta\nb\n" ``` The least-indented line above is ` b`, however, no leading whitespace is removed from the line containing `\ta`. For another example: ```ruby File.write("test.rb", "p <<~EOF\n\tA\n \tB\nEOF\n") ``` `ruby test.rb` gives: ``` "A\nB\n" ``` In this case, the `\t` was removed from the line containing `A`, but more whitespace than that (` \t`) was removed from the line containing `B`. After seeing the first example, I assumed that the documentation was out of date, and that I should fix it to read that `\t` would never be converted into space characters in order to remove leading whitespace. But after the second example, it seems like this is a bug in removing leading whitespace. Can someone please explain what the rules should be on squiggly heredocs? I can implement a fix to adhere to the rules, or can update the documentation, I am just unsure of what the rules should be because the above two examples reflect unexpected behavior in two distinct ways. -- https://bugs.ruby-lang.org/

1 year, 1 month

8
8
0 0

[ruby-core:113014] [Ruby master Bug#18464] RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors

by nagachika (Tomoyuki Chikanaga)

Issue #18464 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 2.6: DONTNEED, 2.7: DONTNEED, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED to 2.6: DONTNEED, 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: REQUIRED ruby_3_1 bdbe6053853c11ffe9b8737eb4da50ed84c9dbd6 merged revision(s) 7bd7aee02e303de27d2cddfc5ef47e612d6782cb. ---------------------------------------- Bug #18464: RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors https://bugs.ruby-lang.org/issues/18464#change-102549 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] * Backport: 2.6: DONTNEED, 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: REQUIRED ---------------------------------------- When a Ractor is created whilst a tracepoint for `RUBY_INTERNAL_EVENT_NEWOBJ` is active (registered with `rb_tracepoint_new`/`rb_tracepoint_enabled`), the interpreter crashes with a null pointer dereference with the following backtrace: ``` [BUG] Segmentation fault at 0x0000000000000000 ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] ... -- C level backtrace information ------------------------------------------- /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_print_backtrace+0xf) [0x10a15fadd] vm_dump.c:759 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) vm_dump.c:1045 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) (null):0 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(bug_report_end+0x0) [0x109f96b81] error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_bug_for_fatal_signal) error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(sigsegv+0x52) [0x10a0be3a2] signal.c:964 /usr/lib/system/libsystem_platform.dylib(_sigtramp+0x1d) [0x7fff20934d7d] /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(gc_event_hook_body+0x4) [0x109fb9d21] gc.c:2214 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath) gc.c:2486 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath_wb_unprotected) gc.c:2507 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_fill+0x0) [0x109fac92e] gc.c:2543 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of0) gc.c:2553 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of) gc.c:2552 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_wb_unprotected_newobj_of) gc.c:2567 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(io_alloc+0x12) [0x109fd341c] io.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_io) io.c:8483 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_stdio) io.c:8514 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_io_prep_stdin) io.c:8532 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_2+0xf7) [0x10a1058a7] thread.c:802 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_native_cond_initialize+0x0) [0x10a1055fb] ./thread_pthread.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(register_cached_thread_and_wait) ./thread_pthread.c:1099 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_1) ./thread_pthread.c:1054 /usr/lib/system/libsystem_pthread.dylib(_pthread_start+0xe0) [0x7fff208ef8fc] ``` (full output is attached). This seems to be because the new Ractor sets up stdio objects (`rb_io_prep_stdin` et. al.), which in turn allocate Ruby objects, before `rb_ec_initialize_vm_stack` is called to set up the initial stack frame. I've attached a patch which works around this by not firing GC event hooks if there is no control frame on the execution context. The patch also includes a test which reproduces the issue using the `objspace` extension; creating a Ractor within an `ObjectSpace.trace_object_allocations` block is enough to trigger the crash. The patch seems to fix things, but if you folk prefer I can also try swapping around the order of `prep_stdio` and `rb_ec_initialize_vm_stack`. ---Files-------------------------------- 0001-Fix-interpreter-crash-caused-by-RUBY_INTERNAL_EVENT_.patch (1.91 KB) crash.log (26.1 KB) ruby_2022-01-08-151326_8927-ktsanaktsidis.crash (18.8 KB) -- https://bugs.ruby-lang.org/

1 year, 1 month

1
0
0 0

2024

2023

2022

ruby-core March 2023