December 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:115459] [Ruby master Bug#20017] 3.3.0dev `rb_thread_profile_frames` crashes when `RUBY_MN_THREADS=1`
by byroot (Jean Boussier) 20 Dec '23

20 Dec '23

Issue #20017 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20017: 3.3.0dev `rb_thread_profile_frames` crashes when `RUBY_MN_THREADS=1` https://bugs.ruby-lang.org/issues/20017 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.3.0dev (2023-11-19T03:01:05Z shopify 9aee12cc28) +MN [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I discovered this while running our internal CI with MaNy enabled, our application crash when trying to profile with StackProf (with postponed jobs disabled): ``` [BUG] Segmentation fault at 0x0000000000000010 ruby 3.3.0dev (2023-11-19T03:01:05Z shopify 9aee12cc28) +MN [x86_64-linux] -- C level backtrace information ------------------------------------------- /usr/local/ruby/bin/ruby(rb_print_backtrace+0x14) [0x55c8f16333e1] /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/vm_dump.c:812 /usr/local/ruby/bin/ruby(rb_vm_bugreport) /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/vm_dump.c:1143 /usr/local/ruby/bin/ruby(rb_bug_for_fatal_signal+0xfc) [0x55c8f17e559c] /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/error.c:1065 /usr/local/ruby/bin/ruby(sigsegv+0x4d) [0x55c8f158091d] /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/signal.c:920 /lib/x86_64-linux-gnu/libc.so.6(0x7ff3db333520) [0x7ff3db333520] /usr/local/ruby/bin/ruby(thread_profile_frames+0x10) [0x55c8f162efd0] /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/vm_backtrace.c:1587 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_buffer_sample+0x28) [0x7ff3b7d2435c] /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:622 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_buffer_sample) /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:604 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_buffer_sample) (null):0 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_signal_handler+0x5) [0x7ff3b7d24545] /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:767 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_signal_handler) /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:722 /lib/x86_64-linux-gnu/libc.so.6(0x7ff3db333520) [0x7ff3db333520] /lib/x86_64-linux-gnu/libc.so.6(pthread_cond_wait+0x24a) [0x7ff3db384a7a] /usr/local/ruby/bin/ruby(rb_native_cond_wait+0xb) [0x55c8f15c8cbb] /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/thread_pthread.c:214 /usr/local/ruby/bin/ruby(ractor_sched_deq) /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/thread_pthread.c:1230 /usr/local/ruby/bin/ruby(nt_start) /tmp/ruby-build/ruby-3.3.0-9aee12cc28cbca40306784e54e38558688caa9f7/thread_pthread.c:2209 ``` For additional context, as far as I know `rb_thread_profile_frames` isn't officially async signal safe, but it happens to be since 3.0 for the interpreter, and 3.3 for YJIT, so StackProf tries not to use postponed jobs for profiling when it can as it leads to much more accurate profiling results. Ref: https://github.com/tmm1/stackprof/issues/221 -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115458] [Ruby master Bug#20016] 3.3.0dev `rb_postponed_job_register_one` crashes when `RUBY_MN_THREADS=1`
by byroot (Jean Boussier) 20 Dec '23

20 Dec '23

Issue #20016 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20016: 3.3.0dev `rb_postponed_job_register_one` crashes when `RUBY_MN_THREADS=1` https://bugs.ruby-lang.org/issues/20016 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.3.0dev (2023-11-22T17:01:13Z shopify c1fc1a00ea) +MN [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I discovered this while running our internal CI with MaNy enabled, our application crash when trying to profile with StackProf: ``` [BUG] Segmentation fault at 0x0000000000000020 ruby 3.3.0dev (2023-11-22T17:01:13Z shopify c1fc1a00ea) +MN [x86_64-linux] -- Machine register context ------------------------------------------------ RIP: 0x000055df5fe38489 RBP: 0x00007f517bc59000 RSP: 0x00007f5123f3c6d0 RAX: 0x0000000000000000 RBX: 0x00007f51596554e0 RCX: 0x00007f517bdb6b40 RDX: 0x0000000000000001 RDI: 0x00007f517bc59000 RSI: 0x0000000000000000 R8: 0x0000000000000000 R9: 0x00000000ffffffff R10: 0x0000000000000000 R11: 0x0000000000000246 R12: 0x0000000000000000 R13: 0x0000000000000000 R14: 0x0000000000001ea5 R15: 0x00007f517bc59104 EFL: 0x0000000000010202 -- C level backtrace information ------------------------------------------- /usr/local/ruby/bin/ruby(rb_print_backtrace+0x14) [0x55df5fe328e1] /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_dump.c:812 /usr/local/ruby/bin/ruby(rb_vm_bugreport) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_dump.c:1143 /usr/local/ruby/bin/ruby(rb_bug_for_fatal_signal+0xfc) [0x55df5ffe509c] /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/error.c:1065 /usr/local/ruby/bin/ruby(sigsegv+0x4d) [0x55df5fd7f19d] /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/signal.c:920 /lib/x86_64-linux-gnu/libc.so.6(0x7f517c277520) [0x7f517c277520] /usr/local/ruby/bin/ruby(rbimpl_atomic_or+0x0) [0x55df5fe38489] /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_trace.c:1691 /usr/local/ruby/bin/ruby(postponed_job_register) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_trace.c:1693 /usr/local/ruby/bin/ruby(postponed_job_register) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_trace.c:1675 /usr/local/ruby/bin/ruby(rb_postponed_job_register_one) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/vm_trace.c:1746 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_signal_handler+0x2d) [0x7f5159655434] /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:763 /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/lib/stackprof/stackprof.so(stackprof_signal_handler) /tmp/bundle/ruby/3.3.0+0/gems/stackprof-0.2.25/ext/stackprof/stackprof.c:722 /lib/x86_64-linux-gnu/libc.so.6(0x7f517c277520) [0x7f517c277520] /lib/x86_64-linux-gnu/libc.so.6(0x7f517c2c6117) [0x7f517c2c6117] /lib/x86_64-linux-gnu/libc.so.6(pthread_cond_wait+0x211) [0x7f517c2c8a41] /usr/local/ruby/bin/ruby(rb_native_cond_wait+0xb) [0x55df5fdc75fb] /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/thread_pthread.c:214 /usr/local/ruby/bin/ruby(ractor_sched_deq) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/thread_pthread.c:1230 /usr/local/ruby/bin/ruby(nt_start) /tmp/ruby-build/ruby-3.3.0-c1fc1a00ea9633961153451d0e927db49c1b268d/thread_pthread.c:2209 /lib/x86_64-linux-gnu/libc.so.6(0x7f517c2c9ac3) [0x7f517c2c9ac3] ``` Ref: https://github.com/tmm1/stackprof/issues/221 -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115648] [Ruby master Feature#20049] Destructive drop_while for Array and Hash
by chucke (Tiago Cardoso) 20 Dec '23

20 Dec '23

Issue #20049 has been reported by chucke (Tiago Cardoso). ---------------------------------------- Feature #20049: Destructive drop_while for Array and Hash https://bugs.ruby-lang.org/issues/20049 * Author: chucke (Tiago Cardoso) * Status: Open * Priority: Normal ---------------------------------------- I propose a "drop_while!" variant for arrays and hashes, which changes the current instance. ```ruby h = {foo: 0, bar: 1, baz: 2} h.drop_while!{|element| key, value = *element; value < 2 } h #=> # => { baz: 2 } ``` -- https://bugs.ruby-lang.org/

4 7

[ruby-core:112262] [Ruby master Feature#19422] Make `--enabled-shared` mandatory on macOS
by nobu (Nobuyoshi Nakada) 20 Dec '23

20 Dec '23

Issue #19422 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #19422: Make `--enabled-shared` mandatory on macOS https://bugs.ruby-lang.org/issues/19422 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal ---------------------------------------- From the troubles around linker on macOS, I propose `--enable-shared` option mandatory on macOS. This patch enables the option by default, and abort if `--disable-shared` option is given explicitly. ```diff diff --git i/configure.ac w/configure.ac index 7db2ab5257c..923ac7d1199 100644 --- i/configure.ac +++ w/configure.ac @@ -504,6 +504,11 @@ AS_CASE(["$target_os"], rb_cv_binary_elf=no : ${enable_shared=yes} ], +[darwin*], [ + AS_IF([test "${enable_shared=yes}" = no], [ + AC_MSG_ERROR([--disable-shared is not supported on this platform]) + ]) +], [hiuxmpp*], [AC_DEFINE(__HIUX_MPP__)]) # by TOYODA Eizi <toyoda(a)npd.kishou.go.jp> AC_PROG_LN_S @@ -3055,14 +3060,7 @@ AC_SUBST(EXTOBJS) : ${LDFLAGS=""} : ${LIBPATHENV=DYLD_FALLBACK_LIBRARY_PATH} : ${PRELOADENV=DYLD_INSERT_LIBRARIES} - AS_IF([test x"$enable_shared" = xyes], [ - # Resolve symbols from libruby.dylib when --enable-shared - EXTDLDFLAGS='$(LIBRUBYARG_SHARED)' - ], [test "x$EXTSTATIC" = x], [ - # When building exts as bundles, a mach-o bundle needs to know its loader - # program to bind symbols from the ruby executable - EXTDLDFLAGS="-bundle_loader '\$(BUILTRUBY)'" - ]) + EXTDLDFLAGS='$(LIBRUBYARG_SHARED)' rb_cv_dlopen=yes], [aix*], [ : ${LDSHARED='$(CC)'} AS_IF([test "$GCC" = yes], [ @@ -3356,10 +3354,6 @@ AS_IF([test x"$cross_compiling" = xyes], [ AC_SUBST(XRUBY_RUBYLIBDIR) AC_SUBST(XRUBY_RUBYHDRDIR) PREP='$(arch)-fake.rb' - AS_CASE(["$enable_shared:$EXTSTATIC:$target_os"], [no::darwin*], [ - # darwin target requires miniruby for linking ext bundles - PREP="$PREP"' miniruby$(EXEEXT)' - ]) RUNRUBY_COMMAND='$(MINIRUBY) -I`cd $(srcdir)/lib; pwd`' RUNRUBY='$(RUNRUBY_COMMAND)' XRUBY='$(MINIRUBY)' ``` -- https://bugs.ruby-lang.org/

4 7

[ruby-core:115667] [Ruby master Bug#20052] reline behaves oddly with redirect (non tty)
by mtasaka (Mamoru Tasaka) 20 Dec '23

20 Dec '23

Issue #20052 has been reported by mtasaka (Mamoru Tasaka). ---------------------------------------- Bug #20052: reline behaves oddly with redirect (non tty) https://bugs.ruby-lang.org/issues/20052 * Author: mtasaka (Mamoru Tasaka) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-09 master 1cbe114d1c) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Forwarded from cucumber/aruba testsute issue https://github.com/cucumber/aruba/issues/910 Looks line reline behaves oddly with redirect (non tty) With `ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]` and `Readline`: ``` $ echo 'foo' | ruby -e 'require "readline"; puts Readline.readline(">")' > bar.log ; cat bar.log >foo foo ``` With `ruby 3.3.0dev (2023-12-09 master 1cbe114d1c) [x86_64-linux]` and `Reline`: ``` $ echo 'foo' | ruby -e 'require "reline"; puts Reline.readline(">")' > bar.log ; cat bar.log >>>f>f>fo>fo>foo>foofoo ``` -- https://bugs.ruby-lang.org/

5 5

[ruby-core:115669] [Ruby master Feature#20053] M:N Threads, now w/ macOS support (kqueue)
by jpcamara (JP Camara) 20 Dec '23

20 Dec '23

Issue #20053 has been reported by jpcamara (JP Camara). ---------------------------------------- Feature #20053: M:N Threads, now w/ macOS support (kqueue) https://bugs.ruby-lang.org/issues/20053 * Author: jpcamara (JP Camara) * Status: Open * Priority: Normal ---------------------------------------- I have a PR here that adds support for macOS to M:N threads using `kqueue`: https://github.com/ruby/ruby/pull/9178 It adds in `kqueue`/`kevent` calls when present in the OS. Technically this would open up support for FreeBSD as well, but I don't have a way of testing that so i'm not sure how well it does or doesn't work there. I wanted to get support going for macOS so more devs can try out M:N threads and test it. I do think there should be a larger discussion around the potential relationship between this and some of the awesome fiber scheduler work over the last few years (and the potential of utilizing anything from io-event?). But that can be a topic for another day - let's get macOS support in the meantime! Disclaimer: C isn't my day-to-day language, so I could definitely use feedback there. I'm also more of a kernel event queue (kqueue, epoll, io_uring) _enthusiast_, but `kqueue` isn't something I have specific experience writing with - just lots of reading code and small toy code up until this point. So someone like @ioquatix would probably be a good person to get eyes on it. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:115814] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by Quintasan 20 Dec '23

20 Dec '23

Issue #18915 has been updated by Quintasan (Michał Zając). matz (Yukihiro Matsumoto) wrote in #note-14: > #note-13 explains my opinion well. What name candidate do you have? > > Matz. How about `AbstractMethodError` - the same as Java? ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-105763 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (1.57 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115813] [Ruby master Feature#18576] Rename `ASCII-8BIT` encoding to `BINARY`
by naruse (Yui NARUSE) 20 Dec '23

20 Dec '23

Issue #18576 has been updated by naruse (Yui NARUSE). I strongly object that we change Encoding#name of ASCII-8BIT encoding into "BINARY" because of compatibility. I don't want people to fix the code which are correctly running now. However supporting people who newly writing a code is reasonable. I agree to add more information in Encoding#inspect and error message. ---------------------------------------- Feature #18576: Rename `ASCII-8BIT` encoding to `BINARY` https://bugs.ruby-lang.org/issues/18576#change-105762 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ### Context I'm now used to it, but something that confused me for years was errors such as: ```ruby >> "fée" + "\xFF".b (irb):3:in `+': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) ``` When you aren't that familiar with Ruby, it's really not evident that `ASCII-8BIT` basically means "no encoding" or "binary". And even when you know it, if you don't read carefully it's very easily confused with `US-ASCII`. The `Encoding::BINARY` alias is much more telling IMHO. ### Proposal Since `Encoding::ASCII_8BIT` has been aliased as `Encoding::BINARY` for years, I think renaming it to `BINARY` and then making asking `ASCII_8BIT` the alias would significantly improve usability without backward compatibility concerns. The only concern I could see would be the consistency with a handful of C API functions: - `rb_encoding *rb_ascii8bit_encoding(void)` - `int rb_ascii8bit_encindex(void)` - `VALUE rb_io_ascii8bit_binmode(VALUE io)` But that's for much more advanced users, so I don't think it's much of a concern. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115764] [Ruby master Feature#20069] Buffer class in stdlib
by pynix (Pynix wang) 20 Dec '23

20 Dec '23

Issue #20069 has been reported by pynix (Pynix wang). ---------------------------------------- Feature #20069: Buffer class in stdlib https://bugs.ruby-lang.org/issues/20069 * Author: pynix (Pynix wang) * Status: Open * Priority: Normal ---------------------------------------- ruby use `String` to deal with bytes, this cause error on irb "invalid byte sequence in utf-8" can we get a builtin class like Buffer or Bytes that represent as hex string -- https://bugs.ruby-lang.org/

7 8

[ruby-core:115731] [Ruby master Bug#20063] Inconsistent behavior with required vs optional parameters
by jemmai (Jemma Issroff) 20 Dec '23

20 Dec '23

Issue #20063 has been reported by jemmai (Jemma Issroff). ---------------------------------------- Bug #20063: Inconsistent behavior with required vs optional parameters https://bugs.ruby-lang.org/issues/20063 * Author: jemmai (Jemma Issroff) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Using repeated anonymous parameters, gives different behavior with the parameters in the same order based on whether they are required or optional. ``` def example_with_optional(_a = 1, _a = 2) _a end def example_with_required(_a, _a) _a end p example_with_optional # 2 p example_with_required(1, 2) # 1 ``` It is unexpected that these two methods would return differently given the parameters are in the same order, based on whether they are optional or required. -- https://bugs.ruby-lang.org/

4 3