July 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118270] [Ruby master Bug#20571] Compile error on for loop
by kddnewton (Kevin Newton) 02 Sep '24

02 Sep '24

Issue #20571 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20571: Compile error on for loop https://bugs.ruby-lang.org/issues/20571 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby for _ in 1.5 or _ in [] end ``` raises a compile error for the current compiler only with ``` -- raw disasm-------- trace: 1 <L000> [sp: 0, unremovable: 1, refcnt: 1] 0000 putobject 1.5 ( 1) 0002 jump <L002> ( 1) <L006> [sp: 1, unremovable: 1, refcnt: 1] <L004> [sp: 1, unremovable: 0, refcnt: 0] 0004 pop ( 1) * 0005 pop ( 1) 0006 putobject false ( 1) <L002> [sp: 1, unremovable: 0, refcnt: 1] 0008 send <calldata:each, 0>, nil ( 1) <L001> [sp: -1, unremovable: 0, refcnt: 2] 0011 leave ( 1) --------------------- test.rb:1: argument stack underflow (-1) test.rb: compile error (SyntaxError) ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:118681] [Ruby master Bug#20651] `false && (1 in 1)` produces argument stack underflow (-1)
by tompng (tomoya ishida) 02 Sep '24

02 Sep '24

Issue #20651 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20651: `false && (1 in 1)` produces argument stack underflow (-1) https://bugs.ruby-lang.org/issues/20651 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-07-24T19:55:04Z master 30eba40f9c) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `false && (1 in 1)` and `true || (1 in 1)` produces argument stack underflow since ruby 3.0 ~~~ # ruby -v ruby 3.4.0dev (2024-07-24T19:55:04Z master 30eba40f9c) [x86_64-linux] # ruby -e "false && (1 in 1)" -- raw disasm-------- trace: 1 0000 putobject false ( 1) 0002 leave ( 1) <L004> [sp: 1, unremovable: 1, refcnt: 1] <L002> [sp: 1, unremovable: 0, refcnt: 0] 0003 pop ( 1) * 0004 pop ( 1) 0005 putobject false ( 1) 0007 leave ( 1) --------------------- -e:1: argument stack underflow (-1) -e: compile error (SyntaxError) ~~~ -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118686] [Ruby master Bug#20653] Memory leak in String#start_with? when regexp times out
by peterzhu2118 (Peter Zhu) 02 Sep '24

02 Sep '24

Issue #20653 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20653: Memory leak in String#start_with? when regexp times out https://bugs.ruby-lang.org/issues/20653 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11247 This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error. This fixes a memory leak in String#start_with when the regexp times out. For example: ```ruby regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do str.start_with?(regex) rescue end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 33216 51936 71152 81728 97152 103248 120384 133392 133520 133616 ``` After: ``` 14912 15376 15824 15824 16128 16128 16144 16144 16160 16160 ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118637] [Ruby master Bug#20641] `lib/bundled_gems.rb` makes `Kernel.require` over 100x slower
by byroot (Jean Boussier) 02 Sep '24

02 Sep '24

Issue #20641 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20641: `lib/bundled_gems.rb` makes `Kernel.require` over 100x slower https://bugs.ruby-lang.org/issues/20641 * Author: byroot (Jean Boussier) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- I just discovered this while profiling Active Record's test suite, and I noticed 40% of the runtime was in `$LOAD_PATH.resolve_feature_path`, so much I thought it was a profiler bug. But it turns out it's real. Various APIs do call `require` late, for instance Psych calls `require 'date'` every single time it parses a date: https://github.com/ruby/psych/blob/be0ba74e5613c20f213403e15914d24944c2652d… I've put together a quick benchmark: ```ruby # frozen_string_literal: true require "bundler/inline" gemfile(true) do source "https://rubygems.org" gem "benchmark-ips" end Benchmark.ips do |x| x.report(RUBY_VERSION) { require "erb" } x.save! "/tmp/require.bench" x.compare! end ``` And the difference is massive: ``` Calculating ------------------------------------- 3.2.2 6.450M (± 1.2%) i/s - 32.822M in 5.089538s Comparison: 3.2.2: 6449899.4 i/s 3.3.3: 46996.8 i/s - 137.24x slower ``` And that's with a small `$LOAD_PATH`, the bigger the application, the worse it is, if I add: ```ruby 100.times do |i| $LOAD_PATH.unshift("/tmp/empty-#{i}") end ``` ``` Calculating ------------------------------------- 3.3.3 6.198k (± 6.1%) i/s - 30.968k in 5.018955s Comparison: 3.2.2: 3380939.6 i/s 3.3.3: 6198.4 i/s - 545.46x slower ``` I'm looking at a way to speed this up, but this is big enough that I believe we should backport the fix. cc @hsbt -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114645] [Ruby master Bug#19865] Segfault when calling user signal handlers during VM shutdown
by ivoanjo (Ivo Anjo) 27 Aug '24

27 Aug '24

Issue #19865 has been reported by ivoanjo (Ivo Anjo). ---------------------------------------- Bug #19865: Segfault when calling user signal handlers during VM shutdown https://bugs.ruby-lang.org/issues/19865 * Author: ivoanjo (Ivo Anjo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Howdy 👋! I work for Datadog [on the `ddtrace` gem](https://github.com/DataDog/dd-trace-rb) . I've found this issue while investigating a [customer crash report](https://github.com/DataDog/dd-trace-rb/issues/2980). ### Background The original issue was found in a production app. A number of things need to be in play to cause it. The [`ruby-odbc`](https://rubygems.org/gems/ruby-odbc) gem provides a way of accessing databases through the ODBC API. It wraps a database connection as a Data object, with a free function that, prior to freeing the native resources, disconnects from the database if the connection was still active. Because disconnecting from the database is a blocking operation, the gem (reasonably, in my opinion), releases the global VM lock before disconnecting. The trigger for the crash is: 1. The app in question used puma, and puma installs a `Signal.trap('TERM')` 2. The database object was still connected when the app started to shut down 3. A VM shutdown starts... 4. Half-way through shutdown, the VM received a SIGTERM signal, and queued it for processing 5. The VM calls the free function on all objects 6. The ruby-odbc gem sees there's an active database connection, and tries to release the GVL to call the blocking disconnect 7. Before releasing the GVL, the VM checks for pending interruptions 8. The VM tries to run the Ruby-level signal handler method half-way through VM shutdown, when you can no longer run Ruby code 9. Segfault ### How to reproduce (Ruby version & script) I was able to reproduce on a minimal example on Ruby 3.2.2 (`ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]`) and recent master (`ruby 3.3.0dev (2023-08-17T07:30:01Z master d26b015e83) [x86_64-linux]`). I've put the test-case up on github as well <https://github.com/DataDog/signal-bug-testcase>, but here's the important bits: `signal-bug-testcase.rb`: ```ruby require 'signal_bug_testcase' Signal.trap("TERM") { puts "Hello, world" } FOO = SignalBugTestcase.new ``` `signal_bug_testcase.c`: ```ruby #include <ruby.h> #include <ruby/thread.h> #include <signal.h> #include <unistd.h> typedef struct { int dummy; } BugTestcase; void *test_nogvl(void *unused) { fprintf(stderr, "GVL released!\n"); return NULL; } static void bug_testcase_free(void* ptr) { fprintf(stderr, "Free getting called! Sending signal...\n"); kill(getpid(), SIGTERM); fprintf(stderr, "SIGTERM signal queued, trying to release GVL...\n"); rb_thread_call_without_gvl(test_nogvl, NULL, NULL, NULL); fprintf(stderr, "After releasing GVL!\n"); free(ptr); } const rb_data_type_t bug_testcase_data_type = { .wrap_struct_name = "SignalBugTestcase", .function = { NULL, bug_testcase_free, NULL }, .flags = RUBY_TYPED_FREE_IMMEDIATELY }; VALUE bug_testcase_alloc(VALUE klass) { BugTestcase* obj = calloc(1, sizeof(BugTestcase)); return TypedData_Make_Struct(klass, BugTestcase, &bug_testcase_data_type, obj); } void Init_signal_bug_testcase(void) { VALUE cBugTestcase = rb_define_class("SignalBugTestcase", rb_cObject); rb_define_alloc_func(cBugTestcase, bug_testcase_alloc); } ``` ### Expectation and result No segfault happens. Interestingly, on Ruby 2.7, the VM exits half-way through but doesn't always segfault, but running it a few times always triggers the issue. On 3.2+ it crashes every time for me. I suspect the right thing here is to no longer accept/try to run any Ruby-level signal handlers after VM shutdown starts. Here's what I see with this test-case: ``` $ bundle exec ruby lib/signal-bug-testcase.rb Free getting called! Sending signal... SIGTERM signal queued, trying to release GVL... lib/signal-bug-testcase.rb:3: [BUG] Segmentation fault at 0x0000000000000007 ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0003 p:---- s:0011 e:000010 CFUNC :puts c:0002 p:0005 s:0006 e:000005 BLOCK lib/signal-bug-testcase.rb:3 [FINISH] c:0001 p:0000 s:0003 E:0001a0 DUMMY [FINISH] -- Ruby level backtrace information ---------------------------------------- lib/signal-bug-testcase.rb:3:in `block in <main>' lib/signal-bug-testcase.rb:3:in `puts' -- Machine register context ------------------------------------------------ RIP: 0x000070aa64cedbe7 RBP: 0x000070aa648e8fd0 RSP: 0x00007ffc5c057608 RAX: 0x0000000000004171 RBX: 0x00007ffc5c057630 RCX: 0x0000000000000001 RDX: 0x00007ffc5c057630 RDI: 0x0000000000000007 RSI: 0x0000000000004171 R8: 0x000000000000021b R9: 0x0000000000000000 R10: 0x000070aa63eff048 R11: 0x0000000000000000 R12: 0x000070aa648e8fd0 R13: 0x0000000000004171 R14: 0x0000000000000007 R15: 0x000070aa648e8ff0 EFL: 0x0000000000010202 -- C level backtrace information ------------------------------------------- ruby-3.2.2/lib/libruby.so.3.2(rb_print_backtrace+0xd) [0x70aa64d5bb5f] ruby-3.2.2/vm_dump.c:785 ruby-3.2.2/lib/libruby.so.3.2(rb_vm_bugreport) ruby-3.2.2/vm_dump.c:1080 ruby-3.2.2/lib/libruby.so.3.2(rb_bug_for_fatal_signal+0xf4) [0x70aa64b52164] ruby-3.2.2/error.c:813 ruby-3.2.2/lib/libruby.so.3.2(sigsegv+0x4d) [0x70aa64cab0fd] ruby-3.2.2/signal.c:964 /lib/x86_64-linux-gnu/libc.so.6(0x70aa64642520) [0x70aa64642520] ruby-3.2.2/lib/libruby.so.3.2(hash_table_index+0x0) [0x70aa64cedbe7] ruby-3.2.2/symbol.h:72 ruby-3.2.2/lib/libruby.so.3.2(rb_id_table_lookup) ruby-3.2.2/id_table.c:230 ruby-3.2.2/lib/libruby.so.3.2(cached_callable_method_entry+0x24) [0x70aa64d356bb] ruby-3.2.2/vm_method.c:1295 ruby-3.2.2/lib/libruby.so.3.2(callable_method_entry_or_negative) ruby-3.2.2/vm_method.c:1365 ruby-3.2.2/lib/libruby.so.3.2(callable_method_entry) ruby-3.2.2/vm_method.c:1402 ruby-3.2.2/lib/libruby.so.3.2(rb_callable_method_entry) ruby-3.2.2/vm_method.c:1409 ruby-3.2.2/lib/libruby.so.3.2(gccct_method_search_slowpath+0x38) [0x70aa64d36258] ruby-3.2.2/vm_eval.c:434 ruby-3.2.2/lib/libruby.so.3.2(rb_call0+0x267) [0x70aa64d4e877] ruby-3.2.2/vm_eval.c:483 ruby-3.2.2/lib/libruby.so.3.2(rb_call+0x32) [0x70aa64d4f406] ruby-3.2.2/vm_eval.c:877 ruby-3.2.2/lib/libruby.so.3.2(rb_funcallv_kw) ruby-3.2.2/vm_eval.c:1074 ruby-3.2.2/lib/libruby.so.3.2(vm_call_cfunc_with_frame+0x127) [0x70aa64d30277] ruby-3.2.2/vm_insnhelper.c:3268 ruby-3.2.2/lib/libruby.so.3.2(vm_sendish+0x97) [0x70aa64d407a4] ruby-3.2.2/vm_insnhelper.c:5080 ruby-3.2.2/lib/libruby.so.3.2(vm_exec_core) ruby-3.2.2/insns.def:820 ruby-3.2.2/lib/libruby.so.3.2(rb_vm_exec+0xd3) [0x70aa64d460d3] ruby-3.2.2/vm.c:2374 ruby-3.2.2/lib/libruby.so.3.2(rb_vm_invoke_proc+0x5f) [0x70aa64d4bfcf] ruby-3.2.2/vm.c:1603 ruby-3.2.2/lib/libruby.so.3.2(vm_call0_body+0x5df) [0x70aa64d4c5cf] ruby-3.2.2/vm_eval.c:274 ruby-3.2.2/lib/libruby.so.3.2(vm_call0_cc+0x77) [0x70aa64d4e7e7] ruby-3.2.2/vm_eval.c:87 ruby-3.2.2/lib/libruby.so.3.2(rb_call0) ruby-3.2.2/vm_eval.c:551 ruby-3.2.2/lib/libruby.so.3.2(rb_call+0x32) [0x70aa64d4f406] ruby-3.2.2/vm_eval.c:877 ruby-3.2.2/lib/libruby.so.3.2(rb_funcallv_kw) ruby-3.2.2/vm_eval.c:1074 ruby-3.2.2/lib/libruby.so.3.2(rb_eval_cmd_kw+0x142) [0x70aa64d4f562] ruby-3.2.2/vm_eval.c:1923 ruby-3.2.2/lib/libruby.so.3.2(signal_exec+0xf6) [0x70aa64caae16] ruby-3.2.2/signal.c:1064 ruby-3.2.2/lib/libruby.so.3.2(rb_threadptr_execute_interrupts+0x36b) [0x70aa64cf7078] ruby-3.2.2/thread.c:2334 ruby-3.2.2/lib/libruby.so.3.2(rb_threadptr_execute_interrupts) ruby-3.2.2/thread.c:2291 ruby-3.2.2/lib/libruby.so.3.2(rb_vm_check_ints+0xb) [0x70aa64cf7ac5] ruby-3.2.2/vm_core.h:1994 ruby-3.2.2/lib/libruby.so.3.2(rb_vm_check_ints) ruby-3.2.2/vm_core.h:1985 ruby-3.2.2/lib/libruby.so.3.2(unblock_function_set) ruby-3.2.2/thread.c:320 ruby-3.2.2/lib/libruby.so.3.2(blocking_region_begin) ruby-3.2.2/thread.c:1485 ruby-3.2.2/lib/libruby.so.3.2(rb_nogvl+0xbf) [0x70aa64cf90cf] ruby-3.2.2/thread.c:1548 signal-bug-testcase-2/lib/signal_bug_testcase.so(fprintf+0x0) [0x70aa6518f299] ../../../../ext/signal_bug_testcase/signal_bug_testcase.c:17 signal-bug-testcase-2/lib/signal_bug_testcase.so(bug_testcase_free) ../../../../ext/signal_bug_testcase/signal_bug_testcase.c:18 ruby-3.2.2/lib/libruby.so.3.2(run_final+0xf) [0x70aa64b73172] ruby-3.2.2/gc.c:4388 ruby-3.2.2/lib/libruby.so.3.2(finalize_list) ruby-3.2.2/gc.c:4407 ruby-3.2.2/lib/libruby.so.3.2(finalize_deferred_heap_pages) ruby-3.2.2/gc.c:4436 ruby-3.2.2/lib/libruby.so.3.2(rb_objspace_call_finalizer+0x350) [0x70aa64b80d70] ruby-3.2.2/gc.c:4573 ruby-3.2.2/lib/libruby.so.3.2(rb_ec_finalize+0x2a) [0x70aa64b5d6d1] ruby-3.2.2/eval.c:168 ruby-3.2.2/lib/libruby.so.3.2(rb_ec_cleanup) ruby-3.2.2/eval.c:262 ruby-3.2.2/lib/libruby.so.3.2(ruby_run_node+0x9d) [0x70aa64b5d91d] ruby-3.2.2/eval.c:330 ruby-3.2.2/bin/ruby(rb_main+0x21) [0x5d5d1295f187] ./main.c:38 ruby-3.2.2/bin/ruby(main) ./main.c:57 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_call_main+0x80) [0x70aa64629d90] ../sysdeps/nptl/libc_start_call_main.h:58 /lib/x86_64-linux-gnu/libc.so.6(call_init+0x0) [0x70aa64629e40] ../csu/libc-start.c:392 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main_impl) ../csu/libc-start.c:379 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main) (null):0 [0x5d5d1295f1d5] ``` ---Files-------------------------------- signal-bug-testcase-main.zip (7.73 KB) -- https://bugs.ruby-lang.org/

2 3

[ruby-core:118535] [Ruby master Feature#20625] Object#chain_of
by matheusrich (Matheus Richard) 24 Aug '24

24 Aug '24

Issue #20625 has been reported by matheusrich (Matheus Richard). ---------------------------------------- Feature #20625: Object#chain_of https://bugs.ruby-lang.org/issues/20625 * Author: matheusrich (Matheus Richard) * Status: Open ---------------------------------------- ## Motivation It's often common to traverse a tree/list-like structure in order to get a chain of elements. This proposal is to add a method to `Object` that allows collecting a chain of elements by applying a block to each element. It doesn't require the root element to be an instance of a specific class or respond to a specific protocol. I think this method could be useful in many cases, since hierarchies like this are common in codebases (e.g. file systems, organizations structures, commit histories, breadcrumbs in web apps, configuration hierarchies, etc.). Here are some examples extracted from real codebases (simplified for the sake of the example/privacy): ```rb # Given a file system structure, get the breadcrumbs of a file or directory def breadcrumbs(root) crumbs = [] current = root while current crumbs << current current = current.parent_dir end crumbs end # Given an employee, get the hierarchy of managers def hierarchy(employee) hierarchy = [] current = employee while current hierarchy << current current = current.manager end hierarchy end ``` ## Implementation The implementation in Ruby could look like this: ```rb class Object def chain_of(&block) chain = [] current = self while current chain << current current = block.call(current) end chain end end ``` Here's an example use: ```rb class ListNode attr_accessor :value, :parent def initialize(value, parent = nil) @value = value @parent = parent end def ancestors chain_of(&:parent).shift end end root = ListNode.new("root") child1 = ListNode.new("child1", root) child2 = ListNode.new("child2", child1) puts child2.ancestors.map(&:value) # => ["child1", "root"] ``` The examples from the motivation section could be rewritten as: ```rb breadcrumbs = root.chain_of(&:parent_directory) hierarchy = employee.chain_of(&:manager) ``` ## Considerations - While I'm including the object by default in the chain, it could be more intuitive to exclude it. In any case, it's easy to remove or add it with `shift`/`unshift`. - On a different note, the method could be named differently (I do like `chain_of`, though). Some alternatives I've considered are `map_chain`, `traverse`, and `trace_path`. - The method assumes that the traversal will finish at some point. If the user has a cyclic structure, it will loop indefinitely. We could stop looping if we find the same element twice. I don't think it's worth the extra complexity. - I'm not sure `Object` is the best place for this method. While it's very general, I think it gives power to the user to decide how to traverse a chain like this without having to rely on a specific class. Maybe a mixin (`Traversable`/`Chainable`) would be more appropriate? Could this fit in `Enumerable`, somehow? Of course, I'm open to suggestions and feedback. Thanks for reading! -- https://bugs.ruby-lang.org/

3 4

[ruby-core:118346] [Ruby master Bug#20586] Some filesystem calls in dir.c are missing error handling and can return incorrect results if interrupted
by ivoanjo (Ivo Anjo) 22 Aug '24

22 Aug '24

Issue #20586 has been reported by ivoanjo (Ivo Anjo). ---------------------------------------- Bug #20586: Some filesystem calls in dir.c are missing error handling and can return incorrect results if interrupted https://bugs.ruby-lang.org/issues/20586 * Author: ivoanjo (Ivo Anjo) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- #### Background Hey! I work for Datadog on the Ruby profiler part of the [`datadog` (previously `ddtrace`)](https://github.com/datadog/dd-trace-rb) gem. A customer reached [out with an issue](https://github.com/DataDog/dd-trace-rb/issues/3450) where enabling the profiler made `Dir.glob` return no files for a given folder: Without profiler: ``` irb(main):002:0> Dir.glob('/gcsfuse/t*') => ["/gcsfuse/test.html", "/gcsfuse/test.txt"] ``` With profiler: ``` irb(main):002:0> Dir.glob('/gcsfuse/t*') => [] ``` It turns out the issue is related to missing error handling in `dir.c`. The Datadog Ruby profiler, like stackprof, pf2 and vernier, uses unix signals to interrupt the currently-active thread and take a sample (usually `SIGPROF`). When some system calls get interrupted by a signal, they return an [EINTR error code](https://man7.org/linux/man-pages/man7/signal.7.html#:~:text=Interrupt… back to the caller. Consider for instance the implementation of `dir_each_entry` in `dir.c`: ```c static VALUE dir_each_entry(VALUE dir, VALUE (*each)(VALUE, VALUE), VALUE arg, int children_only) { struct dir_data *dirp; struct dirent *dp; IF_NORMALIZE_UTF8PATH(int norm_p); GetDIR(dir, dirp); rewinddir(dirp->dir); IF_NORMALIZE_UTF8PATH(norm_p = need_normalization(dirp->dir, RSTRING_PTR(dirp->path))); while ((dp = READDIR(dirp->dir, dirp->enc)) != NULL) { // ... do things } return dir; } ``` If `READDIR` returns `NULL`, then `dir_each_entry` assumes it has iterated the entire directory. But looking [at the man page for `readdir`](https://man7.org/linux/man-pages/man3/readdir.3.html) we see the following sharp edge (emphasis mine): > It returns NULL on reaching the end of the directory stream **or if an error occurred**. So what's happening in this situation is: `readdir` gets interrupted, returns `NULL` + sets errno to `EINTR`. But `dir_each_entry` doesn't check `errno`, so rather than raising an exception to flag the issue, it treats it as if the end of the directory has been reached. #### How to reproduce Reproducing this is somewhat annoying, because it's dependent on timing: the signal must arrive at the exact time the dir API is getting executed. I was able to reproduce this every time by using the [google cloud `gcsfuse`](https://cloud.google.com/storage/docs/gcs-fuse) tool. This somewhat makes sense -- a remote filesystem is much slower than a local one, so there's a much bigger window of opportunity for a signal to arrive while the system call is blocked. Here's an example I included in https://github.com/DataDog/dd-trace-rb/pull/3720: ``` # Not shown: Set up a trial google cloud account, install gcsfuse, create a cloud storage bucket and put in some test files $ gcsfuse test_fs_dd_trace_rb fuse-testing/ $ ls fuse-testing/ hello.txt test.html test.txt # Not shown: Add `datadog` gem to `Gemfile` $ DD_PROFILING_ENABLED=true DD_PROFILING_DIR_INTERRUPTION_WORKAROUND_ENABLED=false bundle exec ddprofrb exec ruby -e "Datadog::Profiling.wait_until_running; pp Dir.children('fuse-testing/')" [] ``` Let me know if you'd like me to try to create a reproducer that does not depend on the `datadog` gem. #### Additional notes I've spent quite some time looking at the `dir.c` sources, and here's the full list of APIs that suffer from issues: * `dir_each_entry` does not check `errno`; all of its users have interruption bugs * `dir_tell` will return -1 instead of the correct position (which means that passing -1 to `dir_seek`/`dir_set_pos` will cause it to not list the directory properly) * `do_opendir` an error in system calls will only sometimes be turned into a raised exception * Indirect callers that pass in rb_glob_error as errfunc: rb_glob, Dir.[], Dir.glob * Indirect callers that pass in 0 as errfunc: ruby_glob, ruby_brace_glob * `glob_opendir` does not check errno; all of its users have interruption bugs * `glob_getent` does not check errno; all of its users have interruption bugs * `nogvl_dir_empty_p` does not check errno (of readdir! it actually checks for opendir); all of its users have interruption bugs Less sure about these: * `do_stat`/`do_lstat` will turn errors into warnings (unclear if enabled or disabled by default) * `need_normalization` calls `fgetattrlist` / `getattrlist` and all errors `(ret != 0)` are treated in the same way * `rb_glob_error` is and `rb_glob_caller` leave exceptions as pending and rely on callers to raise them properly * Error handling of `rb_home_dir_of` and `rb_default_home_dir` are a bit suspicious As a workaround in the Datadog Ruby profiler, in https://github.com/DataDog/dd-trace-rb/pull/3720 I've added monkey patches to all of the Ruby-level APIs that use the above functions and mask out `SIGPROF` so these calls are never interrupted. This solution is does successfully work around the issue, although it prevents the profiler from sampling during these system calls, which will mean less visibility if e.g. these calls are taking a long time. And well, maintaining monkey patches is always problematic for future Ruby compatibility. -- https://bugs.ruby-lang.org/

5 12

[ruby-core:117882] [Ruby master Bug#20490] Process.waitpid2(-1, Process::WNOHANG) misbehaves on Ruby 3.1 & 3.2 with detached process
by stanhu (Stan Hu) 18 Aug '24

18 Aug '24

Issue #20490 has been reported by stanhu (Stan Hu). ---------------------------------------- Bug #20490: Process.waitpid2(-1, Process::WNOHANG) misbehaves on Ruby 3.1 & 3.2 with detached process https://bugs.ruby-lang.org/issues/20490 * Author: stanhu (Stan Hu) * Status: Open * ruby -v: ruby 3.2.4 (2024-04-23 revision af471c0e01) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is a follow-up issue for a bug that I thought was fixed in https://bugs.ruby-lang.org/issues/19837 and duplicated in https://bugs.ruby-lang.org/issues/20181. The following script doesn't terminate quickly in Ruby 3.1.5 and 3.2.4, even with the patches to address https://bugs.ruby-lang.org/issues/19837. It works fine in Ruby 3.3. It appears that the `Process::WNOHANG` argument passed to `Process.wait2` causes this script to spin until the child process stops: ```ruby #!/bin/env ruby Process.spawn({}, "sh -c 'sleep 600'").tap do |pid| puts "detaching PID #{pid}" Process.detach(pid) end forked_pid = fork do loop { sleep 1 } end child_waiter = Thread.new do puts "Waiting for child process to die..." # The spawned process has to exit before this returns in Ruby 3.1 and 3.2 loop do pid, status = Process.wait2(-1, Process::WNOHANG) puts "Exited PID: #{pid}, status: #{status}" break if pid sleep 1 end end process_killer = Thread.new do puts "Killing #{forked_pid}" system("kill #{forked_pid}") end child_waiter.join process_killer.join ``` If I drop the `Process::WNOHANG` argument, it works fine. -- https://bugs.ruby-lang.org/

3 6

[ruby-core:118699] [Ruby master Bug#20654] Floor and ceil have unexpected behaviour when ndigits is large
by peterzhu2118 (Peter Zhu) 18 Aug '24

18 Aug '24

Issue #20654 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20654: Floor and ceil have unexpected behaviour when ndigits is large https://bugs.ruby-lang.org/issues/20654 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11257 There is unexpected behaviour in Integer#floor, Float#floor, Integer#ceil, Float#ceil when ndigits is large such that 10**ndigits is a bignum. Previously, it would return 0 in such cases. However, this would cause unexpected behaviour such as: ```ruby puts -1.floor(-5) # => -100000 puts -1.floor(-10) # => -10000000000 puts -1.floor(-20) # => 0 ``` ```ruby puts 1.ceil(-5) # => 100000 puts 1.ceil(-10) # => 10000000000 puts 1.ceil(-20) # => 0 ``` The PR changes the last result so that it will return -100000000000000000000 and 100000000000000000000, respectively. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118588] [Ruby master Bug#20634] ruby uses 1/2 (7 out of 16) cores for 16 ractors with RUBY_MAX_CPU=16 environment variable set
by skorobogatydmitry (Dmitry Skorobogaty) 09 Aug '24

09 Aug '24

Issue #20634 has been reported by skorobogatydmitry (Dmitry Skorobogaty). ---------------------------------------- Bug #20634: ruby uses 1/2 (7 out of 16) cores for 16 ractors with RUBY_MAX_CPU=16 environment variable set https://bugs.ruby-lang.org/issues/20634 * Author: skorobogatydmitry (Dmitry Skorobogaty) * Status: Open * ruby -v: ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- It's a cont for #20618, feel free to close this one and reopen the original bug. According to `top`, the below code uses 7/16 cores (700%) of my AMD Ryzen 7 5800HS: #!/usr/bin/env ruby 16.times { Ractor.new { 10_000.downto(1) { |i| 100_000.downto(1) { |j| i * j } } } } sleep 30 ``` Command I ran is `RUBY_MAX_CPU=16 ruby test.rb`. I also tried to set `RUBY_MN_THREADS=1` as an addition and the script starts to occupy 8 cores instead of 7 (possibly, due to N:M algo allocating an extra core for the blocked main Ractor). The same code occupies all 16 cores on ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] The problem reproduces on ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x86_64-linux] (ruby from the latest docker image). -- https://bugs.ruby-lang.org/

1 2