Issue #20587 has been updated by ivoanjo (Ivo Anjo). jeremyevans0 (Jeremy Evans) wrote in #note-2:
@ivoanjo Can you be specific about which places this affects? I examined all `opendir` calls in `dir.c`:
* Called in `nogvl_opendir` * Called in `opendir_without_gvl` when VM is not initialized * Called in `nogvl_opendir_at` * Called in `nogvl_dir_empty_p`
Tracing those function calls, if the VM is initialized, it appears that all calls use the `IO_WITHOUT_GVL` macro to release the GVL around the call.
Hey Jeremy, thanks for taking a look at this! You are right -- I think I was overeager in including `readdir` on the list. I'll update my original description to correct for this. Here's my attempt at a more specific report of blocking calls while holding the GVL in `dir.c` (I've omitted cases where the GVL is released): * `fdopendir` -> called by `dir_s_for_fd` * `readdir` -> called by `dir_read`, `dir_each_entry`, `glob_helper`, `glob_getent` * `chdir` -> called by `dir_chdir0` (Although this one is somewhat weird already so perhaps it being blocking is not the weirdest part?) * `fchdir` -> called by `dir_fchdir` (same as `chrdir`) * `chroot` -> called by `dir_s_chroot` * `lstat` -> called by `do_lstat` * `stat` -> called by `do_stat` * `fstatat` -> called by `do_stat`/`do_lstat` * `fgetattrlist` -> called by `need_normalization`, `is_case_sensitive` * `getattrlist` -> called by `is_case_sensitive`, `replace_real_basename`, `rb_dir_s_empty_p`, `need_normalization`, `dir_initialize` * `getpwnam` -> called by `dir_s_home` ---------------------------------------- Bug #20587: dir.c calls blocking filesystem APIs/system calls while holding the GVL https://bugs.ruby-lang.org/issues/20587#change-109041 * Author: ivoanjo (Ivo Anjo) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hey! I work for Datadog on the Ruby profiler part of the [`datadog` (previously `ddtrace`)](https://github.com/datadog/dd-trace-rb) gem. While I was investigating https://bugs.ruby-lang.org/issues/20586, I spotted that there's a number of cases where, in `dir.c`, blocking system calls are being made (e.g. `readdir()`, `opendir()`, etc) without releasing the GVL. This means that if they block for a long time (as happens in the gcsfuse example in https://bugs.ruby-lang.org/issues/20586 ), the Ruby VM will just be blocked and not make any progress. The combination of not releasing the GVL + slow system calls actually makes the issue in https://bugs.ruby-lang.org/issues/20586 more likely to happen with the Datadog profiler, although even if the code releases the GVL the underlying issue could still happen, and this is why I decided to file this bug separately. -- https://bugs.ruby-lang.org/