Issue #22067 has been updated by wks (Kunshan Wang). luke-gru (Luke Gruber) wrote in #note-5:
... some others said they've seen problematic mark functions in some extensions (reading/mutating global state, etc.) ...
jhawthorn (John Hawthorn) wrote in #note-6:
FWIW, `ractor_local_storage_mark`, `rb_thread_sched_mark_zombies`, `rb_iseq_mark_and_move` are examples where mutation happens as part of the mark step. I suspect/hope they do end up safe to run in parallel because they're only mutating their own storage. I don't know whether any C extensions have even less well behaved mark functions, but hopefully we can assume mark is safe to run in parallel and consider that a bug in the extension.
Yes. That's exactly the case I was referring to. For example, `rb_iseq_mark_and_move` tries to do clean-up during marking, and `rb_imemo_mark_and_move` attempts to clean up `imemo_callcache` during reference updating. This breaks the abstraction layer. From GC's point of view, the VM should provide an object-scanning function that simply enumerates fields that hold references, and does nothing else. Cleaning up is part of finalization (e.g. the `callcache` should have used the weak reference semantics to do finalization). That's why Peter introduced the `*_mark_and_move` functions in the first place, i.e. to make a unified field-enumerating function that works for both marking and reference updating (and may even be directly called by `rb_objspace_reachable_objects_from`). `RUBY_TYPED_DECL_MARKING` is even better because the developer of the TypedData doesn't need to write C functions for dmark or dcompact, leaving no room for them to introduce behaviors like attempting to clean up objects or accessing global data during marking. MMTk and other GC modules can benefit from `RUBY_TYPED_DECL_MARKING` because they can interpret the offsets table directly, avoiding the overhead of calling into C functions and setting up callbacks that intercept `rb_gc_impl_mark`. luke-gru (Luke Gruber) wrote in #note-5:
Finalizers need to run on the main Ruby thread while the thread holds the GVL. Maybe I'm confused about the terminology you're using.
jhawthorn (John Hawthorn) wrote in #note-6:
`ObjectSpace::define_finalizer` because it requires the full GVL and needs to be deferred to an interrupt check is awkward and bad for performance (actually there are even more guarantees it _should_ require that aren't currently implemented correctly, like sticking to the defining Ractor)...
Sorry about the confusion. I didn't realize `ObjectSpace::define_finalizer` needs GVL. Java doesn't have GIL, so the threads that process finalizers, `PhantomReference` and `Cleaner` can run concurrently with other Java threads. When I mentioned `ObjectSpace::define_finalizer`, I was referring to a finalization mechanism that is similar to PhantomReference and Cleaner in Java. Specifically, it 1. Doesn't have access to the dead object itself, therefore allowing the memory of the dead object to be immediately reused after GC. 2. Has all the context (e.g. the file descriptor, the memory pointer, etc.) needed to do the cleanup, therefore can be executed even if the dead object is reclaimed. In the MMTk GC module, we have something called "final jobs". Each "final job" is basically a tuple of `(free_func, data)`, and it is executed by calling `free_func(data)`. It is used to do what "zombies" do in CRuby's default GC, but does not leave dead objects in the heap after GC. Currently they are still executed at interrupts like zombies, but it should be possible to offload their execution to a dedicated fthread because currently zombies are for cleaning up IO or file objects, and they shouldn't need the GVL to clean up. So what I think is better for cleaning up TypedData is something like the "final job" (tuple of `(free_func, data)`) that can be executed concurrently with the mutator. It can even run concurrently with the sweeper thread which sweeps the heap pages. In this case, a `RUBY_TYPED_EMBEDDABLE | RUBY_TYPED_DEFAULT_FREE` object can be trivially swept. I wonder how we can present this interface to the extension developer. The simplest API is just a global function that lets the developer register the triple `(object, free_func, data)`, for example, ```c void ruby_register_cleanup_function(VALUE object, void (*free_func)(void*), void *data); ``` which basically means "if `object` is dead, call `free_func(data)`". We can require that this `free_func` must be callable without holding the GVL so that it can be executed concurrently with mutators (I think it is the intended semantics of `RUBY_TYPED_PARALLEL_FREE_SAFE`). Maybe we can give a warning if the `object` is not `RUBY_TYPED_EMBEDDABLE | RUBY_TYPED_DEFAULT_FREE` because if we use this mechanism for finalization, we won't need anything other than the default free, and the finalizer doesn't need to read the object body so it doesn't matter if it is embedded or not. luke-gru (Luke Gruber) wrote in #note-5:
That's too bad if that's the case. Peter seems to think it would, although maybe you meant something by *directly* that I didn't catch.
By "directly" I meant MMTk won't be able to use it if the MMTk GC module still calls `obj_free` on all dead objects during GC. But if we are able to offload the clean-up to mutator time, (e.g. using the `(free_func, data)` approach I mentioned earlier), it will still be applicable to MMTk. MMTk just needs to identify dead objects and enqueue the "final jobs" during GC, and they can be executed during mutator time. (Similarly, the default GC can identify such objects during sweeping, and the `free_func(data)` can be executed in parallel to sweeping.) I don't think resurrecting objects like Java's `Object.finalize` is a good idea. jhawthorn (John Hawthorn) wrote in #note-6:
... `RUBY_TYPED_FREE_IMMEDIATELY` relaxes this requirement to allow it to be run as part of GC rather than deferred...
Thank you for pointing this out. I have always thought `RUBY_TYPED_FREE_IMMEDIATELY` was a constraint (instead of relaxation) that an object must be freed before GC finishes (i.e. cannot be postponed to the mutator time) so that the object's memory can be reclaimed immediately after GC. Currently, MMTk always runs the `obj_free` of *all dead objects* during GC. With the intention of "`RUBY_TYPED_FREE_IMMEDIATELY` being an relaxation" in mind, I think that means the cleaning up of those objects can be postponed to mutator time (in a thread without holding GVL), as long as we can extract the `(free_func, data)` for it. The intention is that it is unprofitable to parallelize the invocation of `obj_free` because `free()` doesn't scale (glibc, jemalloc and tcmalloc scales negatively; mimalloc scales only up to 4x speed up. See Table 1 of [this paper](https://wks.github.io/downloads/pdf/ruby-ismm-2025.pdf)), and most applications only spend a small fraction of time on GC, and the fact that CRuby uses GVL leaves much CPU resource vacant during mutator time. p.s. It is interesting to see that some people want to offload the finalization to GC time because of GVL, and others want to offload the finalization to mutator time because it is not profitable to run in parallel during GC due to the scalability of `free()`. But their goals converge because they both want the finalization to be executed at the appropriate time without keeping other threads (mutators or GC workers) waiting. ---------------------------------------- Feature #22067: New TypedData bit to allow the type to be freed in parallel https://bugs.ruby-lang.org/issues/22067#change-117318 * Author: luke-gru (Luke Gruber) * Status: Open ---------------------------------------- CRuby `TypedData` types are used internally in the VM and in C extensions and currently any `free` functions for these types are run with the VM lock held as well as the VM barrier (or, in the case of MMTk, are not freed in parallel). In short, no other Ruby threads or GC threads can run while these `free` functions are called. In order to allow one or more worker GC threads to free these `TypedData` objects, we need a way to specify that a `TypedData` type can be freed in parallel alongside the Ruby GC thread or Ruby code that is being run by another thread. Otherwise, we cannot free these types in the workers and must rely on the Ruby GC thread to do so. This is because it can be unsafe to call these `free` functions, depending on how they're implemented. Most `TypedData` are safe to free in parallel. The exceptions are those `free` functions that read or modify global state without locking. ### Examples ```c static void example_data_free(void *ptr) { st_delete(live_example_datas, (st_data_t*)&ptr, NULL); // Not thread-safe! } ``` If 2 of these `TypedData` objects are freed at the same time, this could corrupt the `st_table`. A lock must be held when manipulating this table. Because Ruby code can also run alongside a sweep worker, the lock must also be held when adding to or iterating this table. ```c rb_nativethread_lock_t example_data_lock; static void example_data_free(void *ptr) { rb_native_mutex_lock(&example_data_lock); st_delete(live_example_datas, (st_data_t*)&ptr, NULL); rb_native_mutex_unlock(&example_data_lock); } static void live_examples_add(void *ptr) { rb_native_mutex_lock(&example_data_lock); st_insert(live_example_datas, (st_data_t)ptr, (st_data_t)ptr); rb_native_mutex_unlock(&example_data_lock); } static void Init_example(void) { rb_native_mutex_initialize(&example_data_lock); } ``` If a CRuby developer or user wants to make their code compaction-safe, they don’t need to worry about parallel sweep workers because the workers don't run during compaction. ### Proposal I propose adding a new flag to `TypedData` that allows both CRuby developers and extension authors to opt-in to allow their `TypedData` type to be freed in parallel. It could be called something like `RUBY_TYPED_PARALLEL_FREE_SAFE`. In our branch where we are developing parallel sweeping, most `TypedData` internal to the VM are given this flag. I believe C extension authors would opt in as well if they see that parallel sweeping gives good performance benefits to Ruby applications. We would need to document what is safe and what is not safe inside these `free` functions for types that are marked with this bit. If the user needs to use a native mutex to protect their `TypedData` from being corrupted when freed in parallel, they must do so. However, if they lock this native mutex in non-free function code paths as well, they may not allocate objects or use `ruby_xmalloc` while this mutex is held. These are some examples of trickiness when it comes to concurrency in the Ruby VM. I believe having a section in the extension documentation about these types of free functions (or elsewhere such as the Concurrency Guide) would give CRuby developers and extension authors more confidence in adding this bit to their types. ### A possible future: parallel marking? If in the future CRuby gets parallel marking, we believe we probably would need another bit for `TypedData` so we can register them as parallel-mark safe. If that’s the case, it’s unfortunate that authors that want this feature would need to update their extension again with a new bit if this bit were to be made public. One could argue that we should have a single bit that indicates safety for both parallel freeing and parallel marking. However, we believe the specifics of what could and couldn’t be executed inside these `free` and `mark` functions would be too hard to work out today for a combined bit without locking us into a specific parallel marking implementation. ### Alternative We could not expose the parallel-free-safe bit to extension authors and only free internal `TypedData` objects in parallel. However, this does slow the current implementation of parallel sweeping down because even if one of these objects is on a heap page, the Ruby GC thread needs to further post-process the page after the sweep thread sweeps it. It would also limit further optimizations to parallel sweeping. ### Details of the bit Worker threads could only free `TypedData` objects that have this new bit set alongside the `RUBY_TYPED_FREE_IMMEDIATELY` bit. Otherwise, the Ruby GC thread must free them. Also, if the type has `0` or `RUBY_TYPED_DEFAULT_FREE`/`RUBY_DEFAULT_FREE` as the free function, it can be freed in parallel. ### Conclusion Parallel sweeping is being actively developed and adding this bit increases performance of the developing implementation of parallel sweeping. It allows `TypedData` types to be freed by worker thread(s), which decreases GC pause times. Since `TypedData` are also exposed to extension authors, we need to whitelist these types for parallel freeing, as not all of these functions are currently safe. By documenting what is and isn’t safe, extension authors can write correct `free` functions that are safe to be freed by worker threads. Alternatively, we could have this bit be internal to CRuby. ---Files-------------------------------- Screenshot_20260513_185452.webp (99.8 KB) Screenshot_20260513_192449.webp (31.9 KB) Screenshot_20260513_192734.webp (22.4 KB) Screenshot_20260513_192910.webp (27.3 KB) -- https://bugs.ruby-lang.org/