Hi,
I just evaluated the possibility of using Ractors to distribute
processing using various gems with native extensions (some matrix/vector
computations with Numo::Linalg and approximate vector searches) and I've
mixed feelings.
The Ractor design seems very sound to me and I would prefer to use them
to distribute the load on multiple CPUs but in practice I see 2 major
obstacles :
- the first use of Ractor code isn't encouraging, even with Ruby 3.2.0,
as it outputs :
"warning: Ractor is experimental, and the behavior may change in future
versions of Ruby! Also there are many implementation issues."
- when heavy computations are involved you are probably already using
gems with native extensions and when trying to use them in a non-main
Ractor you will almost always get an exception :
"ractor unsafe method called from not main ractor (Ractor::UnsafeError)"
This last problem could be manageable, according to :
https://bugs.ruby-lang.org/issues/17307
native extensions should mark their individual functions as thread safe
using :
#ifdef HAVE_RB_EXT_RACTOR_SAFE
rb_ext_ractor_safe(true);
#endif
I was considering contacting the maintainers of the various gems that we
use to check with them which methods are safe and to see if I can submit
a pull request but I'm a bit hesitant and wonders if Ractors are
actually used and stable enough for maintainers to care.
Do people on this list use Ractors? What for?
Do people code gems with native extensions with them in mind? I've seen
traces of rb_ext_ractor_safe when searching on Github but almost all of
the matches are from load.h from the Ruby headers but not many actual
uses (in the 4 first pages of results Psych and TruffleRuby are the only
exceptions and the first external gem is ruby-extlz4 on the 5th page...).
I've yet to see how the ffi gem can handle Ractors, I already know
there's no trace of Ractors in its code and this could be a complete new
can of worms (or not...).
I'll dig a bit more but in my current position I think I'll favor a
multi-process solution to scale (mostly because I don't have a
definitive list of gems I'll have to patch at this point and time is
short). That's a shame and I'm willing to revisit this later.
Best regards,
Lionel