November 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:115247] [Ruby master Feature#19988] AI for inner code behavior analysis at runtime
by horv77＠protonmail.com (Andras Horvath) 05 Nov '23

05 Nov '23

Issue #19988 has been reported by horv77(a)protonmail.com (Andras Horvath). ---------------------------------------- Feature #19988: AI for inner code behavior analysis at runtime https://bugs.ruby-lang.org/issues/19988 * Author: horv77(a)protonmail.com (Andras Horvath) * Status: Open * Priority: Normal ---------------------------------------- Dear Devs, I've been using Ruby extensively in my work and sometimes I use AI algos to debug inner workings of complex code loops. I wanted to send you the idea because I believe it is very easy to add the following functionality to Ruby and Ruby could be the first programming language ever from the mainstream ones to apply smart AI decisions internally. It could give Ruby a huge marketing power as well with all its advantages. So what is the idea that I already use sometimes to help my development? When I consider some part of the code, it usually involves manipulating many variables in a complex way. And my very simple idea is to get all variables analyzed automatically with an unsupervised, multivariable anomaly detection algo continuously. So when I run it on the variables inside an inner loop or structure, then the AI might be able to spot weird combinations of variable states that may be important to notify the developer of. The language may give a warning about these states or even jump to the entry using binding.irb. Anomaly detection works like when you have an N dimensional array, then every entry is analyzed and the most weird ones can be "spotted" (by showing their indexes of their positions in the array for instance). That's it. The fastest algo could be used for this. It is ISOLATION FOREST. It runs in quasi linear time. So if the user would enable this option at a particular line of code, then the Ruby interpreter would store all variables in an array on every run and would analyze it with the AD algo from time to time (because it takes time). It wouldn't have to analyze at every run because once the anomalous variables combinations get into the array, then they can be found later at any time. If the variable is not numeric (since all AI algo need numeric inputs), then a "text conversion to numeric vector" could be done. I've got a solution for that too. That is also very sophisticated and would boost Ruby's capabilities a lot IMHO. So a very sophisticated inner semantic AI analysis could take place with this option, which is very straightforward and easy to implement. The same could be applied for the behavior analysis of the memory usage. Many concaptual bugs could be tackled with such methods. It would be straightforward to do that from the interpreter side. I'd gladly provide the solution of the AI part. Thanks. Andras -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115243] CHANGE: add sample method to Range
by horv77 05 Nov '23

05 Nov '23

Dear Devs, I'd like to suggest a change. Since the following works: (1..99).first(5) Therefore this one could be logical and useful to work: (1..99).sample(5) Thanks, Andras

1 0

[ruby-core:115242] [Ruby master Bug#18286] Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64
by nobu (Nobuyoshi Nakada) 05 Nov '23

05 Nov '23

Issue #18286 has been updated by nobu (Nobuyoshi Nakada). Does this help it? https://github.com/ruby/ruby/pull/8708 ---------------------------------------- Bug #18286: Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64 https://bugs.ruby-lang.org/issues/18286#change-105157 * Author: ccaviness (Clay Caviness) * Status: Open * Priority: Normal * ruby -v: 3.0.2 * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- A universal arm64/x86_84 ruby binary for macOS built on a x86_64 machine segfaults/is killed when executed on an arm64 machine. To reproduce: * On an Intel Mac: `git clone https://github.com/ruby/ruby && cd ruby && git checkout v3_0_2 && ./autogen.sh && ./configure --with-arch=arm64,x86_64 && make -j$(sysctl -n hw.ncpu)` * Copy the built `./ruby` binary to an Apple Silicon machine * Attempt to execute it Expected: The universal `ruby` binary works correctly on both devices Actual: The universal `ruby` binary crashes with either `Segmentation fault: 11` or `Killed: 9` (this seems to occur if `arm64e` is used instead of `arm64`). Details: I'm attempting to build a universal Ruby for macOS that will run on both Intel (x86_64) and Apple Silicon (arm64) machines. It seemed initially that this was as easy as adding `--with-arch=arm64,x86_64` to `./configure` would do it, as it produced a `ruby` binary that reports as `Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]` This `ruby` works correctly on the Intel machine I built in on, but does not work when copied to an Apple Silicon device. The reverse, however, seems to work. That is, if I build the universal ruby on an Apple Silicon machine, the `ruby` binary that's built seems to work correctly on both Intel and Apple Silicon machines. Intel: ``` $ ./ruby -v ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [universal.x86_64-darwin21] ``` Apple Silicon: ``` $ ./ruby -v Segmentation fault: 11 $ lldb ./ruby (lldb) target create "./ruby" Current executable set to '/Users/crc/ruby' (arm64). (lldb) run Process 77071 launched: '/Users/crc/ruby' (arm64) Process 77071 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x00000001002176b8 ruby`ruby_vm_special_exception_copy + 16 ruby`ruby_vm_special_exception_copy: -> 0x1002176b8 <+16>: ldr x0, [x0, #0x8] 0x1002176bc <+20>: bl 0x10011fed8 ; rb_class_real 0x1002176c0 <+24>: bl 0x10012070c ; rb_obj_alloc 0x1002176c4 <+28>: mov x20, x0 Target 0: (ruby) stopped. (lldb) ^D ``` I also attempted the same thing with ruby 2.7.4 source, with the same result. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115241] [Ruby master Bug#19986] Win32: `HOME` is set to just `HOMEDRIVE` if `HOMEPATH` is unset
by nobu (Nobuyoshi Nakada) 05 Nov '23

05 Nov '23

Issue #19986 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19986: Win32: `HOME` is set to just `HOMEDRIVE` if `HOMEPATH` is unset https://bugs.ruby-lang.org/issues/19986 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- ``` $ env -u HOME -u USERPROFILE -u HOMEPATH HOMEDRIVE=no/such/home ./miniruby.exe -e 'p ENV["HOME"], Dir.home' "no/such/home" "no/such/home" ``` `HOMEDRIVE` should be ignored without `HOMEPATH`, and `HOME` should fallback to `CSIDL_PROFILE`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115225] [Ruby master Feature#16476] Socket.getaddrinfo cannot be interrupted by Timeout.timeout
by mame (Yusuke Endoh) 02 Nov '23

02 Nov '23

Issue #16476 has been updated by mame (Yusuke Endoh). Status changed from Open to Closed #19965 solved the problem, so I'll close it. I completely forgot this ticket, but the concept (pthread_create on every call to getaddrinfo(3)) was the same as in #19965. Just for case, #19965 is a bit more elaborate: * It works as before in environments where pthread is not available. * It includes the same hack for not only `getaddrinfo(3)` but also `getnameinfo(3)`. * `pthread_cancel` is not used because until recently [there was a bug in glibc](https://bugzilla.redhat.com/show_bug.cgi?id=1405071) where `pthread_cancel` of a pthread that is calling `getaddrinfo(3)` would cause subsequent name resolution to stick: * It attempts to use `pthread_setaffinity_np` to reduce speed degradation. ---------------------------------------- Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeout https://bugs.ruby-lang.org/issues/16476#change-105143 * Author: kirs (Kir Shatrov) * Status: Closed * Priority: Normal * Assignee: Glass_saga (Masaki Matsushita) ---------------------------------------- It seems like the blocking syscall done by `Socket.getaddrinfo` blocks Ruby VM in a way that Timeout.timeout has no effect. See reproduction steps in getaddrinfo_interrupt.rb (https://gist.github.com/kirs/00c02ef92e0418578135fe0a6cbd3d7d) This affects all modern Ruby versions, including the latest 2.7.0. Combined with default 10s resolv timeout on many Linux systems, this can have a very noticeable effect on production Ruby apps being not resilient to slow DNS resolutions, and being unable to fail fast even with `Timeout.timeout`. While https://bugs.ruby-lang.org/issues/15553 improves the situation for `Addrinfo.getaddrinfo`, `Socket.getaddrinfo` is still blocking the VM and Timeout has no effect. I'd like to discuss what could be done to make that call non-blocking for threads in Ruby VM. **UPD:** looking closer, I can see that `Socket.getaddrinfo("www.ruby-lang.org", "http")` and `Addrinfo.getaddrinfo("www.ruby-lang.org", "http")` call non-interruptible `getaddrinfo`, while `Addrinfo.getaddrinfo("www.ruby-lang.org", "http", timeout: 10)` calls `getaddrinfo_a`, which is interruptible: ``` ruby # interrupts as expected Timeout.timeout(1) do Addrinfo.getaddrinfo("www.ruby-lang.org", "http", timeout: 10) end ``` I'd maybe suggest that we try to *always* use `getaddrinfo_a` when it's available, including in `Socket.getaddrinfo`. What downsides that would have? I'd be happy to work on a patch. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115224] AI for inner code behavior analysis at runtime
by horv77 02 Nov '23

02 Nov '23

Dear Fellow Devs, I've been using Ruby extensively in my work and sometimes I use AI algos to debug inner workings of complex code loops. I wanted to send you the idea because I believe it is very easy to add the following functionality to Ruby and Ruby could be the first programming language ever from the mainstream ones to apply smart AI decisions internally. It could give Ruby a huge marketing power. So what is the idea that I already use sometimes to help my development? When we consider some part of the code, it usually involves manipulating many variables in a complex way. And my very simple idea is to get all variables analyzed automatically with an unsupervised, multivariable anomaly detection algo continuously. So when we run it on the variables inside an inner loop or structure, then the AI might be able to spot weird combinations of variable states that may be important to notify the developer of. The language may give a warning about these states or even jump to the entry using binding.irb. Anomaly detection works like when you have an N dimensional array, then every entry is analyzed and the most weird ones can be "highlighted" (by showing their indexes of their array for instance). That's it. The fastest algo could be used for this. It is ISOLATION FOREST. It runs in quasi linear time. So if the user enables this option at a particular line of code, then the Ruby core would store all variables in an array on every run and would analyze it with the AD algo from time to time (because it takes time). It wouldn't have to analyze at every run because once the anomalous variables combinations get into the array, then they can be found later at any time. If the variable is not numeric (since all AI algo need numeric inputs), then a "text conversion to numeric vector" could be done. I've got a solution for that too. That is also very sophisticated and would boost Ruby's capabilities a lot. So a very sophisticated inner semantic AI analysis could take place with this option, which is very straightforward and easy to implement. I can help doing that. I also have another idea if you're interested in the first place. Cheers and thanks. Andras

1 0