- ruby-core - ml.ruby-lang.org

[ruby-core:115146] [Ruby master Misc#19971] Confusing arity of a Proc with implicit rest parameter
by andrykonchin (Andrew Konchin) 24 Oct '23

24 Oct '23

Issue #19971 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Misc #19971: Confusing arity of a Proc with implicit rest parameter https://bugs.ruby-lang.org/issues/19971 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal ---------------------------------------- I've noticed that such proc `proc { |a,| }` has arity `1` that means 1 required parameter, but behaves like there is a rest parameter (`proc { |a, *| }`) that has arity `-2` (that means 1 required parameter + rest parameter). So I am wondering whether it's intentional behaviour and what the rational is behind it. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:115124] [Ruby master Bug#19968] make install - RBS gem is not installed
by MSP-Greg (Greg L) 24 Oct '23

24 Oct '23

Issue #19968 has been reported by MSP-Greg (Greg L). ---------------------------------------- Bug #19968: make install - RBS gem is not installed https://bugs.ruby-lang.org/issues/19968 * Author: MSP-Greg (Greg L) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- See failing builds in https://github.com/ruby/ruby-dev-builder/actions/runs/6605670146 Same issue with ruby-loco. I assume gems/bundled_gems.rb will need an update. Also, https://github.com/ruby/rbs/pull/1579 -- https://bugs.ruby-lang.org/

2 3

[ruby-core:115016] [Ruby master Bug#19921] TestYJIT#test_bug_19316 test failure
by vo.x (Vit Ondruch) 23 Oct '23

23 Oct '23

Issue #19921 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19921: TestYJIT#test_bug_19316 test failure https://bugs.ruby-lang.org/issues/19921 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-12 master 52709a4862) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Testing with 52709a4862 on Fedora Rawhide, I observe following failure: ~~~ 94) Failure: TestYJIT#test_bug_19316 [/builddir/build/BUILD/ruby-3.3.0-52709a4862/test/ruby/test_yjit.rb:1126]: Expected no exits, but got {:opt_and=>1} ~~~ -- https://bugs.ruby-lang.org/

5 20

[ruby-core:115068] [Ruby master Bug#19929] Warnings for `mutex_m`, `drb` and `base64` appears while the gem spec has explicit dependencies
by yahonda (Yasuo Honda) 23 Oct '23

23 Oct '23

Issue #19929 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #19929: Warnings for `mutex_m`, `drb` and `base64` appears while the gem spec has explicit dependencies https://bugs.ruby-lang.org/issues/19929 * Author: yahonda (Yasuo Honda) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-17T11:07:57Z :detached: 2a8ac8ead5) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Running some of Ruby on Rails unit test raises these warnings while it has explicit add_dependency entries https://github.com/rails/rails/blob/ae6fa7a048d0a0f79b0f8dd8566c7d499fd6f61… ``` warning: mutex_m which will be not part of the default gems since Ruby 3.4.0. Add mutex_m to your Gemfile. warning: drb which will be not part of the default gems since Ruby 3.4.0. Add drb to your Gemfile. warning: base64 which will be not part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile. ``` ### Steps to reproduce ``` git clone https://github.com/rails/rails cd rails rm Gemfile.lock bundle install cd railties RAILS_STRICT_WARNING=1 bin/test test/engine/commands_test.rb test/engine/test_test.rb ``` ### Expected behavior It should finish successfully. ### Actual behavior It raises these warnings. Rails has the `RAILS_STRICT_WARNING` environment variable to fail if any warnings. ``` warning: mutex_m which will be not part of the default gems since Ruby 3.4.0. Add mutex_m to your Gemfile. warning: drb which will be not part of the default gems since Ruby 3.4.0. Add drb to your Gemfile. warning: base64 which will be not part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile. ``` ``` $ ruby -v ruby 3.3.0dev (2023-10-17T11:07:57Z :detached: 2a8ac8ead5) [x86_64-linux] $ RAILS_STRICT_WARNING=1 bin/test test/engine/commands_test.rb test/engine/test_test.rb /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/notifications/fanout.rb:3: warning: mutex_m which will be not part of the default gems since Ruby 3.4.0. Add mutex_m to your Gemfile. /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/testing/parallelization.rb:3: warning: drb which will be not part of the default gems since Ruby 3.4.0. Add drb to your Gemfile. /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/message_encryptor.rb:4: warning: base64 which will be not part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile. /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/testing/strict_warnings.rb:33:in `warn': /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/message_encryptor.rb:4: warning: base64 which will be not part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile. (RuntimeError) from <internal:warning>:51:in `warn' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:248:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/message_encryptor.rb:4:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/encrypted_file.rb:5:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/activesupport/lib/active_support/encrypted_configuration.rb:4:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/application.rb:9:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails.rb:15:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/all.rb:5:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/test/abstract_unit.rb:15:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/test/engine/commands_test.rb:3:in `<top (required)>' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `require' from /home/yahonda/.rbenv/versions/trunk/lib/ruby/3.3.0+0/bundler/rubygems_integration.rb:250:in `block (2 levels) in replace_require' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:51:in `block in load_tests' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:51:in `each' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:51:in `load_tests' from /home/yahonda/src/github.com/rails/rails/railties/lib/rails/test_unit/runner.rb:43:in `run' from /home/yahonda/src/github.com/rails/rails/tools/test.rb:18:in `<top (required)>' from bin/test:5:in `require_relative' from bin/test:5:in `<main>' $ ruby -v ruby 3.3.0dev (2023-10-17T11:07:57Z :detached: 2a8ac8ead5) [x86_64-linux] $ gem -v 3.5.0.dev $ bundle -v Bundler version 2.5.0.dev $ ``` -- https://bugs.ruby-lang.org/

2 7

[ruby-core:115110] [Ruby master Bug#19966] Prism defines global method instead of singleton method
by nobu (Nobuyoshi Nakada) 19 Oct '23

19 Oct '23

Issue #19966 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19966: Prism defines global method instead of singleton method https://bugs.ruby-lang.org/issues/19966 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED ---------------------------------------- The receiver is ignored and compiled to `definemethod` instead of `definesmethod`. ``` $ ./ruby -v -e 'puts RubyVM::InstructionSequence.compile_prism("def a.foo;end").disasm' ruby 3.3.0dev (2023-10-20T00:12:09Z master 985370406d) [arm64-darwin22] == disasm: #<ISeq:<compiled>@<compiled>:0 (0,0)-(0,13)> 0000 definemethod :foo, foo 0003 putobject :foo 0005 leave == disasm: #<ISeq:foo@<compiled>:0 (0,0)-(0,13)> 0000 putnil 0001 leave ``` ``` $ ./ruby -v -e 'puts RubyVM::InstructionSequence.compile("def a.foo;end").disasm' ruby 3.3.0dev (2023-10-20T00:12:09Z master 985370406d) [arm64-darwin22] == disasm: #<ISeq:<compiled>@<compiled>:1 (1,0)-(1,13)> 0000 putself ( 1)[Li] 0001 opt_send_without_block <calldata!mid:a, argc:0, FCALL|VCALL|ARGS_SIMPLE> 0003 definesmethod :foo, foo 0006 putobject :foo 0008 leave == disasm: #<ISeq:foo@<compiled>:1 (1,0)-(1,13)> 0000 putnil ( 1)[Ca] 0001 leave [Re] ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:112944] [Ruby master Feature#19541] Proposal: Generate frame unwinding info for YJIT code
by kjtsanaktsidis (KJ Tsanaktsidis) 19 Oct '23

19 Oct '23

Issue #19541 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Feature #19541: Proposal: Generate frame unwinding info for YJIT code https://bugs.ruby-lang.org/issues/19541 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal ---------------------------------------- ## What is being propsed? Currently, Ruby crashes with yjit generated code on the stack, `rb_print_backtrace()` is unable to actually show any frames underneath the yjit code. For example, if you send SIGSEGV to a Ruby process running yjit, this is what you see: ``` /ruby/miniruby(rb_print_backtrace+0xc) [0xaaaad0276884] /ruby/vm_dump.c:785 /ruby/miniruby(rb_vm_bugreport) /ruby/vm_dump.c:1093 /ruby/miniruby(rb_bug_for_fatal_signal+0xd0) [0xaaaad0075580] /ruby/error.c:813 /ruby/miniruby(sigsegv+0x5c) [0xaaaad01bedac] /ruby/signal.c:919 linux-vdso.so.1(__kernel_rt_sigreturn+0x0) [0xffff91a3e8bc] /ruby/miniruby(map<(usize, yjit::backend::ir::Insn), (usize, yjit::backend::ir::Insn), yjit::backend::ir::{impl#17}::next_mapped::{closure_env#0}>+0x8c) [0xaaaad03b8b00] /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/option.rs:929 /ruby/miniruby(next_mapped+0x3c) [0xaaaad0291dc0] src/backend/ir.rs:1225 /ruby/miniruby(arm64_split+0x114) [0xaaaad0287744] src/backend/arm64/mod.rs:359 /ruby/miniruby(compile_with_regs+0x80) [0xaaaad028bf84] src/backend/arm64/mod.rs:1106 /ruby/miniruby(compile+0xc4) [0xaaaad0291ae0] src/backend/ir.rs:1158 /ruby/miniruby(gen_single_block+0xe44) [0xaaaad02b1f88] src/codegen.rs:854 /ruby/miniruby(gen_block_series_body+0x9c) [0xaaaad03b0250] src/core.rs:1698 /ruby/miniruby(gen_block_series+0x50) [0xaaaad03b0100] src/core.rs:1676 /ruby/miniruby(branch_stub_hit_body+0x80c) [0xaaaad03b1f68] src/core.rs:2021 /ruby/miniruby({closure#0}+0x28) [0xaaaad02eb86c] src/core.rs:1924 /ruby/miniruby(do_call<yjit::core::branch_stub_hit::{closure_env#0}, *const u8>+0x98) [0xaaaad035ba3c] /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492 [0xaaaad035c9b4] ``` (n.b. - I compiled Ruby with `-fasynchronous-unwind-tables –rdynamic –g` in cflags to make sure gcc generates appropriate unwind info & keeps the symbol tables). Likewise, if you attach gdb to a Ruby process with yjit enabled, gdb can't show thread backtraces through yjit-generated code either. My proposal is that YJIT generate sufficient unwinding and debug information on all platforms to allow both `rb_print_backtrace()` and the platform's debugger (gdb/lldb/WinDbg) to show: * Full stack traces all the way back to `main`. That is, it should be possible to see frames _underneath_ `[0xaaaad035c9b4]` from the backtrace above. * Names for the dynamically generated yjit blocks (e.g. instead of `[0xaaaad035c9b4]`, we should see something like `yjit$$name_of_ruby_method`, where `name_of_ruby_method` is the `label` for the iseq this is JIT'd code for). ## Motivation I have a few motivations for wanting this. Firstly, I feel this functionality is independently useful. When Ruby crashes, the more information we can get, the more likely we are to find the root cause. Likewise, the same principle applies to debugging with gdb - you can get a fuller understanding of what the process is doing if you see the whole stack. I have often found attaching gdb to the Ruby interpreter helps in understanding problems in Ruby code or C extensions and is something I do relatively frequently; yjit breaking that will definitely be inconvenient for me! ## Implementation I have a draft implementation here on how I'd implement this: . It's currently missing tests & platform support (it only works on Linux aarch64). Also, it implements unwind info generation, so unwinding can work _through_ yjit code, but it does not currently emit symbols to give _names_ to those yjit frames. My PR contains a document which explains how the Linux interfaces for registering unwind info for JIT'd code work, so I won't duplicate that information here. The biggest implementation question I had is around the use of Rust crates. Currently, I prototyped my implementation using the gimli & object crates, for generating DWARF info and ELF binaries. However, the yjit build does purposefully does not use cargo & external crates for release builds. There are a few different ways we could go here: * Don't use the gimli & object crates; instead, re-implement all debug info & object file generation code in yjit. * Don't use the crates; instead, link againt C libraries to provide this functionality & call them from Rust (perhaps some combination of libelf, libdw, libbfd, or llvm might do what we need) * Use cargo after all for the release build & download the crates at build-time * Use cargo for the release build, but vendor everything, so the build doesn't need to download anything * Only make unwind info generation available in dev mode where cargo is used, and so mark the gimli/object dependencies as optional in Cargo.toml. We'd need to decide on one of these approaches for this proposal to work. I don't really have a strong sense of the pros/cons of each. (Side note - my PR actually depends on a _fork_ of gimli - I've been discussing adding the needed interfaces upstream here: https://github.com/gimli-rs/gimli/issues/648) ## Benchmarks I ran the yit-bench suite on my branch and compared it to Ruby master: * My branch: https://gist.github.com/KJTsanaktsidis/5741a9f64e5cd75cdf5fedd846091a4f * Ruby master: https://gist.github.com/KJTsanaktsidis/592d3ebcf98f6745dfa3efbd30a25acf This is a (very simple) comparison: ``` -------------- ------------ ------------ --------------- bench yjit (ms) branch (ms) branch/yjit (%) activerecord 97.5 98.5 101.03% hexapdf 2415.3 2458.2 101.78% liquid-c 61.9 63.1 101.94% liquid-render 135.3 135.0 99.78% mail 104.6 105.5 100.86% psych-load 1887.1 1922.0 101.85% railsbench 1544.4 1556.0 100.75% ruby-lsp 88.4 89.5 101.24% sequel 147.5 151.1 102.44% binarytrees 303 305.6 100.86% chunky_png 1075.8 1079.4 100.33% erubi 392.9 392.3 99.85% erubi_rails 14.7 14.7 100.00% etanni 792.3 791.4 99.89% fannkuchredux 3815.9 3813.6 99.94% lee 1030.2 1039.2 100.87% nbody 49.2 49.3 100.20% optcarrot 4142 4143.3 100.03% ruby-json 2860.7 2874.0 100.46% rubykon 7906.6 7904.2 99.97% 30k_ifelse 348.7 345.4 99.05% 30k_methods 828.6 831.8 100.39% cfunc_itself 28.8 28.9 100.35% fib 34.4 34.5 100.29% getivar 115.5 109.7 94.98% keyword_args 37.7 38.0 100.80% respond_to 26 26.1 100.38% setivar 33.8 33.5 99.11% setivar_object 208.7 194.3 93.10% str_concat 52.6 52.2 99.24% throw 23.8 24.1 101.26% -------------- ------------ ------------ --------------- ``` It seems like the performance impact of generating and registering the debug info is marginal. -- https://bugs.ruby-lang.org/

7 12

[ruby-core:112326] [Ruby master Feature#19430] Contribution wanted: DNS lookup by c-ares library
by mame (Yusuke Endoh) 18 Oct '23

18 Oct '23

Issue #19430 has been reported by mame (Yusuke Endoh). ---------------------------------------- Feature #19430: Contribution wanted: DNS lookup by c-ares library https://bugs.ruby-lang.org/issues/19430 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- ## Problem At the present time, Ruby uses `getaddrinfo(3)` to resolve names. Because this function is synchronous, we cannot interrupt the thread performing name resolution until the DNS server returns a response. We can see this behavior by setting blackhole.webpagetest.org (72.66.115.13) as a DNS server, which swallows all packets, and resolving any name: ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C^C^C^C ``` As we see, Ctrl+C does not stop ruby. The current workaround that users can take is to do name resolution in a Ruby thread. ```ruby Thread.new { Addrinfo.getaddrinfo("www.ruby-lang.org", 80) }.value ``` The thread that calls this code is interruptible. (Note that the newly created thread itself will be stuck until the DNS lookup exceeds the time out.) ## Proposal We can solve this problem by using c-ares, which is an asynchronous name resolver, as a backend of `Addrinfo.getaddrinfo`, etc. (@sorah told me about this library, thanks!) https://c-ares.org/ I have created a PoC patch. https://github.com/mame/ruby/commit/547806146993bbc25984011d423dcc0f913b211c By applying this patch, we can interrupt `Addrinfo.getaddrinfo` by Ctrl+C. ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C-e:1:in `getaddrinfo': Interrupt from -e:1:in `<main>' ``` ## Discussion ### About c-ares According to the site of c-ares, some major tools including libcurl, Wireshark, and Apache Arrow are already using c-ares. In the language interpreter, node.js seems to be using c-ares. I am honestly not sure about the compatibility of c-ares with `getaddrinfo(3)`. I guess there is no major incompatibility because I have not experienced any name resolution problem of curl. @akr (who is the author and maintainer of Ruby's socket library) suggested to check if OS-specific name resolution, e.g., WINS on Windows, NIS on Solaris, etc., is supported. He also said that it may be acceptable even if they are not supported. Whether to bundle c-ares source code with ruby would require further discussion. If this proposal is accepted, then c-ares will become a de facto essential dependency for practical use, like gmp, in my opinion. Incidentally, node.js bundles c-ares: https://github.com/nodejs/node/tree/main/deps/cares ### Alternative approaches Recent glibc provides `getaddrinfo_a(3)` which performs asynchronous name resolution. However, this function has a fatal problem of being incompatible with `fork(2)`, which is heavily used in the Ruby ecosystem. In fact, the attempt to use `getaddrinfo_a(3)` (#17134) has been revert because it fails rails tests. (#17220) Another alternative is to have a worker pthread inside Ruby that calls getaddrinfo(3). Instead of calling getaddrinfo(3) directly, `Addrinfo.getaddrinfo` would ask the worker to resolve a name and wait for a response. This method should be able to implement cancellation. (Simply put, this means reimplementation of getaddrinfo_a(3) on our own, taking into account of `fork(2).) This has the advantages: not adding dependencies on external libraries and not having compatibility issues with `getaddrinfo(3)`. However, it is considerably more difficult to implement and maintain. An internal pthread may have a non-trivial impact on the execution efficiency and memory usage. Also, we may need to implement a mechanism to dynamically change the number of workers depending on the load. It would be ideal if we could try and evaluate both approaches. But my current impression is that using c-ares is the quickest and best compromise. ## Contribution wanted I have made it up to the PoC, but don't have much time to complete this. @naruse suggested me to create a ticket asking for contributions. Is anyone interested in this? * This patch changes `rsock_getaddrinfo` to accept a timeout argument. There are several places where Qnil is passed as a timeout (where I add `// TODO` in the PoC). We need to consider what timeout we should pass. * This cares only `getaddrinfo`, but we also need to care `getnameinfo` (and something else if any). There may be some issues I'm not aware of. * I have not yet tested this PoC seriously. It would be great if we could evaluate it with some real apps. Also, it would be great to hear from someone who knows more about c-ares. -- https://bugs.ruby-lang.org/

6 15

[ruby-core:115082] [Ruby master Bug#19932] Regexp o modifier freeze interpolation between method calls
by noraj-acceis (Alexandre ZANNI) 18 Oct '23

18 Oct '23

Issue #19932 has been reported by noraj-acceis (Alexandre ZANNI). ---------------------------------------- Bug #19932: Regexp o modifier freeze interpolation between method calls https://bugs.ruby-lang.org/issues/19932 * Author: noraj-acceis (Alexandre ZANNI) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Taken the following PoC: ```ruby def poc(regexp) hs = [ 'azerty', 'azertyui', 'azertyuiop' ] out = [] hs.each do |h| out << h if /#{regexp}/.match?('azerty') end out end p poc('az') p poc('wxc') ``` I have the following output: ```ruby ["azerty", "azertyui", "azertyuiop"] [] ``` Now because the regexp never change inside the method once set I could add the `o` modifier so that `#{}` interpolation is computed only once. So the PoC becomes: ```ruby def poc(regexp) hs = [ 'azerty', 'azertyui', 'azertyuiop' ] out = [] hs.each do |h| out << h if /#{regexp}/o.match?('azerty') end out end p poc('az') p poc('wxc') ``` Output: ``` ["azerty", "azertyui", "azertyuiop"] ["azerty", "azertyui", "azertyuiop"] ``` So each future call to the method will be equals to the result of the first call because the regexp is "frozen". I was expecting the regexp to be "frozen" only for the current call in an effort of performance but instead it's "frozen" for future calls too. Another example of usage of `o` modifier here: https://learnbyexample.github.io/Ruby_Regexp/modifiers.html#o-modifier So is that a side effect or is that the expected behavior? -- https://bugs.ruby-lang.org/

2 4

[ruby-core:115070] [Ruby master Feature#19930] [Documentation] class Regexp: Character Classes ranges
by noraj-acceis (Alexandre ZANNI) 17 Oct '23

17 Oct '23

Issue #19930 has been reported by noraj-acceis (Alexandre ZANNI). ---------------------------------------- Feature #19930: [Documentation] class Regexp: Character Classes ranges https://bugs.ruby-lang.org/issues/19930 * Author: noraj-acceis (Alexandre ZANNI) * Status: Open * Priority: Normal ---------------------------------------- cf. https://ruby-doc.org/3.2.2/Regexp.html#class-Regexp-label-Character+Classes > POSIX bracket expressions are also similar to character classes. They provide a portable alternative to the above, with the added benefit that they encompass non-ASCII characters. For instance, /\d/ matches only the ASCII decimal digits (0-9); whereas /[[:digit:]]/ matches any character in the Unicode Nd category. Reading this description, we globally expect that metacharacters are ASCII only and that POSIX _bracket expressions_ are Unicode aware. But as _bracket expressions_ are POSIX compliant, for example `[:xdigit:]` use only ASCII range `[A-Fa-f0-9]` and not the `Hex_Digit` Unicode property that is also including the Halfwidth and Fullwidth Forms Number Decimal like `０` (U+FF10, FULLWIDTH DIGIT ZERO). So the above description is confusing as we would expect [[:xdigit:]]` to _encompass non-ASCII characters_ too. On the contrary `[:space:]` will look for `[\p{Z}\t\r\n\v\f]` (`\s` plus `\p{Z}` (Separator)) while the description is talking only about `[:blank:], newline, carriage return`. My point is, in the end, that it's hard to determine what to expect as ranges for character classes while reading the Ruby Regexp documentation alone. To know what is the exact behavior I'll have to read the source code or at least reading the POSIX spec. My feature request is about adding a comparison table like the one on https://www.regular-expressions.info/posixbrackets.html (for Java) with: the POSIX bracket expression, the description, the ASCII exact range, the Unicode exact range, the shorthand metacharacter (ASCII), the long escape sequence (Unicode). So we could know precisely what to expect by reading the doc. ---Files-------------------------------- Screenshot_20231017_154208.png (145 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114250] [Ruby master Bug#19778] mkmf.rb pkg_config() interaction with RbConfig::CONFIG["cflags"]
by rhenium (Kazuki Yamaguchi) 17 Oct '23

17 Oct '23

Issue #19778 has been reported by rhenium (Kazuki Yamaguchi). ---------------------------------------- Bug #19778: mkmf.rb pkg_config() interaction with RbConfig::CONFIG["cflags"] https://bugs.ruby-lang.org/issues/19778 * Author: rhenium (Kazuki Yamaguchi) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-07-21T09:38:29Z master 22f9735587) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- (This was first reported at https://github.com/ruby/openssl/issues/650: The extension's build breaks in a strange way if `RbConfig::CONFIG["*flags"]` contains the path of an OpenSSL installation but `pkg-config` returns the path of a different OpenSSL installation) Commit commit:097c3e9cbbf23718371f08c24b2d2297b039f63f ("mkmf.rb: -I flags to $INCFLAGS", Ruby 2.2) changed how mkmf's `pkg_config()` handles the result of the `pkg-config` command. It now stores the `-I` flags in $INCFLAGS and others in $CFLAGS. mkmf generates a Makefile that compiles source files with: $(CC) $(INCFLAGS) $(CPPFLAGS) $(CFLAGS) and link the final library with: $(LDSHARED) -o $@ $(OBJS) $(LIBPATH) $(DLDFLAGS) $(LOCAL_LIBS) $(LIBS) This "new" behavior of `pkg_config()` is problematic when `RbConfig::CONFIG["{C,CPP}FLAGS"]` also provide `-I` flags and `RbConfig::CONFIG["LDFLAGS"]` provides the matching `-L` flags -- for example, if Ruby is compiled with `./configure --with-opt-dir=<dir>`. This would end up with compiling sources with [...] -I<from pkg-config> -I<from RbConfig> [...] and then linking with [...] -L<from RbConfig> -L<from pkg-config> [...] This doesn't seem right. I don't know which should come earlier, but the order should be consistent. The commit in question clearly describes the change in the commit message, but it doesn't have a linked issue. What is it intended for? Also, what is $INCFLAGS? On the other hand, `dir_config()` would prepend `-I` flags to $CPPFLAGS and `-L` flags to $LIBPATH, so it doesn't have issues with flags from `RbConfig`, albeit in a differently way. -- https://bugs.ruby-lang.org/

5 4