November 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:115358] [Ruby master Feature#11322] OpenUri: RuntimeError: HTTP redirection loop
by hsbt (Hiroshi SHIBATA) 13 Nov '23

13 Nov '23

Issue #11322 has been updated by hsbt (Hiroshi SHIBATA). https://github.com/ruby/open-uri/pull/18 is another approach with `max_redirects` option. ---------------------------------------- Feature #11322: OpenUri: RuntimeError: HTTP redirection loop https://bugs.ruby-lang.org/issues/11322#change-105282 * Author: tbsprs (Tobias Preuss) * Status: Assigned * Priority: Normal * Assignee: akr (Akira Tanaka) ---------------------------------------- Trying to download [this file](http://apps.london.ca/OpenData/ShapeFiles_Zipped/2010_skateboard_parks_shp.zip) from [this website](http://www.london.ca/city-hall/open-data/Pages/Open-Data-Data-Catalogue.aspx) with [`OpenUri`](http://ruby-doc.org/stdlib-2.2.2/libdoc/open-uri/rdoc/OpenURI.html) fails with the runtime error "HTTP redirection loop". Here is how I can reproduce the error: ~~~ > require 'open-uri' => true > open('http://apps.london.ca/OpenData/ShapeFiles_Zipped/2010_skateboard_parks_shp.…') RuntimeError: HTTP redirection loop: http://apps.london.ca/uniquesig87fdc01fb86ce6f0fd235c713015d7d7/uniquesig0/… from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:232:in `open_loop' from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:150:in `open_uri' from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:716:in `open' from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:34:in `open' from (irb):2 from /home/john/.rvm/rubies/ruby-2.2.2/bin/irb:11:in `<main>' ~~~ ---Files-------------------------------- open_uri-redirect-cookie-11322.patch (2.3 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115349] [Ruby master Feature#20002] Print C-level backtrace on uncaught Ruby exceptions
by osyoyu (Daisuke Aritomo) 12 Nov '23

12 Nov '23

Issue #20002 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Feature #20002: Print C-level backtrace on uncaught Ruby exceptions https://bugs.ruby-lang.org/issues/20002 * Author: osyoyu (Daisuke Aritomo) * Status: Open * Priority: Normal ---------------------------------------- When a uncaught Ruby exception occurs, the Ruby-level backtrace is printed. While this information is valuable for debugging Ruby code, this isn't necessarily the case when developing C extensions, since this message has no information about C code. In this example, the printed message only tells us that something is wrong inside `will_raise`. There are not much ways to debug further; the programmer must guess which Ruby API call is causing the exception. A C-level backtrace which includes the `VALUE x = NUM2INT(rb_str_new_cstr("1"));` line would be very helpful here. ``` % ruby main.rb main.rb:9:in `will_raise': no implicit conversion of String into Integer (TypeError) will_raise ^^^^^^^^^^ from main.rb:9:in `bar' from main.rb:4:in `foo' from main.rb:12:in `<main> ``` ```ruby require_relative "./ext" def foo bar end def bar will_raise end foo ``` ```c #include <ruby/ruby.h> VALUE will_raise(VALUE self) { // Do many things // There is no information leading to this line in the printed message VALUE x = NUM2INT(rb_str_new_cstr("1")); // no implicit conversion of String into Integer (TypeError) return Qtrue; } void Init_ext(void) { rb_define_method(rb_mKernel, "will_raise", will_raise, 0); } ``` I'm imagining something similar to the segfault report. In this case, it is clear that some code caused a SIGSEGV in `ext.c:12` (which is `int *p = NULL; *p = 1;`). ``` -- C level backtrace information ------------------------------------------- /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_print_backtrace+0x14) [0x55dd24893174] /home/osyoyu/ruby/build/../vm_dump.c:812 /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_vm_bugreport) /home/osyoyu/ruby/build/../vm_dump.c:1140 /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_bug_for_fatal_signal+0x14c) [0x55dd24a064cc] /home/osyoyu/ruby/build/../error.c:1065 /home/osyoyu/.rbenv/versions/master/bin/ruby(sigsegv+0x51) [0x55dd24801c01] /home/osyoyu/ruby/build/../signal.c:920 /usr/lib/libc.so.6(0x7f6926558710) [0x7f6926558710] /home/osyoyu/ruby-bug/ext.so(will_segv+0x14) [0x7f69264fc244] /home/osyoyu/ruby-bug/ext.c:12 /home/osyoyu/ruby-bug/ext.so(will_segv) (null):0 /home/osyoyu/.rbenv/versions/master/bin/ruby(vm_call_cfunc_with_frame_+0xf4) [0x55dd24885d64] ../vm_insnhelper.c:3477 /home/osyoyu/.rbenv/versions/master/bin/ruby(vm_sendish+0xdd) [0x55dd2486aee1] ../vm_insnhelper.c:5555 /home/osyoyu/.rbenv/versions/master/bin/ruby(vm_exec_core) ../insns.def:835 ruby(vm_exec_loop+0x0) [0x55dd24867c52] /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_vm_exec) /home/osyoyu/ruby/build/../vm.c:2420 /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_ec_exec_node+0x2b) [0x55dd246e9fab] /home/osyoyu/ruby/build/../eval.c:287 /home/osyoyu/.rbenv/versions/master/bin/ruby(ruby_run_node) /home/osyoyu/ruby/build/../eval.c:328 /home/osyoyu/.rbenv/versions/master/bin/ruby(rb_main+0x1c) [0x55dd246e7a85] /home/osyoyu/ruby/build/../main.c:39 /home/osyoyu/.rbenv/versions/master/bin/ruby(main) /home/osyoyu/ruby/build/../main.c:58 /usr/lib/libc.so.6(0x7f6926541cd0) [0x7f6926541cd0] /usr/lib/libc.so.6(__libc_start_main+0x8a) [0x7f6926541d8a] [0x55dd246e7935] ``` As most Ruby users might not be interested in C-level information, this behavior could be made optional through startup options or environment variables. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:115351] Ruby 3.3.0-preview3 Released
by NARUSE, Yui 12 Nov '23

12 Nov '23

We are pleased to announce the release of Ruby 3.3.0-preview3. Ruby 3.3 adds a new parser named Prism, uses Lrama as a parser generator, adds a new pure-Ruby JIT compiler named RJIT, and many performance improvements especially YJIT. ## Prism * Introduced [the Prism parser](https://github.com/ruby/prism) as a default gem * Prism is a portable, error tolerant, and maintainable recursive descent parser for the Ruby language * Prism is production ready and actively maintained, you can use it in place of Ripper * There is [extensive documentation](https://ruby.github.io/prism/) on how to use Prism * Prism is both a C library that will be used internally by CRuby and a Ruby gem that can be used by any tooling which needs to parse Ruby code * Notable methods in the Prism API are: * `Prism.parse(source)` which returns the AST as part of a ParseResult * `Prism.dump(source, filepath)` which returns the serialized AST as a String * `Prism.parse_comments(source)` which returns the comments * You can make pull requests or issues directly on [the Prism repository](https://github.com/ruby/prism) if you are interested in contributing ## Use Lrama instead of Bison * Replace Bison with [Lrama LALR parser generator](https://github.com/yui-knk/lrama) [Feature #19637](https://bugs.ruby-lang.org/issues/19637) * If you have interest, please see [The future vision of Ruby Parser](https://rubykaigi.org/2023/presentations/spikeolaf.html) * Lrama internal parser is replaced with LR parser generated by Racc for maintainability * Parameterizing Rules `(?, *, +)` are supported, it will be used in Ruby parse.y ## RJIT * Introduced a pure-Ruby JIT compiler RJIT and replaced MJIT. * RJIT supports only x86-64 architecture on Unix platforms. * Unlike MJIT, it doesn't require a C compiler at runtime. * RJIT exists only for experimental purposes. * You should keep using YJIT in production. * If you are interested in developing JIT for Ruby, please check out [k0kubun's presentation on Day 3 of RubyKaigi](https://rubykaigi.org/2023/presentations/k0kubun.html#day3). ## YJIT * Major performance improvements over 3.2 * Support for splat and rest arguments has been improved. * Registers are allocated for stack operations of the virtual machine. * More calls with optional arguments are compiled. * Exception handlers are also compiled. * Instance variables no longer exit to the interpreter with megamorphic object shapes. * Unsupported call types no longer exit to the interpreter. * `Integer#!=`, `String#!=`, `Kernel#block_given?`, `Kernel#is_a?`, `Kernel#instance_of?`, `Module#===` are specially optimized. * Now more than 3x faster than the interpreter on optcarrot! * Significantly improved memory usage over 3.2 * Metadata for compiled code uses a lot less memory. * Generate more compact code on ARM64 * Compilation speed is now slightly faster than 3.2. * Add `RubyVM::YJIT.enable` that can enable YJIT at run-time * You can start YJIT without modifying command-line arguments or environment variables. * This can also be used to enable YJIT only once your application is done booting. `--yjit-disable` can be used if you want to use other YJIT options while disabling YJIT at boot. * Option to disable code GC and treat `--yjit-exec-mem-size` as a hard limit * Can produce better copy-on-write behavior on servers using unicorn and forking * `ratio_in_yjit` stat produced by `--yjit-stats` is now avaiable in release builds, a special stats or dev build is no longer required to access most stats. * Exit tracing option now supports sampling * `--trace-exits-sample-rate=N` * `--yjit-perf` is added to facilitate profiling with Linux perf. * More thorough testing and multiple bug fixes ### M:N thread scheduler * M:N thread scheduler was introduced. [[Feature #19842]](https://bugs.ruby-lang.org/issues/19842) * M Ruby threads are managed by N native threads (OS threads) so the thread creation and management cost are reduced. * It can break C-extension compatibility so that M:N thread scheduler is disabled on the main Ractor by default. * `RUBY_MN_THREADS=1` environment variable enables M:N threads on the main Ractor. * M:N threads are enabled on non-main Ractors. * `RUBY_MAX_CPU=n` environment variable sets maximum number of `N` (maximum number of native threads). The default value is 8. * Since only one Ruby thread per Ractor can run at the same time, the number of native threads will be used, which is the smaller of the number specified in `RUBY_MAX_CPU` and the number of running Ractors. So that single Ractor applications (most of applications) will use 1 native thread. * To support blocking operations, more than `N` native threads can be used. ## Other Notable New Features ### Language ## Performance improvements * `defined?(@ivar)` is optimized with Object Shapes. * Name resolution such as `Socket.getaddrinfo` can now be interrupted (in environments where pthreads are available). [Feature #19965](https://bugs.ruby-lang.org/issues/19965) * For this purpose, a pthread is now created whenever calling getaddrinfo or getnameinfo. This incurs some overhead in name resolution (about 2.5x in our experiments). We do not expect the name resolution overhead to be a problem for most applications, but if you observe such, or if you see unexpected affects that you believe are due to this change, please report them. * Environment variable `RUBY_GC_HEAP_REMEMBERED_WB_UNPROTECTED_OBJECTS_LIMIT_RATIO` has been added. [Feature #19571](https://bugs.ruby-lang.org/issues/19571) * Children of old objects are no longer immedately promoted to the old generation in the garbage collector. [Feature #19678](https://bugs.ruby-lang.org/issues/19678) * Support for weak references has been added to the garbage collector. [Feature #19783](https://bugs.ruby-lang.org/issues/19783) ## Other notable changes since 3.2 ### IRB IRB has received several enhancements, including but not limited to: - Advanced `irb:rdbg` integration that provides an equivalent debugging experience to `pry-byebug` ([doc](https://github.com/ruby/irb#debugging-with-irb)) - Pager support for `ls`, `show_source` and `show_cmds` commands. - More accurate and helpful information provided by the `ls` and `show_source` commands. - Experimental autocompletion using type analysis ([doc](https://github.com/ruby/irb#type-based-completion)) - It is now possible to change the font color and font style in the completion dialog by a newly introduced class Reline::Face ([doc](https://github.com/ruby/ruby/blob/master/doc/reline/face.md)) In addition, IRB has also undergone extensive refactoring and received dozens of bug fixes to facilitate easier future enhancements. ## Compatibility issues Note: Excluding feature bug fixes. ### Removed constants The following deprecated constants are removed. ### Removed methods The following deprecated methods are removed. ### Removed environment variables The following deprecated methods are removed. * Environment variable `RUBY_GC_HEAP_INIT_SLOTS` has been deprecated and is a no-op. Please use environment variables `RUBY_GC_HEAP_{0,1,2,3,4}_INIT_SLOTS` instead. [Feature #19785](https://bugs.ruby-lang.org/issues/19785) ## Stdlib compatibility issues ### `ext/readline` is retired * We have `reline` that is pure Ruby implementation compatible with `ext/readline` API. We rely on `reline` in the future. If you need to use `ext/readline`, you can install `ext/readline` via rubygems.org with `gem install readline-ext`. * We no longer need to install libraries like `libreadline` or `libedit`. ## C API updates ### Updated C APIs The following APIs are updated. ### Removed C APIs The following deprecated APIs are removed. ## Standard library updates RubyGems and Bundler warn if users require gem that is scheduled to become the bundled gems in the future version of Ruby. Targeted libraries are: * abbrev * base64 * bigdecimal * csv * drb * getoptlong * mutex_m * nkf * observer * racc * resolv-replace * rinda * syslog The following default gem is added. * prism 0.15.1 The following default gems are updated. * RubyGems 3.5.0.dev * base64 0.2.0 * benchmark 0.3.0 * bigdecimal 3.1.5 * bundler 2.5.0.dev * cgi 0.4.0 * csv 3.2.8 * date 3.3.4 * delegate 0.3.1 * drb 2.2.0 * english 0.8.0 * erb 4.0.3 * etc 1.4.3.dev.1 * fcntl 1.1.0 * fiddle 1.1.2 * fileutils 1.7.2 * find 0.2.0 * getoptlong 0.2.1 * io-console 0.6.1.dev * irb 1.8.3 * logger 1.6.0 * mutex_m 0.2.0 * net-http 0.4.0 * net-protocol 0.2.2 * nkf 0.1.3 * observer 0.1.2 * open-uri 0.4.0 * open3 0.2.0 * openssl 3.2.0 * optparse 0.4.0 * ostruct 0.6.0 * pathname 0.3.0 * pp 0.5.0 * prettyprint 0.2.0 * pstore 0.1.3 * psych 5.1.1.1 * rdoc 6.6.0 * reline 0.3.9 * rinda 0.2.0 * securerandom 0.3.0 * shellwords 0.2.0 * singleton 0.2.0 * stringio 3.0.9 * strscan 3.0.7 * syntax_suggest 1.1.0 * tempfile 0.2.0 * time 0.3.0 * timeout 0.4.1 * tmpdir 0.2.0 * tsort 0.2.0 * un 0.3.0 * uri 0.13.0 * weakref 0.1.3 * win32ole 1.8.10 * yaml 0.3.0 * zlib 3.1.0 The following bundled gem is promoted from default gems. * racc 1.7.3 The following bundled gems are updated. * minitest 5.20.0 * rake 13.1.0 * test-unit 3.6.1 * rexml 3.2.6 * rss 0.3.0 * net-imap 0.4.4 * net-smtp 0.4.0 * rbs 3.2.2 * typeprof 0.21.8 * debug 1.8.0 See GitHub releases like [Logger](https://github.com/ruby/logger/releases) or changelog for details of the default gems or bundled gems. See [NEWS](https://github.com/ruby/ruby/blob/v3_3_0_preview3/NEWS.md) or [commit logs](https://github.com/ruby/ruby/compare/v3_2_0...v3_3_0_preview3) for more details. With those changes, [5207 files changed, 284820 insertions(+), 174773 deletions(-)](https://github.com/ruby/ruby/compare/v3_2_0...v3_3_0_preview3… since Ruby 3.2.0! ## Download * <https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.0-preview3.tar.gz> SIZE: 21550473 SHA1: 2811f191d66dffee0206771873bd990857ae4ed6 SHA256: 0969141be92e67e0edb84a8fb354acc98f01bd78e602a23a0f136045c82f4809 SHA512: 94db07a6958c09809b2e5b597fa55a121074e8bacb3bf588c83cf0d35b07a8b070172035a49d1abf0d8ee364a9ace824f34e677f7327ffe1acdbab0938ac49c4 * <https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.0-preview3.tar.xz> SIZE: 15970144 SHA1: 496600612605f8ebeb955255e98bac73a4cbc045 SHA256: f79afcf122dc7d04fe26cfa4436b9c488b21766fc54b0d2dfb2ba41cd0cdd355 SHA512: d7ab0d703e7884efd31045933409cd68fac1d9941963537ccc8e309ca7c8bee8500a68182135acba22cbdbf4a8ae99f39bf7f0925273eb4fbc3728c0c1ba0c19 * <https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.0-preview3.zip> SIZE: 26618303 SHA1: 6a13e08c7e484d42037c1e2c87c5d0e220f893a0 SHA256: c35bf637a647c2f60148368ffb374db5c258570911794f46b6dfdb98ebfe95d9 SHA512: c4ef2cdcdadeb85ad1d42aedc97f9f3d609b3b01eea2319451cf92c81bd31ae8129b7c91fc68571469dd888c01ea0f48f73234b965db17f6a87404ca382f7794 ## What is Ruby Ruby was first developed by Matz (Yukihiro Matsumoto) in 1993, and is now developed as Open Source. It runs on multiple platforms and is used all over the world especially for web development.

1 0

[ruby-core:115307] [Ruby master Bug#19994] Seemingly require causes segfault??
by mtasaka (Mamoru Tasaka) 12 Nov '23

12 Nov '23

Issue #19994 has been reported by mtasaka (Mamoru Tasaka). ---------------------------------------- Bug #19994: Seemingly require causes segfault?? https://bugs.ruby-lang.org/issues/19994 * Author: mtasaka (Mamoru Tasaka) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-11-08 master c8d4b103a9) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- With commit:c8d4b103a9 , trying to execute stringex-2.8.6 https://github.com/rsl/stringex/releases/tag/v2.8.6 testsuite seems to cause ruby segfault. Trying to minimize code, the following causes segfault for almost all time (about 90%): ``` require 'test/unit' require 'active_record' ActiveRecord::Base.establish_connection adapter: "sqlite3", database: ":memory:" ``` (strangely, with commenting out the first line i.e. `require 'test/unit'` segfault no longer ocuurs...) Backtrace: ``` Thread 1 "ruby-mri" received signal SIGSEGV, Segmentation fault. vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670b78, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3499 3499 int len = cfunc->argc; Missing separate debuginfos, use: dnf debuginfo-install glibc-2.38.9000-17.fc40.x86_64 gmp-6.2.1-5.fc39.x86_64 libgcc-13.2.1-4.fc40.x86_64 libxcrypt-4.4.36-2.fc39.x86_64 libyaml-0.2.5-12.fc39.x86_64 openssl-libs-3.1.4-1.fc40.x86_64 rubygem-json-2.6.3-204.fc40.140.x86_64 rubygem-sqlite3-1.6.1-2.fc40.140.x86_64 sqlite-libs-3.44.0-1.fc40.x86_64 zlib-1.2.13-5.fc40.x86_64 (gdb) bt #0 vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670b78, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3499 #1 0x00007ffff7c5b62c in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:5581 #2 vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:822 #3 0x00007ffff7c739dd in rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2472 #4 0x00007ffff7c61463 in vm_yield_with_cref (is_lambda=0, cref=0x0, kw_splat=0, argv=0x0, argc=0, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:1622 #5 vm_yield (kw_splat=0, argv=0x0, argc=0, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:1630 #6 rb_yield_0 (argv=0x0, argc=0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_eval.c:1362 #7 rb_yield_values2 (argc=0, argv=0x0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_eval.c:1408 #8 0x00007ffff7ad3b4f in rb_ensure (b_proc=0x7fffde0fb990 <monitor_sync_body>, data1=140736888155080, e_proc=0x7fffde0fb6e0 <monitor_sync_ensure>, data2=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/eval.c:1009 #9 0x00007ffff7c540c7 in vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670c20, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3503 #10 0x00007ffff7c56641 in vm_sendish (ec=0x55555555d1c0, reg_cfp=0x7ffff7670c20, cd=<optimized out>, block_handler=<optimized out>, method_explorer=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_callinfo.h:407 #11 0x00007ffff7c5d43f in vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:802 #12 0x00007ffff7c739dd in rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2472 #13 0x00007ffff7b358fb in require_internal (ec=ec@entry=0x55555555d1c0, fname=<optimized out>, fname@entry=140736887218320, exception=exception@entry=1, warn=false) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1247 #14 0x00007ffff7b35e4e in rb_require_string_internal (fname=140736887218320) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1346 #15 0x00007ffff7b35f08 in rb_require_string (fname=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1339 #16 0x00007ffff7c540c7 in vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670cc8, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3503 #17 0x00007ffff7c57b1f in vm_call_alias (ec=0x55555555d1c0, cfp=0x7ffff7670cc8, calling=0x7fffffffce00) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3873 #18 0x00007ffff7c5b62c in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:5581 #19 vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:822 #20 0x00007ffff7c739dd in rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2472 #21 0x00007ffff7b358fb in require_internal (ec=ec@entry=0x55555555d1c0, fname=<optimized out>, fname@entry=140736885373720, exception=exception@entry=1, warn=false) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1247 #22 0x00007ffff7b35e4e in rb_require_string_internal (fname=140736885373720) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1346 #23 0x00007ffff7b35f08 in rb_require_string (fname=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1339 #24 0x00007ffff7c540c7 in vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670d70, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3503 #25 0x00007ffff7c57b1f in vm_call_alias (ec=0x55555555d1c0, cfp=0x7ffff7670d70, calling=0x7fffffffd3f0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3873 #26 0x00007ffff7c5b62c in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:5581 #27 vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:822 #28 0x00007ffff7c73bfe in vm_exec_loop (result=<optimized out>, tag=0x7fffffffd480, state=<optimized out>, ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2499 #29 rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2478 #30 0x00007ffff7b358fb in require_internal (ec=ec@entry=0x55555555d1c0, fname=<optimized out>, fname@entry=140736888126320, exception=exception@entry=1, warn=false) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1247 #31 0x00007ffff7b35e4e in rb_require_string_internal (fname=140736888126320) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1346 #32 0x00007ffff7b35f08 in rb_require_string (fname=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1339 #33 0x00007ffff7c540c7 in vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670e18, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3503 #34 0x00007ffff7c57b1f in vm_call_alias (ec=0x55555555d1c0, cfp=0x7ffff7670e18, calling=0x7fffffffd9e0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3873 #35 0x00007ffff7c5b62c in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:5581 --Type <RET> for more, q to quit, c to continue without paging-- #36 vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:822 #37 0x00007ffff7c739dd in rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2472 #38 0x00007ffff7b358fb in require_internal (ec=ec@entry=0x55555555d1c0, fname=<optimized out>, fname@entry=140736882997040, exception=exception@entry=1, warn=false) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1247 #39 0x00007ffff7b35e4e in rb_require_string_internal (fname=140736882997040) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1346 #40 0x00007ffff7b35f08 in rb_require_string (fname=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/load.c:1339 #41 0x00007ffff7c540c7 in vm_call_cfunc_with_frame_ (ec=0x55555555d1c0, reg_cfp=0x7ffff7670ec0, calling=<optimized out>, argc=<optimized out>, argv=<optimized out>, stack_bottom=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3503 #42 0x00007ffff7c57193 in vm_call_alias (calling=0x7fffffffe040, cfp=0x7ffff7670ec0, ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:3873 #43 vm_call_method_each_type (ec=0x55555555d1c0, cfp=0x7ffff7670ec0, calling=0x7fffffffe040) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:4475 #44 0x00007ffff7c5b62c in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm_insnhelper.c:5581 #45 vm_exec_core (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/redhat-linux-build/insns.def:822 #46 0x00007ffff7c73bfe in vm_exec_loop (result=<optimized out>, tag=0x7fffffffe0d0, state=<optimized out>, ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2499 #47 rb_vm_exec (ec=0x55555555d1c0) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/vm.c:2478 #48 0x00007ffff7adbc0a in rb_ec_exec_node (ec=ec@entry=0x55555555d1c0, n=n@entry=0x7fffdc3de580) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/eval.c:287 #49 0x00007ffff7addb03 in ruby_run_node (n=0x7fffdc3de580) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/eval.c:328 #50 0x0000555555555197 in rb_main (argv=0x7fffffffe4a8, argc=2) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/main.c:39 #51 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/ruby-3.3.0~20231108.0143gitc8d4b103a9-183.fc40.285.x86_64/main.c:58 (gdb) p cfunc $1 = (const rb_method_cfunc_t *) 0xb (gdb) deta Detaching from program: /usr/bin/ruby-mri, process 345 /usr/share/rubygems/rubygems.rb:1220: [BUG] Segmentation fault at 0x000000000000001b ruby 3.3.0dev (2023-11-08 master c8d4b103a9) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0022 p:---- s:0126 e:000125 CFUNC c:0021 p:0007 s:0123 e:000120 METHOD /usr/share/rubygems/rubygems.rb:1220 c:0020 p:0015 s:0115 e:000114 BLOCK <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:42 [FINISH] c:0019 p:---- s:0104 e:000103 [Inferior 1 (process 345) detached] CFUNC :synchronize c:0018 p:0017 s:0100 e:000099 METHOD <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:39 c:0017 p:0006 s:0094 e:000093 TOP /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3/pragmas.rb:1 [FINISH] c:0016 p:---- s:0091 e:000090 CFUNC :require c:0015 p:0023 s:0086 e:000085 METHOD <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128 c:0014 p:0020 s:0080 e:000079 TOP /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3/database.rb:3 [FINISH] c:0013 p:---- s:0077 e:000076 CFUNC :require c:0012 p:0023 s:0072 e:000071 METHOD <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128 c:0011 p:0033 s:0066 e:000065 TOP /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3.rb:9 [FINISH] c:0010 p:---- s:0063 e:000062 CFUNC :require c:0009 p:0023 s:0058 e:000057 METHOD <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128 c:0008 p:0078 s:0052 e:000051 TOP /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/sqlite3_adapter.rb:14 [FINISH] c:0007 p:---- s:0049 e:000048 CFUNC :require c:0006 p:0023 s:0044 e:000043 METHOD <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128 c:0005 p:0051 s:0038 e:000037 METHOD /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_handler.rb:268 c:0004 p:0066 s:0027 e:000026 METHOD /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_handler.rb:129 c:0003 p:0049 s:0013 e:000012 METHOD /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_handling.rb:52 c:0002 p:0020 s:0006 e:000005 EVAL ./trial.rb:4 [FINISH] c:0001 p:0000 s:0003 E:0018a0 DUMMY [FINISH] -- Ruby level backtrace information ---------------------------------------- ./trial.rb:4:in `<main>' /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_handling.rb:52:in `establish_connection' /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_handler.rb:129:in `establish_connection' /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_handler.rb:268:in `resolve_pool_config' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' /usr/share/gems/gems/activerecord-7.0.8/lib/active_record/connection_adapters/sqlite3_adapter.rb:14:in `<top (required)>' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3.rb:9:in `<top (required)>' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3/database.rb:3:in `<top (required)>' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:128:in `require' /usr/share/gems/gems/sqlite3-1.6.1/lib/sqlite3/pragmas.rb:1:in `<top (required)>' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:39:in `require' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:39:in `synchronize' <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:42:in `block in require' /usr/share/rubygems/rubygems.rb:1220:in `find_unresolved_default_spec' ``` Looks like `cfunc` points to invalid address. Currently I am unable to minimize the code further... -- https://bugs.ruby-lang.org/

3 6

[ruby-core:115345] [Ruby master Feature#18996] Proposal: Introduce new APIs to reline for changing dialog UI colours
by st0012 (Stan Lo) 11 Nov '23

11 Nov '23

Issue #18996 has been updated by st0012 (Stan Lo). @hasumikin proposed a more powerful and mature design and it has been merged. So I think both this and #19010 can be closed :-) ---------------------------------------- Feature #18996: Proposal: Introduce new APIs to reline for changing dialog UI colours https://bugs.ruby-lang.org/issues/18996#change-105270 * Author: st0012 (Stan Lo) * Status: Open * Priority: Normal ---------------------------------------- ### TL;DR I want to add APIs to `reline` for changing its dialog item's colors. The APIs I want to add actually have been merged but becaue: 1. This is a design change 2. The maintainer @aycabta is not available to approve nor reject them I want to raise it here to decide if we should: 1. Drop them 2. Modify them 3. Officiallty accept them ### Background After version `1.4`, `irb` provides autocompletion support, which is a great feature and has increased many developers' productivity significantly. But there's an user-experience issue: the completion items' UI colors (set in `reline`) [are not configurable](https://github.com/ruby/reline/blob/9ab5850444b49aff8e360a84e…. So depending on the user's terminal theme, some may find it hard to use because the background and the text having low-contrast colors, like this: ![](https://user-images.githubusercontent.com/3303032/148653612-e3dff786-1a10-4923-a0eb-3975cae10a7f.png) And if that happens, the user has no way to fix it. This caused users to open issues like: - https://github.com/ruby/irb/issues/351 - https://github.com/ruby/irb/issues/328 for being able to change it. Some users even decided to disable it completely because the colors are unreadable to them. I have also seen people sharingtips for disabling this feature: [example](https://twitter.com/sdogruyol/status/1538512030449254400). So I believe it may be bothering many developers. Personally I really like this feature but the background also bothers me: ![Screenshot 2022-09-07 at 22 55 12](https://user-images.githubusercontent.com/5079556/188990620-5ec7ba0c-97… And that's why I want to improve it by making the colors configurable and potentially also by providing simple light/dark themes from `irb`. ### Proposal For the dialog UI, there are 2 element states: `highlighted` and `default`. In `irb`'s case, the selected completion candidate will be `highlighted`, and the rest of options will be `default`. And each state has 2 colors: `foreground (text)` and `background (block)`. This means the `reline` should allow `irb` and/or users to configure: - Default items' foreground color - Default items' background color - Highlighted items' foreground color - Highlighted items' background color That brings us to these APIs: - `Reline.dialog_default_fg_color` - `Reline.dialog_default_bg_color` - `Reline.dialog_highlight_fg_color` - `Reline.dialog_highlight_bg_color` And because `reline` only supports coloring through ANSI sequences, these APIs only has 8 available colors if we exclude their bright variants: - Black - Red - Green - Yellow - Blue - Magenta - Cyan - White Given the limited options and also to prevent users from entering non-color ANSI sequences, these APIs only take color names directly: - :black - :red - :green - :yellow - :blue - :magenta - :cyan - :white Example: ```rb Reline.dialog_default_bg_color = :black puts Reline.dialog_default_bg_color_sequence #=> 40 Reline.dialog_default_fg_color = :white puts Reline.dialog_default_fg_color_sequence #=> 37 Reline.dialog_highlight_bg_color = :blue puts Reline.dialog_highlight_bg_color_sequence #=> 34 Reline.dialog_highlight_fg_color = :black puts Reline.dialog_highlight_fg_color_sequence #=> 30 ``` I have made a [proof of concept PR](https://github.com/ruby/irb/pull/380) on `irb` to show what these APIs can achieve if they or similar ones are adopted. #### Related PRs The related changes are made through multiple PRs: - [Initial APIs PR by @pocari](https://github.com/ruby/reline/pull/413) - [PR to improve the APIs and make them safer to use](https://github.com/ruby/reline/pull/454) - [PR to rename the APIs](https://github.com/ruby/reline/pull/456) #### Other Thoughts This is more of a concern on the `irb` part, but to make the UI looks comfortable, I think it's better to follow these conditions: 1. An item's foreground and background colors should have high contrast with each other so the texts (foreground) are readable. 2. For the `highlighted` item, its background color should be easily distinguishable from the rest of `default` items. 3. When using dark terminal themes, the `default` items' background is better to be dark as well. ---Files-------------------------------- Screenshot 2022-10-18 at 15.25.36.png (9.7 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115339] [Ruby master Bug#20000] Backport: Fix OpenSSL.fips_mode and OpenSSL::PKey.read in OpenSSL 3 FIPS.
by jaruga (Jun Aruga) 10 Nov '23

10 Nov '23

Issue #20000 has been reported by jaruga (Jun Aruga). ---------------------------------------- Bug #20000: Backport: Fix OpenSSL.fips_mode and OpenSSL::PKey.read in OpenSSL 3 FIPS. https://bugs.ruby-lang.org/issues/20000 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- Ruby 3.3 (master branch) includes Ruby OpenSSL library (openssl gem) to fix OpenSSL.fips_mode and OpenSSL::PKey.read in OpenSSL 3 FIPS. And I want to see the following 5 commits to fix the issues will be backported to Ruby 3.2, 3.1 and 3.0. * Fix OpenSSL.fips_mode in OpenSSL 3 FIPS. https://github.com/ruby/openssl/pull/608 https://github.com/ruby/ruby/commit/678d41bc51fe31834eec0b653ba0e47de5420aa0 * Fix OpenSSL::PKey.read in OpenSSL 3 FIPS. The following 2 commits modify the `ossl_pkey_read_generic`. Maybe it's easy to combine and squash the 2 commits to 1 commit to backport. * https://github.com/ruby/openssl/pull/615 https://github.com/ruby/ruby/commit/2a4834057b30a26c38ece3961b370c0b2ee59380 * https://github.com/ruby/openssl/pull/669 https://github.com/ruby/ruby/commit/b0ec1db8a72c530460abd9462ac75845362886bd * Enable tests in OpenSSL FIPS. This commit is needed to run the tests properly in OpenSSL FIPS. * https://github.com/ruby/openssl/pull/615 https://github.com/ruby/ruby/commit/920bc71284f417f9044b0dc1822b1d29a8fc61e5 * ssl: use ffdhe2048 from RFC 7919 as the default DH group parameters https://github.com/ruby/openssl/pull/674 https://github.com/ruby/ruby/commit/b6d7cdc2bad0eadbca73f3486917f0ec7a475814 ## Ruby and included Ruby OpenSSL (ruby/openssl) version Here is the bundled ruby/openssl version for each Ruby. You can check the version number in the `ext/openssl/lib/openssl/version.rb`. The only ruby/openssl version 3.2.0 includes the 5 commits above. * Ruby 3.3 * master: 3.2.0 * Ruby 3.2: * The brnach ruby_3_2: 3.1.0 * The latest patch version tag v3_2_2: 3.1.0 * Ruby 3.1: * The branch ruby_3_1: 3.0.1 * The latest patch version tag v3_1_4: 3.0.1 * Ruby 3.0: * The branch ruby_3_0: 2.2.2 * The latest patch version tag v3_0_6: 2.2.2 The possible ways to backport are * Include ruby/openssl 3.2.0 in Ruby 3.2, 3.1 and 3.0. I think this is the easiest option. * Backport the 5 commits above in ruby/ruby. * If ruby/ruby only wants to use ruby/openssl only increasing patch version, we may need some work in ruby/openssl side to reelase the ruby/openssl gem 3.1.z, 3.0.z and 2.2.z. I think this is the hardest option. The reason why the oldest Ruby version to be backported is 3.0 is because the oldest Ruby version using OpenSSL 3 is 3.0 in CentOS 9 stream and RHEL 9 main branch, in our maintaining Linux distributions: Fedora, CentOS Stream and RHEL. ## References of the patch files As a reference, I would share our managing patches created from the 5 commits above in Fedora and CentOS 9 Stream. * Fedora rawhide (f40): Based Ruby: 3.2.2: https://gitlab.com/redhat/centos-stream/rpms/ruby/-/blob/c9s/ruby.spec?ref_… Patch12, 13, 14, and 15 are the patch files. * CentOS 9 Stream, Ruby 3.1 stream: Based Ruby: 3.1.2 https://gitlab.com/redhat/centos-stream/rpms/ruby/-/blob/stream-ruby-3.1-rh… Patch30, 31, 32, and 33 are the patch files. * CentOS 9 Stream, default: Based Ruby: 3.04. https://gitlab.com/redhat/centos-stream/rpms/ruby/-/blob/c9s/ruby.spec?ref_… Patch 65, 66, 67, 68 are the patch files. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113107] [Ruby master Bug#19576] Backport request: Gemfile.lock resolving is broken with bundler shipped with Ruby 3.1.4
by jprokop (Jarek Prokop) 10 Nov '23

10 Nov '23

Issue #19576 has been reported by jprokop (Jarek Prokop). ---------------------------------------- Bug #19576: Backport request: Gemfile.lock resolving is broken with bundler shipped with Ruby 3.1.4 https://bugs.ruby-lang.org/issues/19576 * Author: jprokop (Jarek Prokop) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This is a backport request for bundler, that regressed in 2.3.36 in a specific situation. Newer and older bundler versions that ship with Ruby are not problematic, only the version that ships with Ruby version >= 3.1.3. A few weeks ago, we discovered a bug in resolving in bundler shipped with Ruby 3.1.3 and 3.1.4: Bundler version: ``` $ bundler --version Bundler version 2.3.26 ``` Affected rubies `ruby -v`: First: ``` $ ruby -v ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux] ``` Second: ``` $ruby -v ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] ``` Initial bug report with reproducer and more in-depth description can be found here: https://github.com/sclorg/s2i-ruby-container/issues/469 Using the following Gemfile for a rails app: https://github.com/sclorg/rails-ex/blob/67b7a61eae9efa1088ff3f634ae316e1022… bundler locks up in trying to resolve Nokogiri for Ruby 3.1, but keeps failing because it keeps using incompatible built binary gem instead of falling back to installing and building the binary extension of Nokogiri locally. We craft this Gemfile to be usable from Ruby 2.5 up to Ruby 3.1, as the app is used mainly for testing. I have created a patch to fix the situation, see the attached files. There are 2 of them, one contains the fix and the other one contains the test from the rubygems repo PR#6225. full commit available here: https://src.fedoraproject.org/fork/jackorp/rpms/ruby/c/5ef600a8f40b76de5636… The patches are created from the following upstream changes in bundler: https://github.com/rubygems/rubygems/pull/6225 and adapted: https://github.com/rubygems/rubygems/commit/7b64c64262a7a980c0eb23b96ea56cf… for the PR#6225. With the fix applied I no longer have issues doing `bundle install` with our Gemfile.lock. ---Files-------------------------------- rubygem-bundler-2.3.26-Tests-from-bundler-PR-6225.patch (1.82 KB) rubygem-bundler-2.3.26-Provide-fix-for-bundler-Gemfile-resolving-regression.patch (5.21 KB) -- https://bugs.ruby-lang.org/

5 7

[ruby-core:115023] [Ruby master Bug#19924] Character literal escaped \xFF stops parsing
by kddnewton (Kevin Newton) 10 Nov '23

10 Nov '23

Issue #19924 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #19924: Character literal escaped \xFF stops parsing https://bugs.ruby-lang.org/issues/19924 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When you have a character literal, you can have an escaped character. For things that don't actually need escaping, this works out to just returning the character itself. For example: ``` ruby ?\d # => "d" ``` This works for every codepoint in ASCII-8bit _except_ 0xFF. When that byte is hit, the entire parser will stop parsing and just return `nil`. ``` ruby eval(<<~RUBY) # encoding: ascii-8bit p ?\\\xFF puts "this will never be parsed" RUBY ``` If you try to parse it with `RubyVM::AbstractSyntaxTree`, it's clear the parser just stops: ``` ruby RubyVM::AbstractSyntaxTree.parse(<<~RUBY) # encoding: ascii-8bit p ?\\\xFF puts "this will never be parsed" RUBY # => (SCOPE@1:0-2:1 tbl: [] args: nil body: (VCALL@2:0-2:1 :p)) ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114422] [Ruby master Feature#19842] Intorduce M:N threads
by ko1 (Koichi Sasada) 10 Nov '23

10 Nov '23

Issue #19842 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #19842: Intorduce M:N threads https://bugs.ruby-lang.org/issues/19842 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) ---------------------------------------- This ticket proposes to introduce M:N threads to improve Threads/Ractors performance. ## Background Ruby threads (RT in short) are implemented from old Ruby versions and they have the following features: * Can be created with simple notation `Thread.new{}` * Can be switched to another ready Ruby thread by: * Time-slice. * I/O blocking. * Synchronization such as Mutex features. * And other blocking reasons. * Can be interruptible by: * OS-deliver signals (only for the main thread). * `Thread#kill`. * Can be terminated by: * the end of each Ruby thread. * the end of the main thread (and other Ruby threads are killed). Ruby 1.8 and erlier versions uses M:1 threads (green threads, user level threads, .... the word 1:N threads is more popular but to make this explanation consistent I use "M:1" term here) which manages multiple Ruby threads on 1 native thread. (Native threads are provided by C interfaces such as Pthreads. In many cases, native threads are OS threads, but there are also user-level implementations, such as user-level pthread libraries in theory. Therefore, they are referred to as native threads in this article and NT in short) If a Ruby thread T1 blocked because of a I/O operation, Ruby interpreter switches to the next ready Ruby thread T2. The I/O operation will be monitors by a `select()` (or similar) functionality and if the I/O is ready, T1 is marked as a ready thread and T1 will be resumed soon. However, when a Ruby thread issues some other blocking operations such as `gethostbyname()`, Ruby interpreter can not swtich to any other Ruby thread while `gethostbyname()` is not finished. We named two types blocking operations: * Managed blocking operations * I/O (most of read/write) * manage by I/O multiplexing API (select, poll, epoll, kqueue, IOCP, io_uring, ...) * Sleeping * Synchronization (Mutex, Queue, ...) * Unmanaged operations * All other blocking operations not listed above, written in C * Huge number calculation like `Bignum#*` * DNS lookup * I/O (can not detect block-able or not by multiplexing API) * open on FIFO, close on NFS, ... * flock and other locking mechanism * library call which uses blocking operations * `libfoo` has `foo_func()` and `foo_func()` waits DNS lookup. A Ruby extension `foo-ruby` can call `foo_func()`. With these terms we can say that M:1 threads can suport managed blocking operations but can not support unmanaged operations (can not make progress other Ruby threads) without further tricks. Note that if the `select()`-like system calls say a `fd` is ready, but the I/O opeartion for `fd` can be blocked because of some contention (read by another thread or process, for example). M:1 threads has another disadvantage that it can not run in parallel because only a native thread is used. From Ruby 1.9 we had implemented 1:1 thread which means a Ruby thread has a corresponding native thread. To make implementation easy we also introduced a GVL. Only a Ruby thread acquires GVL can run. With 1:1 model, we can support managed blocking oprations and unmanaged blocking operations by releasing GVL. When a Ruby thread want to issue a blocking operation, the Ruby thread releases GVL and another ready Ruby threads continue to run. We don't care the blocking operation is managed or unmanaged. (We can not make some of unmanaged blocking operations interruptible (stop by Ctrl-C for example)). Advantages of 1:1 threads to the M:1 threads is: * Easy to handle blocking operations by releasing GVL. * We can utilize parallelism with multiple native threads by releasing GVL. Disadvantages of 1:1 threads to the M:1 threads is: * Overhead to make many native threads for many Ruby threads * We can not make huge number of Ruby threads and Ractors on 1:1 threads. * Thread switching overhead by GVL because inter-core communication is needed. From Ruby 3.0 we introduced fiber scheduler mechanism to maintain multiple fibers Differences between Ruby 1.8 M:1 threads are: * No timeslice (only switch fibers by managed blocking operations) * Ruby users can make own schedulers for apps with favorite underlying mechanism Disadvantages are similar to M:1 threads. Another disadvantages is we need to consider about Fiber's behavior. From Ruby 3.0 we also introduced Ractors. Ractors can run in parallel because of separating most of objects. 1 Ractor creates 1 Ruby thread, so Ractors has same disadvantages of 1:1 threads. For example, we can not make huge number of Ractors. ## Goal Our goal is making lightweight Ractors on lightweight Ruby threads. To enable this goal we propose to implement M:N threads on MRI. M:N threads manages M Ruby threads on N native threads, with limited N (~= CPU core numbers for example). Advantages of M:N threads are: 1. We can run N ractors on N native threads simultaneously if the machine has N cores. 2. We can make huge number of Ruby threads or Ractors because we don't need huge number of native threads 3. We can support unmanaged blocking operations by locking a native thread to a Ruby thread which issues an unmanaged blocking operation. 4. We can make our own Ruby threads or Ractors scheduler instead of the native thread (OS) scheduler. Disadvantages of M:N threads are: 1. It is complex implmentation and it can be hard. 2. It can introduce incompatibility especaially on TLS (Thread local storage). 3. We need to maitain our own scheduler. Without using multiple Ractors, it is similar to Ruby 1.8 M:1 threads. The difference with M:1 threads are locking NT mechanism to support unmanaged blocking operations. Another advantage is that it is easy to fallback to 1:1 threads by locking all of corresponding native threads to Ruby threads. ## Proposed design ### User facing changes If a program only has a main Ractor (i.e., most Ruby programs), the user will not face any changes by default. On main Ractor, all threads are 1:1 threads by default and there is no compatibility issue. `RUBY_MN_THREADS=1` envrionment variable is given, main Ractor enables M:N threads. Note that the main thread locks NT by default because the initial NT is special in some case. I'm not sure we can relax this limitation. On the multiple Ractors, N (+ alpha) native threads run M ractors. Now there is no way to disable M:N threads on multiple Ractors because there are only a few multi-Ractor programs and no compatibility issues. Maximum number of N can be specified by `RUBY_MAX_PROC=N`. 8 by default but this value should be specified with the number of CPU processors (cores). ### TLS issue On M:N threads a Ruby thread (RT1) migrates from a native thread (NT1) to NT2, ... so that TLS on native code can be a problem. For example, RT1 calls a library function `foo()` and it set TLS1 on NT1. After migrating RT1 to NT2, RT1 calls `foo()` again but there is no TLS1 record because TLS1 is recorded only on NT1. On this case, RT1 should be run on NT1 while using native library foo. To avoid such prbolem, we need the following features: * 1:1 threads on main Ractor by default * functionality to lock the NT for RT, maybe `Thread#lock_native_thread` and `Thread#unlock_native_thread` API is needed. For example, Go language has `runtime.LockOSThread()` and `runtime.UnlockOSThread()` for this purpose. * Or C-API only for this purpose? (not fixed yet) Thankfully, the same problem can occur with Fiber scheduler (and of course Ruby 1.8 M:1 threads), but I have not heard of it being much of a problem, so I expect that TLS will not be much of an issue. ### Unmanaged blocking operations From Ruby 1.9 (1:1 threads), the `nogvl(func)` API is used for most blocking operations to keep the threading system healthy. In other words, `nogvl(func)` represents that the given function is blocking operation. To support unmanaged blocking operations, we lock a native thread for the Ruby thread which issues blocking operation. If the blocking operations doesn't finish soon, other Ruby threads can not run because a RT locks NT. In this case, another system monitoring thread named "Timer thread" (historical name and TT in short) creates another NT to run ready other Ruby threads. This TT's behavior is the same as the behavior of "sysmon" in the Go language. We named locked NT as dedicated native threads (DNT) and other NT as shared native threads (SNT). The upper bound by `RUBY_MAX_PROC` affects the number of SNT. In other words, the number of DNT is not limited (it is same that the number of NT on 1:1 threads are not limited). ### Managed blocking operations Managed blocking operations are multiplexing by `select()`-like functions on the Timer thread.. Now only `epoll()` is supported. I/O operation flow (read on fd1) on Ruby thread RT1: 1. check the ready-ness of fd1 by `poll(timeout = 0)`, goto step 4. 2. register fd1 to Timer thread (TT) epoll and resume another ready Ruby thread. 3. If TT detects that the fd1 is ready, make RT1 as ready thread. 4. When RT1 is resumed, then do `read()` by locking corresponding NT1. `sleep(n)` operation flow on Ruby thread RT1: 1. register timeout of RT1 to TT epoll. 2. If TT detects the timeout of RT1 (n seconds), TT makes RT1 as a ready Ruby thread. ### Internal design * 2 level scheduling * Ruby threads of a Ractor is managed by 1:N threads * Ruby threads of different Ractors are managed by M:N threads * Timer thread has several duties 1. Monitoring I/O (or other event) ready-ness 2. Monitoring timeout 3. Produce timeslice signals 4. Help OS signal delivering (On pthread environment) recent Ruby doesn't make timer thread but MaNy implementation makes TT anytime. it can be improved. ## Implementation The code name is MaNy project, it is from MN threads. https://github.com/ko1/ruby/tree/many2 The implementation is not matured (debugging now). ## Measurements See RubyKaigi 2023 slides: https://atdot.net/~ko1/activities/2023_rubykaigi2023.pdf ## Discussion * Enable/disable * default behavior * how to switch the behavior * Should we lock the NT for main thread anytime? * Ruby/C API to lock the native threads ## Misc This description will be improved more later. -- https://bugs.ruby-lang.org/

7 29

[ruby-core:115325] [Ruby master Feature#19995] Proposal: Signal._trap as analogue to Process._fork
by kjtsanaktsidis (KJ Tsanaktsidis) 09 Nov '23

09 Nov '23

Issue #19995 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Feature #19995: Proposal: Signal._trap as analogue to Process._fork https://bugs.ruby-lang.org/issues/19995 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal ---------------------------------------- This is a proposal to define a method `Signal#_trap`, which would function as a single place to be wrapped by monitoring libraries for detecting when signal handlers are entered and exited. ## Motivation #1 - stack-based context leaks into signal handlers When writing monitoring & instrumentation libraries, we often want to add context to emitted logs/metrics/etc based on the current call stack. For example, `ActiveSupport::TaggedLogging` allows you to write code like this: ``` $logger = ActiveSupport::TaggedLogging.new(Logger.new($stderr)) def method_one $logger.tagged("method_one") do method_two end end def method_two $logger.tagged("method_two") do $logger.info "hello there" end end method_one # logs "[method_one] [method_two] hello there" ``` Signal handlers can complicate this picture though because they can run at arbitrary points in the program. Using a threadlocal variable to store context, as is commonly done, means that the invoked signal handler shares the context of whatever thread it interrupted. For example, this program will log the "method_one" tag from inside the signal handler: ``` # n.b. - need to use the mono_logger gem instead of stdlib ::Logger because ::Logger contains a mutex, # which cannot be used from a trap handler. $logger = ActiveSupport::TaggedLogging.new(MonoLogger.new($stderr)) Signal.trap(:TERM) do $logger.tagged("term_handler") do $logger.info "goodbye there" exit end end def method_one $logger.tagged("method_one") do Process.kill :TERM, Process.pid sleep end end method_one # logs "[method_one] [term_handler] goodbye there" ``` This is, in my opinion, undesirable. The fact that the signal handler happened to interrupt at this point and not some other point does not mean that the handler has anything at all to do with `method_one`. I would like a way to, in the logger implementation, detect that we have entered a signal handler and switch to a different context stack, so that the above example printed "[term_handler] goodbye there" instead. ## Motivation #2 - avoiding mutex use in signal handlers This motivation is hinted at by the use of the `mono_logger` gem in the preceding example. It’s not legal to use a mutex in a trap handler, which means `::Logger` can’t be used. However, the mutex is there for a good reason - it synchronises writes to the log device amongst multiple threads so that messages are written atomically, even if they’re longer than `PIPE_BUF` (the max size write the OS guarantees is atomic in a single write syscall). Ideally, we would like to use the mutex-based implementation in normal code, but avoid the mutex in trap handlers (possibly writing to a different stream e.g. `$stderr` instead of `$stdout`). For an instrumentation implementation to do this, it needs a way to detect if it’s currently running inside a trap handler. ## Proposed solution I would like to add a "hook" method `Signal._trap`, which works in a similar way to `Process._fork`. Essentially, when Ruby invokes a signal handler, instead of directly invoking the registered proc, it would invoke `Signal._trap`, and _that_ would invoke the proc. This means that by prepending to `Signal.singleton_class`, you could write code which wraps around all signal handlers. My first example could then be re-written like so: ``` $logger = ActiveSupport::TaggedLogging.new(MonoLogger.new($stderr)) Signal.singleton_class.prepend(Module.new do def _trap(signo) old_tags = $logger.pop_tags 1_000_000_000 # ActiveSupport hack; there's no "pop_all" method. super $logger.push_tags old_tags end end) Signal.trap(:TERM) do $logger.tagged("term_handler") do $logger.info "goodbye there" exit end end def method_one $logger.tagged("method_one") do Process.kill :TERM, Process.pid sleep end end method_one # logs "[term_handler] goodbye there" ``` ## Polyfill for older Rubies I believe `Signal._trap` can be fully implemented in a gem for older versions of Ruby; something like the following will work: https://gist.github.com/KJTsanaktsidis/0b263c76523a16a049fa5a035e868a68. This is, again, analogous to how monitoring libraries would handle fork hooking before `_fork` was added - possible, but very tricky and ugly. ## Alternative solution Another option which might help solve the two problems I outlined is to expose the `ec->interrupt_mask |= TRAP_INTERRUPT_MASK flag` as a method on `Signal` or a special `$MAGIC_GLOBAL` variable. This would essentially let code detect whether or not it’s in a trap handler, but not execute specific code around trap handlers. ---- Thanks for your time friends, looking forward to any feedback or discussion! -- https://bugs.ruby-lang.org/

1 0