May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118110] [Ruby master Bug#20515] --with-gmp is not working - GMP support won't be built
by sorah (Sorah Fukumori) 15 Jun '24

15 Jun '24

Issue #20515 has been reported by sorah (Sorah Fukumori). ---------------------------------------- Bug #20515: --with-gmp is not working - GMP support won't be built https://bugs.ruby-lang.org/issues/20515 * Author: sorah (Sorah Fukumori) * Status: Open * Assignee: sorah (Sorah Fukumori) * Target version: 3.3 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- gmp integration has been broken since https://bugs.ruby-lang.org/issues/20494 18eaf0be905e3e251423b42d6f4e56b7cae1bc3b including just released 3.3.2 due to missing HAVE_GMP_H. -- https://bugs.ruby-lang.org/

3 7

[ruby-core:117905] [Ruby master Bug#20493] Segfault on rb_io_getline_fast
by josegomezr (Jose Gomez) 15 Jun '24

15 Jun '24

Issue #20493 has been reported by josegomezr (Jose Gomez). ---------------------------------------- Bug #20493: Segfault on rb_io_getline_fast https://bugs.ruby-lang.org/issues/20493 * Author: josegomezr (Jose Gomez) * Status: Open * ruby -v: 3.3.1 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- We've spotted a consistent segfault when running bundle install with `--jobs 4` When running: `bundle install -j 4` we'd get a Segfault at: ``` /usr/lib64/ruby/3.3.0/rubygems/ext/builder.rb:93: [BUG] Segmentation fault at 0x0000000000000014 ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux-gnu] ``` Full [log is available here][0]. I could not find a shorter reproducer besides using bundler with `--jobs 4` or `--jobs 8`. Here's a sample command to trigger the behavior (it creates the Gemfile and calls bundler) [1]. We installed all debug symbols and narrowed down the location of the segfault to `rb_io_getline_fast` in io.c At [line 4001][3] `str` is `T_NONE`, which makes further usage down the line in [`io_enc_str`][4] raise a null pointer dereference. With the notes from [extension.rdoc - Appendix E. RB_GC_GUARD to protect from premature GC][8] I've prepared a patched ruby 3.3.1 package that does not segfault. It's on [OBS Project home:josegomezr:branches:ruby/ruby3.3][6]. Adding a `RB_GC_GUARD` on `rb_io_getline_fast` @ `io.c:4004` just before the return ```diff --- ruby3.3.orig/ruby-3.3.1/io.c +++ ruby3.3/ruby-3.3.1/io.c @@ -4004,6 +4004,7 @@ rb_io_getline_fast(rb_io_t *fptr, rb_enc ENC_CODERANGE_SET(str, cr); fptr->lineno++; + RB_GC_GUARD(str); return str; } ``` Fixes the segfault in our tests. `bundle` finish the installation and the image is built. I've set up a project in OBS to provide reproduceables. - [ruby3.3.1 package][5]. - [ruby3.3.1 base image with enough dependencies to reproduce][7] with [the reproducer script][1]. And the corresponding container is exported in the `containers-patched` repository. Here I leave the docker images generated by OBS: - 3.3.1 [without patches, segfaults.] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers/b… ``` - 3.3.1 [with patch, does not fail] ``` registry.opensuse.org/home/josegomezr/branches/ruby/containers/containers-p… ``` [0]: https://gist.github.com/josegomezr/441c271cc731b0ec57213cb98743a699 [1]: https://gist.github.com/josegomezr/e17129bf2df33f3bea60e84a616a8322 [2]: https://gist.github.com/josegomezr/6f81878c979af334efee59b8f2225e58 [3]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4001 [4]: https://github.com/ruby/ruby/blob/v3_3_1/io.c#L4003 [5]: https://build.opensuse.org/package/show/devel:languages:ruby/ruby3.3 [6]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby/ruby3… [7]: https://build.opensuse.org/package/show/home:josegomezr:branches:ruby:conta… [8]: https://github.com/ruby/ruby/blob/master/doc/extension.rdoc#label-Appendix+… -- https://bugs.ruby-lang.org/

4 13

[ruby-core:116182] [Ruby master Bug#20180] Inconsistent evaluation of `**{}` depending on position in array
by ozydingo (Andrew Schwartz) 15 Jun '24

15 Jun '24

Issue #20180 has been reported by ozydingo (Andrew Schwartz). ---------------------------------------- Bug #20180: Inconsistent evaluation of `**{}` depending on position in array https://bugs.ruby-lang.org/issues/20180 * Author: ozydingo (Andrew Schwartz) * Status: Open * Priority: Normal * ruby -v: 3.3 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduced on ruby:3.3 docker container The evaluation of `**{}` differs if it appears alone (evaluates as empty / no content) in an array vs after another element (evaluates as an empty Hash). ```rb args = []; kwargs = {} [*args] # => [] [**kwargs] # => [] [*args, **kwargs] # => [{}] [*args] + [**kwargs] == [*args, **kwargs] => false ``` I claim this violates the Principle of Least Surprise. I will admit that beyond a thin example I will give below, I am struggling to come up with a more convincing pragmatic reason that this should be addressed, but due to how surprising it is and the bugs it cause our team I wanted to submit it for tracking. This may be related to https://bugs.ruby-lang.org/issues/20064?tab=notes though the issues are distinct. Specifically, in my use case, I am writing a class responsible for adapting arguments in one form to another (json object to method args and vice versa). My tests broken when adding support for keyword args due to expectations of no args: ``` # RSpec expect(foo).to have_received(:bar).with([]) # This now need `with([], {})` ``` Again, this is a bit thin as by itself this isn't problematic as I can add the `{}`. However, this does require that my test knows more about the implementation that it should. That is, my implementation might be ``` if kwargs.present? call(*args, **kwargs) else call(*args) end ``` This change does not change the behavior of the class, but will break the test. I therefore think this behavior of `**kwargs` is problematic. -- https://bugs.ruby-lang.org/

6 7

[ruby-core:116838] [Ruby master Bug#20280] Error at invalid encoding symbols
by nobu (Nobuyoshi Nakada) 15 Jun '24

15 Jun '24

Issue #20280 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20280: Error at invalid encoding symbols https://bugs.ruby-lang.org/issues/20280 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Parsing an invalid encoding symbol raises an `EncodingError`. It must be a `SyntaxError`. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116790] [Ruby master Bug#20270] Options with `--parser=prism`
by nobu (Nobuyoshi Nakada) 12 Jun '24

12 Jun '24

Issue #20270 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20270: Options with `--parser=prism` https://bugs.ruby-lang.org/issues/20270 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ### `--dump` option Currently `parsetree` and `prism_parsetree` bits are separated, but it seems meaningless as far as `--parser` option selects only one parser. Why doesn't simply `--dump=parsetree` dump AST by parse.y, or PRISM AST if `--parser=prism` is given? Actually, `ruby --dump=parsetree --parser=prism` just segfaults, because these bits and `--parser` option are separately handled. ### streaming code from stdin ```sh-session $ echo end | ruby --parser=prism - ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker. -: warning: Prism support for streaming code from stdin is not currently supported ``` Nothing reported is fine, since it is not currently supported. I was a bit curious that it looks like trying something after the warning. ```C if (strcmp(opt->script, "-") == 0) { int xflag = opt->xflag; VALUE rb_source = open_load_file(opt->script_name, &xflag); opt->xflag = xflag != 0; rb_warn("Prism support for streaming code from stdin is not currently supported"); error = pm_parse_string(&result, rb_source, opt->script_name); } ``` Note that `open_load_file` returns `rb_stdin` when `-` is given. This object is a `RFile`, not a `RString`, and `RSTRING_PTR(rb_source)` accesses after the object boundary. It should just bail out, not try obviously wrong thing. ### Fix https://github.com/nobu/ruby/tree/options-refactor -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116039] [Ruby master Bug#20154] aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment
by jprokop (Jarek Prokop) 10 Jun '24

10 Jun '24

Issue #20154 has been reported by jprokop (Jarek Prokop). ---------------------------------------- Bug #20154: aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment https://bugs.ruby-lang.org/issues/20154 * Author: jprokop (Jarek Prokop) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Recently a GH PR was merged <https://github.com/ruby/ruby/pull/9306> For PAC/BTI support on ARM CPUs for Coroutine.S. Without proper compilation support in configure.ac it segfaults Ruby with fibers on CPUs where PAC is supported: https://bugs.ruby-lang.org/issues/20085 At the time of writing, configure.ac appends the first option from a list for flag `-mbranch-protection` that successfully compiles a program <https://github.com/ruby/ruby/blob/master/configure.ac#L829>, to XCFLAGS and now also ASFLAGS to fix issue 20085 for Ruby master. This is suboptimal for Fedora as we set -mbranch-protection=standard by default in C{,XX}FLAGS: ``` CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ' export CFLAGS CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' export CXXFLAGS ``` And the appended flag overrides distribution's compilation configuration, which in this case ends up omitting BTI instructions and only using PAC. Would it make sense to check if such flags exist and not overwrite them if they do? Serious proposals: 1. Simplest fix that does not overwrite what is set in the distribution and results in higher security is simply prepending the list of options with `-mbranch-protection=standard`, it should cause no problems on ARMv8 CPUs and forward, BTI similarly to PAC instructions result into NOP, it is only extending the capability. See attached 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch 2. Other fix that sounds more sane IMO and dodges this kind of guessing where are all the correct places for the flag is what another Fedora contributor Florian Weimer suggested: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… "The reliable way to do this would be to compile a C file and check whether that enables __ARM_FEATURE_PAC_DEFAULT, and if that's the case, define a *different* macro for use in the assembler implementation. This way, you don't need to care about the exact name of the option." IOW instead of using __ARM_FEATURE_* directly in that code, define a macro in the style of "USE_PAC" with value of the feature if it is defined, I think that way we shouldn't need to append ASFLAGS anymore. However it's also important to catch the value of those macros as their values have meaning, I have an idea how to do that but I'd get on that monday earliest. ---Files-------------------------------- 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch (1004 Bytes) -- https://bugs.ruby-lang.org/

4 10

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:116117] [Ruby master Misc#20170] Drop support for GCC < 11
by kddnewton (Kevin Newton) 07 Jun '24

07 Jun '24

Issue #20170 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20170: Drop support for GCC < 11 https://bugs.ruby-lang.org/issues/20170 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:115595] [Ruby master Bug#20043] `defined?` checks for method existence but only sometimes
by tenderlovemaking (Aaron Patterson) 07 Jun '24

07 Jun '24

Issue #20043 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20043: `defined?` checks for method existence but only sometimes https://bugs.ruby-lang.org/issues/20043 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-05T21:25:34Z master 56eccb350b) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When an expression is passed to `defined?`, it will _sometimes_ check if a method in a sub-expression is defined and sometimes it won't. For example: ``` $ ./miniruby -e'p defined?(a)' nil $ ./miniruby -e'p defined?([a])' nil ``` In the above case, Ruby will check whether or not the method `a` is defined, and it returns `nil`. However, if you use a splat, it will not check: ``` $ ./miniruby -e'p defined?([*a])' "expression" ``` The same thing seems to happen with method parameters: ``` $ ./miniruby -e'p defined?(itself)' "method" $ ./miniruby -e'p defined?(itself(a))' nil $ ./miniruby -e'p defined?(itself(*a))' "method" ``` Oddly, `defined?` will check contents of arrays, but _won't_ check contents of hashes: ``` $ ./miniruby -e'p defined?([[[[a]]]])' nil $ ./miniruby -e'p defined?({ a => a })' "expression" ``` I think all of the cases that refer to `a` should check whether or not `a` is defined regardless of splats or hashes. -- https://bugs.ruby-lang.org/

7 9

[ruby-core:117866] [Ruby master Misc#20488] Document source file size restrictions
by kddnewton (Kevin Newton) 06 Jun '24

06 Jun '24

Issue #20488 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20488: Document source file size restrictions https://bugs.ruby-lang.org/issues/20488 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I was hoping we might be able to decide on an official maximum size for a source file in terms of bytes/lines/columns/etc. Ruby uses fixed integer sizes to represent line numbers, column numbers, offsets, etc. These can overflow with files that are too big. Sometimes they will fail with cryptic messages like: ``` ruby: negative string size (or size too big) (ArgumentError) ``` (That's for 2^31 "a"s in a row.) Note that for python for the same file you will get: ``` OverflowError: Parser column offset overflow - source line is too big ``` and for perl you will get: ``` Identifier too long at test.pl line 1. ``` For files with 2^32 newlines, Ruby just crashes on my machine with the current parser (prism finishes, but its newline counter overflows so it gets all the offsets wrong). Would it be okay to say: * maximum line: 31 bits * maximum column: 32 bits * maximum file byte size: 32 bits This would also help with memory savings — I would like to only use 32 bits for offsets in the file, as opposed to the current 64 bits I'm using (which seems unnecessarily large). -- https://bugs.ruby-lang.org/

1 1