May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116182] [Ruby master Bug#20180] Inconsistent evaluation of `**{}` depending on position in array
by ozydingo (Andrew Schwartz) 15 Jun '24

15 Jun '24

Issue #20180 has been reported by ozydingo (Andrew Schwartz). ---------------------------------------- Bug #20180: Inconsistent evaluation of `**{}` depending on position in array https://bugs.ruby-lang.org/issues/20180 * Author: ozydingo (Andrew Schwartz) * Status: Open * Priority: Normal * ruby -v: 3.3 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduced on ruby:3.3 docker container The evaluation of `**{}` differs if it appears alone (evaluates as empty / no content) in an array vs after another element (evaluates as an empty Hash). ```rb args = []; kwargs = {} [*args] # => [] [**kwargs] # => [] [*args, **kwargs] # => [{}] [*args] + [**kwargs] == [*args, **kwargs] => false ``` I claim this violates the Principle of Least Surprise. I will admit that beyond a thin example I will give below, I am struggling to come up with a more convincing pragmatic reason that this should be addressed, but due to how surprising it is and the bugs it cause our team I wanted to submit it for tracking. This may be related to https://bugs.ruby-lang.org/issues/20064?tab=notes though the issues are distinct. Specifically, in my use case, I am writing a class responsible for adapting arguments in one form to another (json object to method args and vice versa). My tests broken when adding support for keyword args due to expectations of no args: ``` # RSpec expect(foo).to have_received(:bar).with([]) # This now need `with([], {})` ``` Again, this is a bit thin as by itself this isn't problematic as I can add the `{}`. However, this does require that my test knows more about the implementation that it should. That is, my implementation might be ``` if kwargs.present? call(*args, **kwargs) else call(*args) end ``` This change does not change the behavior of the class, but will break the test. I therefore think this behavior of `**kwargs` is problematic. -- https://bugs.ruby-lang.org/

6 7

[ruby-core:116838] [Ruby master Bug#20280] Error at invalid encoding symbols
by nobu (Nobuyoshi Nakada) 15 Jun '24

15 Jun '24

Issue #20280 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20280: Error at invalid encoding symbols https://bugs.ruby-lang.org/issues/20280 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Parsing an invalid encoding symbol raises an `EncodingError`. It must be a `SyntaxError`. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116790] [Ruby master Bug#20270] Options with `--parser=prism`
by nobu (Nobuyoshi Nakada) 11 Jun '24

11 Jun '24

Issue #20270 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20270: Options with `--parser=prism` https://bugs.ruby-lang.org/issues/20270 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ### `--dump` option Currently `parsetree` and `prism_parsetree` bits are separated, but it seems meaningless as far as `--parser` option selects only one parser. Why doesn't simply `--dump=parsetree` dump AST by parse.y, or PRISM AST if `--parser=prism` is given? Actually, `ruby --dump=parsetree --parser=prism` just segfaults, because these bits and `--parser` option are separately handled. ### streaming code from stdin ```sh-session $ echo end | ruby --parser=prism - ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker. -: warning: Prism support for streaming code from stdin is not currently supported ``` Nothing reported is fine, since it is not currently supported. I was a bit curious that it looks like trying something after the warning. ```C if (strcmp(opt->script, "-") == 0) { int xflag = opt->xflag; VALUE rb_source = open_load_file(opt->script_name, &xflag); opt->xflag = xflag != 0; rb_warn("Prism support for streaming code from stdin is not currently supported"); error = pm_parse_string(&result, rb_source, opt->script_name); } ``` Note that `open_load_file` returns `rb_stdin` when `-` is given. This object is a `RFile`, not a `RString`, and `RSTRING_PTR(rb_source)` accesses after the object boundary. It should just bail out, not try obviously wrong thing. ### Fix https://github.com/nobu/ruby/tree/options-refactor -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116039] [Ruby master Bug#20154] aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment
by jprokop (Jarek Prokop) 09 Jun '24

09 Jun '24

Issue #20154 has been reported by jprokop (Jarek Prokop). ---------------------------------------- Bug #20154: aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment https://bugs.ruby-lang.org/issues/20154 * Author: jprokop (Jarek Prokop) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Recently a GH PR was merged <https://github.com/ruby/ruby/pull/9306> For PAC/BTI support on ARM CPUs for Coroutine.S. Without proper compilation support in configure.ac it segfaults Ruby with fibers on CPUs where PAC is supported: https://bugs.ruby-lang.org/issues/20085 At the time of writing, configure.ac appends the first option from a list for flag `-mbranch-protection` that successfully compiles a program <https://github.com/ruby/ruby/blob/master/configure.ac#L829>, to XCFLAGS and now also ASFLAGS to fix issue 20085 for Ruby master. This is suboptimal for Fedora as we set -mbranch-protection=standard by default in C{,XX}FLAGS: ``` CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ' export CFLAGS CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' export CXXFLAGS ``` And the appended flag overrides distribution's compilation configuration, which in this case ends up omitting BTI instructions and only using PAC. Would it make sense to check if such flags exist and not overwrite them if they do? Serious proposals: 1. Simplest fix that does not overwrite what is set in the distribution and results in higher security is simply prepending the list of options with `-mbranch-protection=standard`, it should cause no problems on ARMv8 CPUs and forward, BTI similarly to PAC instructions result into NOP, it is only extending the capability. See attached 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch 2. Other fix that sounds more sane IMO and dodges this kind of guessing where are all the correct places for the flag is what another Fedora contributor Florian Weimer suggested: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… "The reliable way to do this would be to compile a C file and check whether that enables __ARM_FEATURE_PAC_DEFAULT, and if that's the case, define a *different* macro for use in the assembler implementation. This way, you don't need to care about the exact name of the option." IOW instead of using __ARM_FEATURE_* directly in that code, define a macro in the style of "USE_PAC" with value of the feature if it is defined, I think that way we shouldn't need to append ASFLAGS anymore. However it's also important to catch the value of those macros as their values have meaning, I have an idea how to do that but I'd get on that monday earliest. ---Files-------------------------------- 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch (1004 Bytes) -- https://bugs.ruby-lang.org/

4 10

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:116117] [Ruby master Misc#20170] Drop support for GCC < 11
by kddnewton (Kevin Newton) 07 Jun '24

07 Jun '24

Issue #20170 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20170: Drop support for GCC < 11 https://bugs.ruby-lang.org/issues/20170 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:115595] [Ruby master Bug#20043] `defined?` checks for method existence but only sometimes
by tenderlovemaking (Aaron Patterson) 07 Jun '24

07 Jun '24

Issue #20043 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20043: `defined?` checks for method existence but only sometimes https://bugs.ruby-lang.org/issues/20043 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-05T21:25:34Z master 56eccb350b) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When an expression is passed to `defined?`, it will _sometimes_ check if a method in a sub-expression is defined and sometimes it won't. For example: ``` $ ./miniruby -e'p defined?(a)' nil $ ./miniruby -e'p defined?([a])' nil ``` In the above case, Ruby will check whether or not the method `a` is defined, and it returns `nil`. However, if you use a splat, it will not check: ``` $ ./miniruby -e'p defined?([*a])' "expression" ``` The same thing seems to happen with method parameters: ``` $ ./miniruby -e'p defined?(itself)' "method" $ ./miniruby -e'p defined?(itself(a))' nil $ ./miniruby -e'p defined?(itself(*a))' "method" ``` Oddly, `defined?` will check contents of arrays, but _won't_ check contents of hashes: ``` $ ./miniruby -e'p defined?([[[[a]]]])' nil $ ./miniruby -e'p defined?({ a => a })' "expression" ``` I think all of the cases that refer to `a` should check whether or not `a` is defined regardless of splats or hashes. -- https://bugs.ruby-lang.org/

7 9

[ruby-core:117866] [Ruby master Misc#20488] Document source file size restrictions
by kddnewton (Kevin Newton) 06 Jun '24

06 Jun '24

Issue #20488 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20488: Document source file size restrictions https://bugs.ruby-lang.org/issues/20488 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I was hoping we might be able to decide on an official maximum size for a source file in terms of bytes/lines/columns/etc. Ruby uses fixed integer sizes to represent line numbers, column numbers, offsets, etc. These can overflow with files that are too big. Sometimes they will fail with cryptic messages like: ``` ruby: negative string size (or size too big) (ArgumentError) ``` (That's for 2^31 "a"s in a row.) Note that for python for the same file you will get: ``` OverflowError: Parser column offset overflow - source line is too big ``` and for perl you will get: ``` Identifier too long at test.pl line 1. ``` For files with 2^32 newlines, Ruby just crashes on my machine with the current parser (prism finishes, but its newline counter overflows so it gets all the offsets wrong). Would it be okay to say: * maximum line: 31 bits * maximum column: 32 bits * maximum file byte size: 32 bits This would also help with memory savings — I would like to only use 32 bits for offsets in the file, as opposed to the current 64 bits I'm using (which seems unnecessarily large). -- https://bugs.ruby-lang.org/

1 1

[ruby-core:117535] [Ruby master Misc#20432] Proposal for workflow changes related to teeny releases
by ufuk (Ufuk Kayserilioglu) 06 Jun '24

06 Jun '24

Issue #20432 has been reported by ufuk (Ufuk Kayserilioglu). ---------------------------------------- Misc #20432: Proposal for workflow changes related to teeny releases https://bugs.ruby-lang.org/issues/20432 * Author: ufuk (Ufuk Kayserilioglu) * Status: Open ---------------------------------------- ## Aim - The cadence of CRuby stable releases is very important for how the project is perceived. Users of CRuby want to get more frequent releases with bug and security fixes, so that they feel confident that their projects and businesses continue running smoothly and safely. - More frequent releases make the community see that the CRuby project is active and thriving. It also allows people to keep upgrading to the latest versions of Ruby with newer features and better security. - The current cadence of teeny releases are causing concerns in the community, and there have been multiple examples of late releases in [3.0 series](https://www.reddit.com/r/ruby/comments/r14z39/ruby_303_released/), [3.1 and 3.2 series](https://www.reddit.com/r/ruby/comments/18n54es/why_have_they_not_up… and now [3.3 series](https://bugs.ruby-lang.org/issues/20085). This is creating confusion and doubt in our community and making the community to [ask for a change in bug fix release process](https://bugs.ruby-lang.org/issues/20422). - We all agree that release management is difficult and branch maintainers have a hard job. If we can be making their jobs a little bit easier, they will have more time and opportunities to make more frequent releases. - This proposal aims to improve some of the workflows around stable branch release management so that branch maintainers have the opportunity to make more frequent bug fix and security releases, thus, addressing some of the concerns from the community. ## Proposal(s) 1. A lot of the time of a release manager seems to be spent doing backports to the corresponding stable branch. The current workflow asks bug reports to populate the fields for which stable versions need the fix backported to, and release managers are responsible for keeping up with this list and creating backport commits on stable branches. This process is fragile, since the release managers are usually the people with the least amount of context on the particulars of a bug-fix and have to apply the same changes to a different branch and resolve any potential conflicts. This is a task best done by people who already have the most amount of context on the change: the original bugfix author. As a result, my suggestion is to make it mandatory for any change that needs to be backported to have backport PRs opened (or corresponding backport patches attached to the Redmine tickets) by the author for the relevant stable branches. That way the authors of changes will be responsible for making sure that the tests pass and the changes apply cleanly on older branches, allowing branch maintainers to come in and merge the PRs, freeing up their time and focus. Since backport PRs will be opened by authors of changes, but only merged by branch maintainers, this will still allow maintainers to control what goes into stable branches. 2. It also seems like branch maintainers are trying to synchronize releases across different Ruby versions, which is causing delays in getting teeny releases out of the door. In my opinion, each stable branch should be responsible for its own teeny releases and there should be no need to synchronize the cadence between different Ruby versions. This will mean less time waiting for other branch maintainers who might be busy with other life priorities, unblocking branch maintainers to be able to do teeny releases whenever it is convenient. 3. Finally, it is my observation that branch maintainers usually make teeny releases whenever there is a security incident that needs to be addressed. A security fix is most definitely a good reason to make a teeny release, but a teeny release should not wait for a security concern to be addressed. There are many other important bug fixes constantly being backported (or needing backports) such as fixes for crashes, fixes for memory retention/leaks or fixes for other behaviour regressions. It is important for end-users to have those addressed as timely as possible, so I would suggest that branch maintainers aim to make about 6-7 teeny releases every year (some of which will be security releases) in order to deliver value to end-users continuously. ## Conclusions Continuous integration and continuous delivery are one of the greatest changes that have happened in the software engineering sector in the recent decades. They allow us to shorten the time of delivery of new features to end-users and reduce the amount of work-in-progress that is waiting for come into the work, thus reducing waste. By adopting some of the workflow changes that I have suggested above, and maybe other changes that I might have missed, the CRuby core team has an opportunity to make a positive impact on all the users of the Ruby language and to increase the quality perception of the project as a result. For that reason, I hope my proposals are considered and implemented by the Core team. Thanks for your consideration. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117470] [Ruby master Bug#20416] IO#read doesn't preserve buffer encoding if `maxlen = nil`
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20416 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20416: IO#read doesn't preserve buffer encoding if `maxlen = nil` https://bugs.ruby-lang.org/issues/20416 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `IO#read(nil, buffer)` doesn't change buffer's encoding, but it does when `maxlen` parameter is specified: ```ruby buffer = "".encode(Encoding::ISO_8859_1) File.open("test.rb").read(1000, buffer).encoding # => #<Encoding:ISO-8859-1> File.open("test.rb").read(nil, buffer).encoding # => #<Encoding:UTF-8> ``` I would expect changing buffer encoding in all the cases. -- https://bugs.ruby-lang.org/

4 5