April 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116838] [Ruby master Bug#20280] Error at invalid encoding symbols
by nobu (Nobuyoshi Nakada) 15 Jun '24

15 Jun '24

Issue #20280 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20280: Error at invalid encoding symbols https://bugs.ruby-lang.org/issues/20280 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Parsing an invalid encoding symbol raises an `EncodingError`. It must be a `SyntaxError`. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116790] [Ruby master Bug#20270] Options with `--parser=prism`
by nobu (Nobuyoshi Nakada) 11 Jun '24

11 Jun '24

Issue #20270 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20270: Options with `--parser=prism` https://bugs.ruby-lang.org/issues/20270 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ### `--dump` option Currently `parsetree` and `prism_parsetree` bits are separated, but it seems meaningless as far as `--parser` option selects only one parser. Why doesn't simply `--dump=parsetree` dump AST by parse.y, or PRISM AST if `--parser=prism` is given? Actually, `ruby --dump=parsetree --parser=prism` just segfaults, because these bits and `--parser` option are separately handled. ### streaming code from stdin ```sh-session $ echo end | ruby --parser=prism - ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker. -: warning: Prism support for streaming code from stdin is not currently supported ``` Nothing reported is fine, since it is not currently supported. I was a bit curious that it looks like trying something after the warning. ```C if (strcmp(opt->script, "-") == 0) { int xflag = opt->xflag; VALUE rb_source = open_load_file(opt->script_name, &xflag); opt->xflag = xflag != 0; rb_warn("Prism support for streaming code from stdin is not currently supported"); error = pm_parse_string(&result, rb_source, opt->script_name); } ``` Note that `open_load_file` returns `rb_stdin` when `-` is given. This object is a `RFile`, not a `RString`, and `RSTRING_PTR(rb_source)` accesses after the object boundary. It should just bail out, not try obviously wrong thing. ### Fix https://github.com/nobu/ruby/tree/options-refactor -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116039] [Ruby master Bug#20154] aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment
by jprokop (Jarek Prokop) 09 Jun '24

09 Jun '24

Issue #20154 has been reported by jprokop (Jarek Prokop). ---------------------------------------- Bug #20154: aarch64: configure overrides `-mbranch-protection` if it was set in CFLAGS via environment https://bugs.ruby-lang.org/issues/20154 * Author: jprokop (Jarek Prokop) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Recently a GH PR was merged <https://github.com/ruby/ruby/pull/9306> For PAC/BTI support on ARM CPUs for Coroutine.S. Without proper compilation support in configure.ac it segfaults Ruby with fibers on CPUs where PAC is supported: https://bugs.ruby-lang.org/issues/20085 At the time of writing, configure.ac appends the first option from a list for flag `-mbranch-protection` that successfully compiles a program <https://github.com/ruby/ruby/blob/master/configure.ac#L829>, to XCFLAGS and now also ASFLAGS to fix issue 20085 for Ruby master. This is suboptimal for Fedora as we set -mbranch-protection=standard by default in C{,XX}FLAGS: ``` CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Werror=implicit-function-declaration -Werror=implicit-int -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ' export CFLAGS CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' export CXXFLAGS ``` And the appended flag overrides distribution's compilation configuration, which in this case ends up omitting BTI instructions and only using PAC. Would it make sense to check if such flags exist and not overwrite them if they do? Serious proposals: 1. Simplest fix that does not overwrite what is set in the distribution and results in higher security is simply prepending the list of options with `-mbranch-protection=standard`, it should cause no problems on ARMv8 CPUs and forward, BTI similarly to PAC instructions result into NOP, it is only extending the capability. See attached 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch 2. Other fix that sounds more sane IMO and dodges this kind of guessing where are all the correct places for the flag is what another Fedora contributor Florian Weimer suggested: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… "The reliable way to do this would be to compile a C file and check whether that enables __ARM_FEATURE_PAC_DEFAULT, and if that's the case, define a *different* macro for use in the assembler implementation. This way, you don't need to care about the exact name of the option." IOW instead of using __ARM_FEATURE_* directly in that code, define a macro in the style of "USE_PAC" with value of the feature if it is defined, I think that way we shouldn't need to append ASFLAGS anymore. However it's also important to catch the value of those macros as their values have meaning, I have an idea how to do that but I'd get on that monday earliest. ---Files-------------------------------- 0001-aarch64-Check-mbranch-protection-standard-first-to-u.patch (1004 Bytes) -- https://bugs.ruby-lang.org/

4 10

[ruby-core:116460] [Ruby master Bug#20218] aset/masgn/op_asgn with keyword arguments
by jeremyevans0 (Jeremy Evans) 07 Jun '24

07 Jun '24

Issue #20218 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20218: aset/masgn/op_asgn with keyword arguments https://bugs.ruby-lang.org/issues/20218 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found that use of keyword arguments in multiple assignment is broken in 3.3 and master: ```ruby h = {a: 1} o = [] def o.[]=(*args, **kw) replace([args, kw]) end # This segfaults as RHS argument is not a hash o[1, a: 1], _ = [1, 2] # This passes the RHS argument as keywords to the method, treating keyword splat as positional argument o[1, **h], _ = [{b: 3}, 2] o # => [[1, {:a=>1}], {:b=>3}] ``` Before 3.3, keyword arguments were treated as positional arguments. This is similar to #19918, but for keyword arguments instead of block arguments. @matz indicated he wanted to prohibit block arguments in aset/masgn and presumably also op_asgn (making them SyntaxErrors). Can we also prohibit keyword arguments in aset/masgn/op_asgn? Note that aset treats keyword arguments as regular arguments: ```ruby o[1, a: 1] = 2 o # => [[1, {:a=>1}, 2], {}] o[1, **h] = {b: 3} o # => [[1, {:a=>2}, {:b=>3}], {}] ``` While op_asgn treats keyword arguments as keywords: ```ruby h = {a: 2} o = [] def o.[](*args, **kw) concat([:[], args, kw]) x = Object.new def x.+(v) [:x, v] end x end def o.[]=(*args, **kw) concat([:[]=, args, kw]) end o[1, a: 1] += 2 o # => [:[], [1], {:a=>1}, :[]=, [1, [:x, 2]], {:a=>1}] o.clear o[1, **h] += {b: 3} o # => [:[], [1], {:a=>2}, :[]=, [1, [:x, {:b=>3}]], {:a=>2}] ``` -- https://bugs.ruby-lang.org/

7 17

[ruby-core:116117] [Ruby master Misc#20170] Drop support for GCC < 11
by kddnewton (Kevin Newton) 07 Jun '24

07 Jun '24

Issue #20170 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20170: Drop support for GCC < 11 https://bugs.ruby-lang.org/issues/20170 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:115595] [Ruby master Bug#20043] `defined?` checks for method existence but only sometimes
by tenderlovemaking (Aaron Patterson) 07 Jun '24

07 Jun '24

Issue #20043 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Bug #20043: `defined?` checks for method existence but only sometimes https://bugs.ruby-lang.org/issues/20043 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-05T21:25:34Z master 56eccb350b) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When an expression is passed to `defined?`, it will _sometimes_ check if a method in a sub-expression is defined and sometimes it won't. For example: ``` $ ./miniruby -e'p defined?(a)' nil $ ./miniruby -e'p defined?([a])' nil ``` In the above case, Ruby will check whether or not the method `a` is defined, and it returns `nil`. However, if you use a splat, it will not check: ``` $ ./miniruby -e'p defined?([*a])' "expression" ``` The same thing seems to happen with method parameters: ``` $ ./miniruby -e'p defined?(itself)' "method" $ ./miniruby -e'p defined?(itself(a))' nil $ ./miniruby -e'p defined?(itself(*a))' "method" ``` Oddly, `defined?` will check contents of arrays, but _won't_ check contents of hashes: ``` $ ./miniruby -e'p defined?([[[[a]]]])' nil $ ./miniruby -e'p defined?({ a => a })' "expression" ``` I think all of the cases that refer to `a` should check whether or not `a` is defined regardless of splats or hashes. -- https://bugs.ruby-lang.org/

7 9

[ruby-core:117535] [Ruby master Misc#20432] Proposal for workflow changes related to teeny releases
by ufuk (Ufuk Kayserilioglu) 06 Jun '24

06 Jun '24

Issue #20432 has been reported by ufuk (Ufuk Kayserilioglu). ---------------------------------------- Misc #20432: Proposal for workflow changes related to teeny releases https://bugs.ruby-lang.org/issues/20432 * Author: ufuk (Ufuk Kayserilioglu) * Status: Open ---------------------------------------- ## Aim - The cadence of CRuby stable releases is very important for how the project is perceived. Users of CRuby want to get more frequent releases with bug and security fixes, so that they feel confident that their projects and businesses continue running smoothly and safely. - More frequent releases make the community see that the CRuby project is active and thriving. It also allows people to keep upgrading to the latest versions of Ruby with newer features and better security. - The current cadence of teeny releases are causing concerns in the community, and there have been multiple examples of late releases in [3.0 series](https://www.reddit.com/r/ruby/comments/r14z39/ruby_303_released/), [3.1 and 3.2 series](https://www.reddit.com/r/ruby/comments/18n54es/why_have_they_not_up… and now [3.3 series](https://bugs.ruby-lang.org/issues/20085). This is creating confusion and doubt in our community and making the community to [ask for a change in bug fix release process](https://bugs.ruby-lang.org/issues/20422). - We all agree that release management is difficult and branch maintainers have a hard job. If we can be making their jobs a little bit easier, they will have more time and opportunities to make more frequent releases. - This proposal aims to improve some of the workflows around stable branch release management so that branch maintainers have the opportunity to make more frequent bug fix and security releases, thus, addressing some of the concerns from the community. ## Proposal(s) 1. A lot of the time of a release manager seems to be spent doing backports to the corresponding stable branch. The current workflow asks bug reports to populate the fields for which stable versions need the fix backported to, and release managers are responsible for keeping up with this list and creating backport commits on stable branches. This process is fragile, since the release managers are usually the people with the least amount of context on the particulars of a bug-fix and have to apply the same changes to a different branch and resolve any potential conflicts. This is a task best done by people who already have the most amount of context on the change: the original bugfix author. As a result, my suggestion is to make it mandatory for any change that needs to be backported to have backport PRs opened (or corresponding backport patches attached to the Redmine tickets) by the author for the relevant stable branches. That way the authors of changes will be responsible for making sure that the tests pass and the changes apply cleanly on older branches, allowing branch maintainers to come in and merge the PRs, freeing up their time and focus. Since backport PRs will be opened by authors of changes, but only merged by branch maintainers, this will still allow maintainers to control what goes into stable branches. 2. It also seems like branch maintainers are trying to synchronize releases across different Ruby versions, which is causing delays in getting teeny releases out of the door. In my opinion, each stable branch should be responsible for its own teeny releases and there should be no need to synchronize the cadence between different Ruby versions. This will mean less time waiting for other branch maintainers who might be busy with other life priorities, unblocking branch maintainers to be able to do teeny releases whenever it is convenient. 3. Finally, it is my observation that branch maintainers usually make teeny releases whenever there is a security incident that needs to be addressed. A security fix is most definitely a good reason to make a teeny release, but a teeny release should not wait for a security concern to be addressed. There are many other important bug fixes constantly being backported (or needing backports) such as fixes for crashes, fixes for memory retention/leaks or fixes for other behaviour regressions. It is important for end-users to have those addressed as timely as possible, so I would suggest that branch maintainers aim to make about 6-7 teeny releases every year (some of which will be security releases) in order to deliver value to end-users continuously. ## Conclusions Continuous integration and continuous delivery are one of the greatest changes that have happened in the software engineering sector in the recent decades. They allow us to shorten the time of delivery of new features to end-users and reduce the amount of work-in-progress that is waiting for come into the work, thus reducing waste. By adopting some of the workflow changes that I have suggested above, and maybe other changes that I might have missed, the CRuby core team has an opportunity to make a positive impact on all the users of the Ruby language and to increase the quality perception of the project as a result. For that reason, I hope my proposals are considered and implemented by the Core team. Thanks for your consideration. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117470] [Ruby master Bug#20416] IO#read doesn't preserve buffer encoding if `maxlen = nil`
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20416 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20416: IO#read doesn't preserve buffer encoding if `maxlen = nil` https://bugs.ruby-lang.org/issues/20416 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `IO#read(nil, buffer)` doesn't change buffer's encoding, but it does when `maxlen` parameter is specified: ```ruby buffer = "".encode(Encoding::ISO_8859_1) File.open("test.rb").read(1000, buffer).encoding # => #<Encoding:ISO-8859-1> File.open("test.rb").read(nil, buffer).encoding # => #<Encoding:UTF-8> ``` I would expect changing buffer encoding in all the cases. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:115334] [Ruby master Feature#19998] Emit deprecation warnings when the old (non-Typed) Data_XXX API is used
by byroot (Jean Boussier) 06 Jun '24

06 Jun '24

Issue #19998 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19998: Emit deprecation warnings when the old (non-Typed) Data_XXX API is used https://bugs.ruby-lang.org/issues/19998 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- The replacement API was introduced in Ruby 1.9.2 (2010) [Feature #3064], and the old untyped data API was marked a deprecated in the documentation as of Ruby 2.3.0 (2015): https://github.com/ruby/ruby/commit/98544c372d948717de22afc86d162e411f1fb5f1 However, as of today, no deprecation warning of any sort is emitted when the API is used. Ultimately removing the old untyped API would allow to easily reclaim 8B on every `T_DATA` which could be used by the newly introduced embedded TypedData objects https://github.com/ruby/ruby/pull/7440 Was there any discussions about removing the old API at some point? Could we emit a warning when the API is used? Either verbose or non-verbose, to at least surface the issue to gem owners? This would help drive the effort to fix outdated gems, allowing to eventually remove this old API in the future. Looking at our big applications dependencies, there seem to be a small minority of gems still using the old API: - https://rubygems.org/gems/unf_ext - https://rubygems.org/gems/rotoscope - https://rubygems.org/gems/re2 - https://rubygems.org/gems/mysql2 Migrating them over doesn't seem too hard. -- https://bugs.ruby-lang.org/

6 11

[ruby-core:117027] [Ruby master Bug#20319] Singleton class is being frozen lazily in some cases
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20319 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20319: Singleton class is being frozen lazily in some cases https://bugs.ruby-lang.org/issues/20319 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've noticed suspicious behaviour (it doesn't affect anything in practice for me though) when an object becomes frozen only its own singleton class becomes frozen immediately. A singleton class of the immediate singleton class becomes frozen lazily after `#singleton_class` method call: ```ruby o = Object.new klass = o.singleton_class.singleton_class o.freeze puts klass.frozen? # false <== here we expect true puts o.singleton_class.singleton_class.frozen? # true puts klass.frozen? # true ``` I would expect all created singleton classes in an object singleton classes chain to become frozen immediately. -- https://bugs.ruby-lang.org/

4 7