June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118216] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Eregon (Benoit Daloze) 06 Jun '24

06 Jun '24

Issue #6648 has been updated by Eregon (Benoit Daloze). @mame CRuby already needs to get arguments as an array to parse command-line flags, so `RbConfig.ruby_args` just exposes that. If CRuby can parse these Ruby command-line flags, for sure we can save them in some kind of array. IIRC these extra complications are only relevant in `.bat` files, the C main still receives an array of arguments on Windows. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108705 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117866] [Ruby master Misc#20488] Document source file size restrictions
by kddnewton (Kevin Newton) 06 Jun '24

06 Jun '24

Issue #20488 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20488: Document source file size restrictions https://bugs.ruby-lang.org/issues/20488 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I was hoping we might be able to decide on an official maximum size for a source file in terms of bytes/lines/columns/etc. Ruby uses fixed integer sizes to represent line numbers, column numbers, offsets, etc. These can overflow with files that are too big. Sometimes they will fail with cryptic messages like: ``` ruby: negative string size (or size too big) (ArgumentError) ``` (That's for 2^31 "a"s in a row.) Note that for python for the same file you will get: ``` OverflowError: Parser column offset overflow - source line is too big ``` and for perl you will get: ``` Identifier too long at test.pl line 1. ``` For files with 2^32 newlines, Ruby just crashes on my machine with the current parser (prism finishes, but its newline counter overflows so it gets all the offsets wrong). Would it be okay to say: * maximum line: 31 bits * maximum column: 32 bits * maximum file byte size: 32 bits This would also help with memory savings — I would like to only use 32 bits for offsets in the file, as opposed to the current 64 bits I'm using (which seems unnecessarily large). -- https://bugs.ruby-lang.org/

1 1

[ruby-core:118212] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by mame (Yusuke Endoh) 06 Jun '24

06 Jun '24

Issue #6648 has been updated by mame (Yusuke Endoh). I am afraid if it is more difficult than expected to do "launch subprocess Ruby instances with the same settings". I am not very familiar with Windows, but I have heard that there is no concept of "an array of command-line arguments" in Windows. A command line is represented as a single string. On Windows, `system("exe", "ary1", "ary2")` is converted to a single string and executed via the shell (sometimes, I am not sure the condition). This exotic command line argument handling in Windows can lead to [vulnerabilities](https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/). What I'm trying to say is, it could be difficult to guarantee `exec(RbConfig.ruby, *RbConfig.ruby_args, RbConfig.script, *RbConfig.script_args)` will always achieve "launch subprocess Ruby instances with the same settings". If you really want to "launch subprocess Ruby instances with the same settings", we might want to consider a more dedicated API for it, instead of parsing the command line to a string array and passing it to `Kernel#exec`. ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108700 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117535] [Ruby master Misc#20432] Proposal for workflow changes related to teeny releases
by ufuk (Ufuk Kayserilioglu) 06 Jun '24

06 Jun '24

Issue #20432 has been reported by ufuk (Ufuk Kayserilioglu). ---------------------------------------- Misc #20432: Proposal for workflow changes related to teeny releases https://bugs.ruby-lang.org/issues/20432 * Author: ufuk (Ufuk Kayserilioglu) * Status: Open ---------------------------------------- ## Aim - The cadence of CRuby stable releases is very important for how the project is perceived. Users of CRuby want to get more frequent releases with bug and security fixes, so that they feel confident that their projects and businesses continue running smoothly and safely. - More frequent releases make the community see that the CRuby project is active and thriving. It also allows people to keep upgrading to the latest versions of Ruby with newer features and better security. - The current cadence of teeny releases are causing concerns in the community, and there have been multiple examples of late releases in [3.0 series](https://www.reddit.com/r/ruby/comments/r14z39/ruby_303_released/), [3.1 and 3.2 series](https://www.reddit.com/r/ruby/comments/18n54es/why_have_they_not_up… and now [3.3 series](https://bugs.ruby-lang.org/issues/20085). This is creating confusion and doubt in our community and making the community to [ask for a change in bug fix release process](https://bugs.ruby-lang.org/issues/20422). - We all agree that release management is difficult and branch maintainers have a hard job. If we can be making their jobs a little bit easier, they will have more time and opportunities to make more frequent releases. - This proposal aims to improve some of the workflows around stable branch release management so that branch maintainers have the opportunity to make more frequent bug fix and security releases, thus, addressing some of the concerns from the community. ## Proposal(s) 1. A lot of the time of a release manager seems to be spent doing backports to the corresponding stable branch. The current workflow asks bug reports to populate the fields for which stable versions need the fix backported to, and release managers are responsible for keeping up with this list and creating backport commits on stable branches. This process is fragile, since the release managers are usually the people with the least amount of context on the particulars of a bug-fix and have to apply the same changes to a different branch and resolve any potential conflicts. This is a task best done by people who already have the most amount of context on the change: the original bugfix author. As a result, my suggestion is to make it mandatory for any change that needs to be backported to have backport PRs opened (or corresponding backport patches attached to the Redmine tickets) by the author for the relevant stable branches. That way the authors of changes will be responsible for making sure that the tests pass and the changes apply cleanly on older branches, allowing branch maintainers to come in and merge the PRs, freeing up their time and focus. Since backport PRs will be opened by authors of changes, but only merged by branch maintainers, this will still allow maintainers to control what goes into stable branches. 2. It also seems like branch maintainers are trying to synchronize releases across different Ruby versions, which is causing delays in getting teeny releases out of the door. In my opinion, each stable branch should be responsible for its own teeny releases and there should be no need to synchronize the cadence between different Ruby versions. This will mean less time waiting for other branch maintainers who might be busy with other life priorities, unblocking branch maintainers to be able to do teeny releases whenever it is convenient. 3. Finally, it is my observation that branch maintainers usually make teeny releases whenever there is a security incident that needs to be addressed. A security fix is most definitely a good reason to make a teeny release, but a teeny release should not wait for a security concern to be addressed. There are many other important bug fixes constantly being backported (or needing backports) such as fixes for crashes, fixes for memory retention/leaks or fixes for other behaviour regressions. It is important for end-users to have those addressed as timely as possible, so I would suggest that branch maintainers aim to make about 6-7 teeny releases every year (some of which will be security releases) in order to deliver value to end-users continuously. ## Conclusions Continuous integration and continuous delivery are one of the greatest changes that have happened in the software engineering sector in the recent decades. They allow us to shorten the time of delivery of new features to end-users and reduce the amount of work-in-progress that is waiting for come into the work, thus reducing waste. By adopting some of the workflow changes that I have suggested above, and maybe other changes that I might have missed, the CRuby core team has an opportunity to make a positive impact on all the users of the Ruby language and to increase the quality perception of the project as a result. For that reason, I hope my proposals are considered and implemented by the Core team. Thanks for your consideration. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117470] [Ruby master Bug#20416] IO#read doesn't preserve buffer encoding if `maxlen = nil`
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20416 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20416: IO#read doesn't preserve buffer encoding if `maxlen = nil` https://bugs.ruby-lang.org/issues/20416 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `IO#read(nil, buffer)` doesn't change buffer's encoding, but it does when `maxlen` parameter is specified: ```ruby buffer = "".encode(Encoding::ISO_8859_1) File.open("test.rb").read(1000, buffer).encoding # => #<Encoding:ISO-8859-1> File.open("test.rb").read(nil, buffer).encoding # => #<Encoding:UTF-8> ``` I would expect changing buffer encoding in all the cases. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:115334] [Ruby master Feature#19998] Emit deprecation warnings when the old (non-Typed) Data_XXX API is used
by byroot (Jean Boussier) 06 Jun '24

06 Jun '24

Issue #19998 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19998: Emit deprecation warnings when the old (non-Typed) Data_XXX API is used https://bugs.ruby-lang.org/issues/19998 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- The replacement API was introduced in Ruby 1.9.2 (2010) [Feature #3064], and the old untyped data API was marked a deprecated in the documentation as of Ruby 2.3.0 (2015): https://github.com/ruby/ruby/commit/98544c372d948717de22afc86d162e411f1fb5f1 However, as of today, no deprecation warning of any sort is emitted when the API is used. Ultimately removing the old untyped API would allow to easily reclaim 8B on every `T_DATA` which could be used by the newly introduced embedded TypedData objects https://github.com/ruby/ruby/pull/7440 Was there any discussions about removing the old API at some point? Could we emit a warning when the API is used? Either verbose or non-verbose, to at least surface the issue to gem owners? This would help drive the effort to fix outdated gems, allowing to eventually remove this old API in the future. Looking at our big applications dependencies, there seem to be a small minority of gems still using the old API: - https://rubygems.org/gems/unf_ext - https://rubygems.org/gems/rotoscope - https://rubygems.org/gems/re2 - https://rubygems.org/gems/mysql2 Migrating them over doesn't seem too hard. -- https://bugs.ruby-lang.org/

6 11

[ruby-core:117027] [Ruby master Bug#20319] Singleton class is being frozen lazily in some cases
by andrykonchin (Andrew Konchin) 06 Jun '24

06 Jun '24

Issue #20319 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #20319: Singleton class is being frozen lazily in some cases https://bugs.ruby-lang.org/issues/20319 * Author: andrykonchin (Andrew Konchin) * Status: Open * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've noticed suspicious behaviour (it doesn't affect anything in practice for me though) when an object becomes frozen only its own singleton class becomes frozen immediately. A singleton class of the immediate singleton class becomes frozen lazily after `#singleton_class` method call: ```ruby o = Object.new klass = o.singleton_class.singleton_class o.freeze puts klass.frozen? # false <== here we expect true puts o.singleton_class.singleton_class.frozen? # true puts klass.frozen? # true ``` I would expect all created singleton classes in an object singleton classes chain to become frozen immediately. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:114038] [Ruby master Bug#19749] Confirm correct behaviour when attaching private method with `#define_method`
by itarato (Peter Arato) 06 Jun '24

06 Jun '24

Issue #19749 has been reported by itarato (Peter Arato). ---------------------------------------- Bug #19749: Confirm correct behaviour when attaching private method with `#define_method` https://bugs.ruby-lang.org/issues/19749 * Author: itarato (Peter Arato) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This issue is a special case of https://bugs.ruby-lang.org/issues/19745: Should dynamically added private methods via `.singleton_class.send(:define_method,...` be accessible publicly? See the following example? ```ruby private def bar; end foo = Object.new foo.singleton_class.send(:define_method, :bar, method(:bar)) foo.bar # No error. ``` The script above runs fine on latest Ruby 3.3. Is this correct to ignore the fact that the added method (method(:bar)) is private? This came up during a TruffleRuby investigation (https://github.com/oracle/truffleruby/issues/3134) where the result for the same script is: `private method 'bar' called for #<Object:0xc8> (NoMethodError)` -- https://bugs.ruby-lang.org/

5 14

[ruby-core:117990] [Ruby master Misc#20504] Interpolated string literal in regexp encoding handling
by kddnewton (Kevin Newton) 06 Jun '24

06 Jun '24

Issue #20504 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20504: Interpolated string literal in regexp encoding handling https://bugs.ruby-lang.org/issues/20504 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- There is some very odd behavior that I'm not sure is intentional or not, so I'm looking for guidance. In here: ```ruby # encoding: us-ascii interp = "\x80" regexp = /#{interp}/ ``` the `regexp` variable is a ascii-8bit regular expression with the byte interpolated into the middle. However, if you inline that interpolation: ```ruby # encoding: us-ascii regexp = /#{"\x80"}/ ``` you get a syntax error, saying it's an invalid multi-byte character. I'm not sure what the rule is here, as it seems inconsistent. Is this the correct behavior? I would prefer if it would create an ascii-8bit regular expression like the first example, which would be consistent. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118194] [Ruby master Bug#18622] const_get still looks in Object, while lexical constant lookup no longer does
by matz (Yukihiro Matsumoto) 06 Jun '24

06 Jun '24

Issue #18622 has been updated by matz (Yukihiro Matsumoto). Status changed from Open to Closed Sorry, I forgot to reply. Due to the compatibility concern, I'd like to keep this behavior even though it's inconsistent with direct constant accesses. Matz. ---------------------------------------- Bug #18622: const_get still looks in Object, while lexical constant lookup no longer does https://bugs.ruby-lang.org/issues/18622#change-108677 * Author: Eregon (Benoit Daloze) * Status: Closed * ruby -v: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux] * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- There is some inconsistency here between literal constant lookup and the meta API (const_get). Lexical constant lookup no longer uses a special case for Object, and this is good as it avoids surprises: #11547 However, `const_get` still looks in Object, even though that's confusing, inconsistent and IMHO shouldn't really happen. ```ruby module ConstantSpecsTwo Foo = :cs_two_foo end module ConstantSpecs end p ConstantSpecs.const_get("ConstantSpecsTwo::Foo") # => :cs_two_foo p ConstantSpecs::ConstantSpecsTwo::Foo # => const_get.rb:9:in `<main>': uninitialized constant ConstantSpecs::ConstantSpecsTwo (NameError) ``` I think we should change it so both behave the same (i.e., NameError). It's like if `cd /foo/bar` would go to `/bar` if `/foo/bar` does not exist and `/bar` does. `const_get` is a meta API so it cannot know the surrounding `Module.nesting`, but so I think it should consider the receiver of `const_get` as the only nesting (so just `ConstantSpecs` in this case, not `Object`). Note this does not affect nested constants inside the `const_get` like `Foo` above (another way to look at the inconsistency that the first component is treated differently). From https://bugs.ruby-lang.org/issues/11547#note-19 -- https://bugs.ruby-lang.org/

1 0