March 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116218] [Ruby master Bug#20186] Ripper does not generate AST for some keywords in Ruby 3.3
by kzkn (Kazuki Nishikawa) 24 Apr '24

24 Apr '24

Issue #20186 has been reported by kzkn (Kazuki Nishikawa). ---------------------------------------- Bug #20186: Ripper does not generate AST for some keywords in Ruby 3.3 https://bugs.ruby-lang.org/issues/20186 * Author: kzkn (Kazuki Nishikawa) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Ripper does not generate AST for the below keyword: - break - next - redo - retry - yield Ruby 3.2 ``` $ ruby -v ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] $ ruby -rripper -e "p Ripper.sexp('break')" [:program, [[:break, []]]] $ ruby -rripper -e "p Ripper.sexp('next')" [:program, [[:next, []]]] $ ruby -rripper -e "p Ripper.sexp('redo')" [:program, [[:redo]]] $ ruby -rripper -e "p Ripper.sexp('retry')" [:program, [[:retry]]] $ ruby -rripper -e "p Ripper.sexp('yield')" [:program, [[:yield0]]] ``` Ruby 3.3 ``` $ ruby -v ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] $ ruby -rripper -e "p Ripper.sexp('break')" nil $ ruby -rripper -e "p Ripper.sexp('next')" nil $ ruby -rripper -e "p Ripper.sexp('redo')" nil $ ruby -rripper -e "p Ripper.sexp('retry')" nil $ ruby -rripper -e "p Ripper.sexp('yield')" nil ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116687] [Ruby master Bug#20262] Regex mismatch between Ruby 3.2.2 and 3.3.0
by weilandia (Nick Weiland) 24 Apr '24

24 Apr '24

Issue #20262 has been reported by weilandia (Nick Weiland). ---------------------------------------- Bug #20262: Regex mismatch between Ruby 3.2.2 and 3.3.0 https://bugs.ruby-lang.org/issues/20262 * Author: weilandia (Nick Weiland) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This might be a duplicate of https://bugs.ruby-lang.org/issues/20098, but I cannot make it match with the backref so maybe not. Below example matches in 3.2.2 but not in 3.3.0 ``` ruby str = "------------abcdefg------------#3895912" re = /()\1\b\w*[a-zA-Z-]*\d+[\w-]{3,}\w+\b/ re.match?(str) ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116456] [Ruby master Feature#20215] Introduce `IO#readable?`
by ioquatix (Samuel Williams) 22 Apr '24

22 Apr '24

Issue #20215 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #20215: Introduce `IO#readable?` https://bugs.ruby-lang.org/issues/20215 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- There are some cases where, as an optimisation, it's useful to know whether more data is potentially available. We already have `IO#eof?` but the problem with using `IO#eof?` is that it can block indefinitely for sockets. Therefore, code which uses `IO#eof?` to determine if there is potentially more data, may hang. ```ruby def make_request(path = "/") client = connect_remote_host # HTTP/1.0 request: client.write("GET #{path} HTTP/1.0\r\n\r\n") # Read response client.gets("\r\n") # => "HTTP/1.0 200 OK\r\n" # Assuming connection close, there are two things the server can do: # 1. peer.close # 2. peer.write(...); peer.close if client.eof? # <--- Can hang here! puts "Connection closed" # Avoid yielding as we know there definitely won't be any data. else puts "Connection open, data may be available..." # There might be data available, so yield. yield(client) end ensure client&.close end make_request do |client| puts client.read # <--- Prefer to wait here. end ``` The proposed `IO#readable?` is similar to `IO#eof?` but rather than blocking, would simply return false. The expectation is the user will subsequently call `read` which may then wait. The proposed implementation would look something like this: ```ruby class IO def readable? !!self.closed? end end class BasicSocket # Is it likely that the socket is still connected? # May return false positive, but won't return false negative. def readable? return false unless super # If we can wait for the socket to become readable, we know that the socket may still be open. result = self.recv_nonblock(1, MSG_PEEK, exception: false) # No data was available - newer Ruby can return nil instead of empty string: return false if result.nil? # Either there was some data available, or we can wait to see if there is data avaialble. return !result.empty? || result == :wait_readable rescue Errno::ECONNRESET # This might be thrown by recv_nonblock. return false end end ``` For `IO` itself, when there is buffered data, `readable?` would also return true immediately, similar to `eof?`. This is not shown in the above implementation as I'm not sure if there is any Ruby method which exposes "there is buffered data". -- https://bugs.ruby-lang.org/

7 16

[ruby-core:117157] [Ruby master Misc#20336] DevMeeting-2024-04-17
by mame (Yusuke Endoh) 18 Apr '24

18 Apr '24

Issue #20336 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20336: DevMeeting-2024-04-17 https://bugs.ruby-lang.org/issues/20336 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- # The next dev meeting **Date: 2024/04/17 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/04/14. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

10 14

[ruby-core:117341] [Ruby master Feature#20396] ObjectSpace.dump_all(string_value: false): skip dumping the String contents
by byroot (Jean Boussier) 18 Apr '24

18 Apr '24

Issue #20396 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20396: ObjectSpace.dump_all(string_value: false): skip dumping the String contents https://bugs.ruby-lang.org/issues/20396 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- `ObjectSpace.dump_all` is a very useful method to debug memory leaks and such, hence is frequently needed in production. But since all the 7bit strings content is included in the dump, it incur the risk of leaking personal data, or secrets. Also, in many case the strings content isn't that helpful and is just making the dump much bigger for no good reason. And only pure-ASCII strings are dumped this way, which means all the tools that process these dumps should already be compatible with a dump without any string content. ### Feature I propose to add another optional parameter to `dump_all`: `string_value: false`. When passed, no String content is ever dumped regardless of its coderange. Implementation: https://github.com/ruby/ruby/pull/10382 cc @zzak -- https://bugs.ruby-lang.org/

4 7

[ruby-core:116892] [Ruby master Misc#20287] DevMeeting before or after RubyKaigi
by duerst 18 Apr '24

18 Apr '24

Issue #20287 has been reported by duerst (Martin Dürst). ---------------------------------------- Misc #20287: DevMeeting before or after RubyKaigi https://bugs.ruby-lang.org/issues/20287 * Author: duerst (Martin Dürst) * Status: Open * Priority: Normal ---------------------------------------- RubyKaigi itself runs from May 15-17 (Wed-Fri) in Naha. It would be nice to have a DevMeeting before or afterwards. We should try to plan ahead, so that people can make the right flight and hotel reservations early. I have never been to Okinawa before and don't know any facilities that would be available. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:117147] [Ruby master Feature#20335] `Thread.each_caller_location` should accept the same arguments as `caller` and `caller_locations`
by byroot (Jean Boussier) 17 Apr '24

17 Apr '24

Issue #20335 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20335: `Thread.each_caller_location` should accept the same arguments as `caller` and `caller_locations` https://bugs.ruby-lang.org/issues/20335 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- `Thread.each_caller_location` was added to Ruby 3.2 as part of [Feature #16663] and is a very useful API for emitting warnings with a proper source location and similar use cases. However in many of the cases where I used it, or seen it used, it was needed to skip the first, or a couple frames: Examples: Sorbet: https://github.com/Shopify/sorbet/blob/b27a14c247ace7cabdf0f348bfb11fdf0b7e… ```ruby def self.find_caller skiped_first = false Thread.each_caller_location do |loc| unless skiped_first skiped_first = true next end next if loc.path&.start_with?("<internal:") return loc if yield(loc) end nil end ``` @fxn's PR: https://github.com/ruby/ruby/blob/9c2e686719a5a4df5ea0b8a3b6a373ca6003c229/… ```ruby frames_to_skip = 2 location = nil Thread.each_caller_location do |cl| if frames_to_skip >= 1 frames_to_skip -= 1 next end # snipp... ``` ### Proposal I think it would be very useful if `Thread.each_caller_location` accepted the same arguments as `caller` and `caller_locations`: ```ruby #each_caller_location(start = 1, length = nil) #each_caller_location(range) ``` @jeremyevans0 what do you think? -- https://bugs.ruby-lang.org/

6 12

[ruby-core:115911] [Ruby master Bug#20089] Fiber#kill transfers to root fiber
by rmosolgo (Robert Mosolgo) 17 Apr '24

17 Apr '24

Issue #20089 has been reported by rmosolgo (Robert Mosolgo). ---------------------------------------- Bug #20089: Fiber#kill transfers to root fiber https://bugs.ruby-lang.org/issues/20089 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I was hoping to use `Fiber#kill` to clean up formerly `.transfer`-d Fibers and work around https://bugs.ruby-lang.org/issues/20081, but I found that `Fiber#kill` has a similar control flow jump behavior. Is this on purpose, or a bug? Here's a script to test the behavior: ```ruby manager = Fiber.new do worker = Fiber.new do puts "2. Begin Worker" manager.transfer puts "This should never print -- killed" end puts "1. Transfer to Worker" worker.transfer puts "3. Killing Worker" worker.kill puts "4. Finished manager" end manager.transfer puts "5. Finished script" ``` I expected items `1` through `5` to be printed in order, but in fact, `4` is never printed: ``` $ ruby fiber_transfer_test.rb 1. Transfer to Worker 2. Begin Worker 3. Killing Worker 5. Finished script ``` It seems like `worker.kill` is transferring control to the top-level fiber instead of giving it back to `manager`. I also tried having the thread kill _itself_, hoping it would return to the fiber that originally `.transfer`ed to it, but it also seems to jump out: ```ruby manager = Fiber.new do worker = Fiber.new do puts "2. Begin Worker" manager.transfer Fiber.current.kill puts "This should never print -- killed" end puts "1. Transfer to Worker" worker.transfer puts "3. Killing Worker" worker.transfer puts "4. Finished manager" end manager.transfer puts "5. Finished script" ``` Prints: ``` 1. Transfer to Worker 2. Begin Worker 3. Killing Worker 5. Finished script ``` -- https://bugs.ruby-lang.org/

2 5

[ruby-core:117225] [Ruby master Feature#20347] Separate docs task from all
by hsbt (Hiroshi SHIBATA) 17 Apr '24

17 Apr '24

Issue #20347 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Feature #20347: Separate docs task from all https://bugs.ruby-lang.org/issues/20347 * Author: hsbt (Hiroshi SHIBATA) * Status: Assigned * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- I would like to migrate RDoc as bundled gems at Ruby 3.5. We need to handle `install-doc` and related task with that. I removed `docs` task from `all` and re-order `docs` task at `install-all`. https://github.com/ruby/ruby/pull/10282 It works with RDoc as bundled gems. We can keep current behavior with this migration. I hope to merge changes for `all` task and re-order `install-all` task at Ruby 3.4. https://github.com/ruby/ruby/pull/10282/commits/b160083175aed062c320b8d76ea… After that, the default `make` task will not generate rdoc. We need to add `make all docs` for that. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
by eightbitraptor (Matthew Valentine-House) 16 Apr '24

16 Apr '24

Issue #20351 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Feature #20351: Optionally extract common GC routines into a DSO https://bugs.ruby-lang.org/issues/20351 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open ---------------------------------------- [Github PR#10302](https://github.com/ruby/ruby/pull/10302) **NOTE: This proposal does not change the default build of Ruby, and therefore should NOT cause performance degradation for Ruby built in the usual way** Our long term goal is to standardise Ruby's GC interface, allowing alternative GC implementations to be used. This will be acheived by optionally building Ruby's GC as a shared object; enabling it to be replaced at runtime using using `LD_LIBRARY_PATH`. eg: ``` LD_LIBRARY_PATH=/custom_gc_location ruby script.rb ``` This ticket proposes the first step towards this goal. A new experimental build option, `--enable-shared-gc`, that will compile and link a module into the built `ruby` binary as a shared object - `miniruby` will remain statically linked to the existing GC in all cases. Similar methods of replacing functionality relied on by Ruby have precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and `free` at runtime. Although this project will be the first time a technique such as this has been used to replace core Ruby functionality. This flag will be marked as experimental & **disabled by default**. [The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the absolute minimum code required to test it's implementation (a single debug function). The implementation of the new build flag is based on the existing implementation of `--enable-shared` and behaves as follows: - `--enable-shared --enable-shared-gc` This will build both `libruby` and `librubygc` as shared objects. `ruby` will link dynamically to both `libruby` and `librubygc`. - `--disable-shared --enable-shared-gc` This will build `librubygc` as a shared object, and build `libruby` as a static object. `libruby` will link dynamically to `librubygc` and `ruby` will be statically linked to `libruby`. - `--disable-shared-gc` **This will be the default**, and when this case is true the build behaviour will be exactly the same as it is currently. ie. the existing Ruby GC will be built and linked statically into either `ruby` or `libruby.so` depending on the state of `--enable-shared`. We are aware that there will be a small performance penalty from moving the GC logic into a shared object, but this is an opt-in configuration turned on at build time intended to be used by experienced users. Still, we anticipate that, even with this configuration turned on, this penalty will be negligible compared the the benefit that being able to use high performance GC algorithms will provide. This performance penalty is also the reason that **this feature will be disabled by default**. There will be no performance impact for anyone compiling Ruby in the usual manner, without explicitly enabling this feature. We have discussed this proposal with @matz who has approved our work on this project - having a clear abstraction between the VM and the GC will enable us to iterate faster on improvements to Ruby's existing GC. ## Motivation In the long term we want to provide the ability to override the current Ruby GC implementation in order to: * Experiment with modern high-performance GC implementations, such as Immix, G1, LXR etc. * Easily split-test changes to the GC, or the GC tuning, in production without having to rebuild Ruby * Easily use debug builds of the GC to help identify production problems and bottlenecks without having to rebuild Ruby * Encourage the academic memory management research community to consider Ruby for their research (the current work on [MMTk & Ruby]() is a good example of this). ## Future work The initial implementation of the shared GC module in this PR is deliberately small, and exists only for testing the build system integration. The next steps are to identify boundaries between the GC and the VM and begin to extract common functionality into this GC wrapper module to serve as the foundation of our GC interface. ## Who's working on this - Matt Valentine-House (@eightbitraptor) - Aaron Patterson (@tenderlove) - Peter Zhu(@peterzhu2118) - Eileen Uchitelle(@eileencodes) -- https://bugs.ruby-lang.org/

2 4