September 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119161] [Ruby master Bug#20735] prism emits "ambiguous `&`" warning
by mame (Yusuke Endoh) 13 Sep '24

13 Sep '24

Issue #20735 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20735: prism emits "ambiguous `&`" warning https://bugs.ruby-lang.org/issues/20735 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: kddnewton (Kevin Newton) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ``` $ ./local/bin/ruby -w --parser=parse.y -e 'def foo(&) = nil; foo &:bar' $ ./local/bin/ruby -w --parser=prism -e 'def foo(&) = nil; foo &:bar' -e:1: warning: ambiguous `&` has been interpreted as an argument prefix ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119180] [Ruby master Bug#20740] prism_compile coding issue for PM_FORWARDING_SUPER_NODE
by luke-gru (Luke Gruber) 13 Sep '24

13 Sep '24

Issue #20740 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #20740: prism_compile coding issue for PM_FORWARDING_SUPER_NODE https://bugs.ruby-lang.org/issues/20740 * Author: luke-gru (Luke Gruber) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found a coding issue in prism_compile.c but I can't find a bug that would result from it. The `current_block` is not set back when a block is given. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:119152] [Ruby master Bug#20731] warning: mismatched indentations at 'when' appears with prism enabled ruby
by yahonda (Yasuo Honda) 13 Sep '24

13 Sep '24

Issue #20731 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #20731: warning: mismatched indentations at 'when' appears with prism enabled ruby https://bugs.ruby-lang.org/issues/20731 * Author: yahonda (Yasuo Honda) * Status: Open * ruby -v: ruby 3.4.0dev (2024-09-12T20:03:28Z master 0fc8422a05) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ### Steps to reproduce 1. Create a `case_sample.rb` file ``` foo = :bar case foo when :bar p 'foo is :bar' when :baz # do something else # do something end ``` 2. Run this file against the latest Ruby 3.4.0dev with -w option to show warnings ``` $ ruby -w case_sample.rb ``` ### Expected behavior It should show `"foo is :bar"` without any warnings as it runs against Ruby 3.3.5 or Ruby 3.4.0dev before prism is used by default ``` $ ruby -w case_sample.rb "foo is :bar" $ ruby -v ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux] $ ``` ``` $ ruby -w case_sample.rb "foo is :bar" $ ruby -v ruby 3.4.0dev (2024-09-12T17:04:10Z master f2919bd11c) [x86_64-linux] $ ``` ### Actual behavior It raises `warning: mismatched indentations at 'when' with 'case' ` warnings as follows. ``` $ ruby -w case_sample.rb case_sample.rb:3: warning: mismatched indentations at 'when' with 'case' at 2 case_sample.rb:5: warning: mismatched indentations at 'when' with 'case' at 2 "foo is :bar" $ ruby -v ruby 3.4.0dev (2024-09-12T20:03:28Z master 0fc8422a05) +PRISM [x86_64-linux] $ ``` ### Background This repro is based on the tzinfo code below. https://github.com/tzinfo/tzinfo/blob/d5893c99bbcd842af86023f02a84fac1659b3… https://github.com/tzinfo/tzinfo/blob/d5893c99bbcd842af86023f02a84fac1659b3… -- https://bugs.ruby-lang.org/

4 5

[ruby-core:119154] [Ruby master Bug#20732] prism emits "mismatched indentations" warning in eval
by mame (Yusuke Endoh) 13 Sep '24

13 Sep '24

Issue #20732 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20732: prism emits "mismatched indentations" warning in eval https://bugs.ruby-lang.org/issues/20732 * Author: mame (Yusuke Endoh) * Status: Open * Assignee: kddnewton (Kevin Newton) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby eval(<<-EOC) if @check end EOC ``` ``` $ ruby --parser=parse.y -w t.rb $ ruby --parser=prism -w t.rb (eval at t.rb:1):2: warning: mismatched indentations at 'end' with 'if' at 1 ``` This issue actually occurs in rss gem. https://github.com/ruby/rss/blob/8a5b056eab591525e9c14ae1cd6e97ebae386be8/l… https://github.com/ruby/rss/blob/8a5b056eab591525e9c14ae1cd6e97ebae386be8/l… -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119132] Segfault using ruby C on MacOS (Intel Catalina and M2 Sonoma)
by martin.kufner＠quiz.baby 13 Sep '24

13 Sep '24

Hey guys, I was experienced at C about 20 years ago, since 15 years I'm coding with Ruby. I now need to write a Ruby C extension, and I fail with just calling simple commands like rb_p. I try to integrate a fast C websocket server into my quiz website (Rails cable is too slow) In order to communicate I need a UNIX socket in C to receive commands from the Rails server. I used the xcode standard compiler, and also tried homebrew gcc-14 It compiles fine all set up with extconf.rb When I call QbGameServer.init, it creates the server, and after a second wait tries to connect. The c thread is created and gets the message, However when I call rb_p or rb_funcall it hangs (M2) or segfauls (Intel) What do I do wrong? Thanks a lot! Martin I created a QbGameServerModule, extending my QbGameServer and call module QbGameServer SOCKET_PATH="/tmp/sock/qb-game-server" extend QbGameServerModule def self.init # Start the server (non-blocking) self.start_rpc(SOCKET_PATH) do |message| puts "Received from client: #{message}" end sleep 1 send_socket end def self.send_socket # Create a new Unix socket socket = UNIXSocket.new(SOCKET_PATH) # Example data to send to the server (Ruby hash converted to JSON) data_to_send = { message: "Hello, server!", timestamp: Time.now.to_s }.to_json # Send the JSON data to the server puts "Sent to server: #{data_to_send}" socket.puts(data_to_send) # Read the response from the server response = socket.gets.chomp # Read the response from the socket puts "Received from server: #{response}" ensure # Close the socket connection after the communication is done socket.close if socket end end On the C side I have the ext/qb_game_server/qb_game_server.c #include "ruby.h" #include <sys/types.h> #include <sys/socket.h> #include <sys/un.h> #include <unistd.h> #include <pthread.h> #include <string.h> #define MAX_CLIENTS 10 #define BUFFER_SIZE 1024 VALUE QbGameServerModule = Qnil; static VALUE rpc_proc = Qnil; static char socket_file_path[108]; // Maximum length for UNIX domain socket path void *client_handler(void *sock) { int client_socket = *(int *)sock; char buffer[BUFFER_SIZE]; ssize_t bytes_read; while ((bytes_read = read(client_socket, buffer, sizeof(buffer) - 1)) > 0) { buffer[bytes_read] = '\0'; printf("Received C %s\n", buffer); VALUE arg = rb_str_new_cstr(buffer); // rb_gc_register_address(&arg); printf("1\n"); // DOES REACH // rb_gc(); printf("2\n"); // DOES REACH rb_p(arg); // ***** DOES NOT REACH HERE***** // On the osx sonoma M2 it just hangs, on osx Intel Catalina it shows a segfault printf("3\n"); // rb_gc_unregister_address(&arg); // rb_funcall(rpc_proc, rb_intern("call"), 1, arg); } close(client_socket); return NULL; } void *server_thread(void *arg) { int server_socket = socket(AF_UNIX, SOCK_STREAM, 0); struct sockaddr_un server_addr; int client_sockets[MAX_CLIENTS]; pthread_t threads[MAX_CLIENTS]; int client_count = 0; memset(&server_addr, 0, sizeof(server_addr)); server_addr.sun_family = AF_UNIX; strncpy(server_addr.sun_path, socket_file_path, sizeof(server_addr.sun_path) - 1); unlink(socket_file_path); bind(server_socket, (struct sockaddr *)&server_addr, sizeof(server_addr)); listen(server_socket, MAX_CLIENTS); while (1) { client_sockets[client_count] = accept(server_socket, NULL, NULL); pthread_create(&threads[client_count], NULL, client_handler, &client_sockets[client_count]); client_count++; } close(server_socket); return NULL; } static VALUE start_rpc(VALUE self, VALUE socket_file) { Check_Type(socket_file, T_STRING); strncpy(socket_file_path, StringValueCStr(socket_file), sizeof(socket_file_path) - 1); if (rb_block_given_p()) { rpc_proc = rb_block_proc(); } else { rb_raise(rb_eArgError, "A block must be provided as a callback"); } pthread_t server_tid; pthread_create(&server_tid, NULL, server_thread, NULL); return Qnil; } void Init_qb_game_server() { printf("Init QbGameServerModule\n"); QbGameServerModule = rb_define_module("QbGameServerModule"); rb_define_method(QbGameServerModule, "start_rpc", start_rpc, 1); }

1 3

[ruby-core:119136] [Ruby master Bug#20725] Inconsistent handling of doubly-unbounded ranges in Range#overlap?
by jeromepl (Jerome Parent-Levesque) 12 Sep '24

12 Sep '24

Issue #20725 has been reported by jeromepl (Jerome Parent-Levesque). ---------------------------------------- Bug #20725: Inconsistent handling of doubly-unbounded ranges in Range#overlap? https://bugs.ruby-lang.org/issues/20725 * Author: jeromepl (Jerome Parent-Levesque) * Status: Open * ruby -v: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The new `Range#overlap?` method does not seem to have a clear definition of how it behaves when passed ranges with nil begin **and** end `(nil..)`. In the following two cases, it seems to treat this type of range as an "infinite" range that covers the entire space and therefore overlaps with anything: ``` (nil..nil).overlap?(nil..) # => true (nil..nil).overlap?(3..) # => true ``` However it behaves differently in this case: ``` (nil..nil).overlap?(..3) # => false ``` Note that this also behaves differently than the ActiveSupport implementation of #overlap? in a Ruby version < 3.3 such as 3.2: ``` require "active_support" (nil..nil).overlap?(..3) # => true ``` I think `(nil..nil)` ranges should always be considered to be completely unbounded, i.e. infinite and covering the whole space. They should therefore always return `true` when overlapping with any non-empty range. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119135] CPU-bound load affects the timeout thread
by yuri.kanivetsky＠gmail.com 12 Sep '24

12 Sep '24

Hi, I'm not sure if it's a bug. Take a look at the following gist: https://gist.github.com/x-yuri/253f76df6287441f64b5eaee418813c0 It's supposedly a minimal reproducible example of what I ran into a couple of days ago. In my case it was basically a sort of a background jobs service. It received messages, processed them and sent the result to another service. Every message was processed and sent in a separate thread. And it made a number of http requests as part of processing messages (net/http which uses timeout()). It worked normally for a couple of hours (10-20 threads), but at some point the number of threads reached the thread pool's max_threads (320), http requests started to time out (later then they should, after 60-100 seconds instead of 3 seconds), and probably things started to get slow. Then I noticed the following line: patterns.any? { |pattern| quote.match(pattern['regex']) } ? 1 : 0 It was performed for every quote (10 times) in every thread, there were 300+ patterns and they (pattern['regex']) were strings. I moved converting strings to regexps to initialization (before starting to accept messages and creating threads) and it seems to work now with ~10 active threads. I'm trying to understand what exactly happened and how to avoid it or what awaits me in the future. Is there some critical load that breaks things? I guess in theory the number of threads should increase linearly with the load. But it looks like in this case there's some critical load that just makes things stop working. Or maybe for some time some negative effects get accumulated and then things break. My conjecture is that it has something to do with GIL. But what exactly happens? Or what can I do to further investigate the issue? I'm going to try to run ruby with RUBY_DEBUG_LOG and try to examine the debug output. I guess I need to figure out why the timeout thread doesn't always get enough time. Does something block the threads? How are they scheduled? When do they switch? Any pointers or links to the code are welcome. Regards, Yuri

2 2

[ruby-core:119129] [Ruby master Feature#20724] Update to Unicode 16.0
by duerst 11 Sep '24

11 Sep '24

Issue #20724 has been reported by duerst (Martin Dürst). ---------------------------------------- Feature #20724: Update to Unicode 16.0 https://bugs.ruby-lang.org/issues/20724 * Author: duerst (Martin Dürst) * Status: Open * Assignee: duerst (Martin Dürst) ---------------------------------------- Unicode 16.0 has been released (see https://www.unicode.org/versions/Unicode16.0.0/) We should update to 16.0 after updating to 15.1. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118388] [Ruby master Feature#20594] A new String method to append bytes while preserving encoding
by byroot (Jean Boussier) 11 Sep '24

11 Sep '24

Issue #20594 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20594: A new String method to append bytes while preserving encoding https://bugs.ruby-lang.org/issues/20594 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Context When working with binary protocols such as `protobuf` or `MessagePack`, you may often need to assemble multiple strings of different encoding: ```ruby Post = Struct.new(:title, :body) do def serialize(buf) buf << 255 << title.bytesize << title << 255 << body.bytesize << body end end Post.new("Hello", "World").serialize("somedata".b) # => "somedata\xFF\x05Hello\xFF\x05World" #<Encoding:ASCII-8BIT> ``` The problem in the above case, is that because `Encoding::ASCII_8BIT` is declared as ASCII compatible, if one of the appended string contains bytes outside the ASCII range, string is automatically promoted to another encoding, which then leads to encoding issues: ```ruby Post.new("H€llo", "Wôrld").serialize("somedata".b) # => incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError) ``` In many cases, you want to append to a String without changing the receiver's encoding. The issue isn't exclusive to binary protocols and formats, it also happen with ASCII protocols that accept arbitrary bytes inline, like Redis's RESP protocol or even HTTP/1.1. ### Previous discussion There was a similar feature request a while ago, but it was abandoned: https://bugs.ruby-lang.org/issues/14975 ### Existing solutions You can of course always cast the strings you append to avoid this problem: ```ruby Post = Struct.new(:title, :body) do def serialize(buf) buf << 255 << title.bytesize << title.b << 255 << body.bytesize << body.b end end ``` But this cause a lot of needless allocations. You'd think you could also use `bytesplice`, but it actually has the same issue: ```ruby Post = Struct.new(:title, :body) do def serialize(buf) buf << 255 << title.bytesize buf.bytesplice(buf.bytesize, title.bytesize, title) buf << 255 << body.bytesize buf.bytesplice(buf.bytesize, body.bytesize, title) end end Post.new("H€llo", "Wôrld").serialize("somedata".b) # => 'String#bytesplice': incompatible character encodings: BINARY (ASCII-8BIT) and UTF-8 (Encoding::CompatibilityError) ``` And even if it worked, it would be very unergonomic. ### Proposal: a `byteconcat` method A solution to this would be to add a new `byteconcat` method, that could be shimed as: ```ruby class String def byteconcat(*strings) strings.map! do |s| if s.is_a?(String) && s.encoding != encoding s.dup.force_encoding(encoding) else s end end concat(*strings) end end Post = Struct.new(:title, :body) do def serialize(buf) buf.byteconcat( 255, title.bytesize, title, 255, body.bytesize, body, ) end end Post.new("H€llo", "Wôrld").serialize("somedata".b) # => "somedata\xFF\aH\xE2\x82\xACllo\xFF\x06W\xC3\xB4rld" #<Encoding:ASCII-8BIT> ``` But of course a builtin implementation wouldn't need to dup the arguments. Like other `byte*` methods, it's the responsibility of the caller to ensure the resulting string has a valid encoding, or to deal with it if not. ### Method name and signature #### Name This proposal suggests `String#byteconcat`, to mirror `String#concat`, but other names are possible: - `byteappend` (like `Array#append`) - `bytepush` (like `Array#push`) #### Signature This proposal makes `byteconcat` accept either `String` or `Integer` (in char range) arguments like `concat`. I believe it makes sense for consistency and also because it's not uncommon for protocols to have some byte based segments, and Integers are more convenient there. The proposed method also accept variable arguments for consistency with `String#concat`, `Array#push`, `Array#append`. The proposed method returns self, like `concat` and others. ### YJIT consideration I consulted @maximecb about this proposal, and according to her, accepting variable arguments makes it harder for YJIT to optimize. I suspect consistency with other APIs trumps the performance consideration, but I think it's worth mentioning. -- https://bugs.ruby-lang.org/

12 31

[ruby-core:119112] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by knu (Akinori MUSHA) 10 Sep '24

10 Sep '24

Issue #18368 has been updated by knu (Akinori MUSHA). The compatibility with `(:a .. :z).step(3)` has also been restored. https://github.com/ruby/ruby/pull/11573 ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-109705 * Author: zverok (Victor Shepelev) * Status: Open ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0