September 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114181] [Ruby master Bug#19767] [Not really a bug, but more a not ideal notification] "historical binary regexp match" when using the "n" modifier in a ruby regex
by rubyFeedback (robert heiler) 29 Sep '23

29 Sep '23

Issue #19767 has been reported by rubyFeedback (robert heiler). ---------------------------------------- Bug #19767: [Not really a bug, but more a not ideal notification] "historical binary regexp match" when using the "n" modifier in a ruby regex https://bugs.ruby-lang.org/issues/19767 * Author: rubyFeedback (robert heiler) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- To get my knowledge about ruby regexes up-to-date I have been going through this tutorial/book here at: https://learnbyexample.github.io/Ruby_Regexp/unicode.html One example they provide is this, with some odd characters: 'fox:αλεπού'.scan(/\w+/n) This will match the found word ("fox"), but it also reports the following warning: warning: historical binary regexp match /.../n against UTF-8 string Now: this may be obvious to others, but to me personally I am not sure what a "historical" binary regexp match actually is. I assume it may have meant that this was more used in the past, and may be discouraged now? Or is something else meant? What does "historical" mean in this context? I may not be the only one who does not fully understand the term historical. Most of ruby's warnings are fairly easy to understand, but this one seems odd. Right now I do not know whether we can use the "n" modifier in a regex - not that I really have a good use case for it (I am using UTF-8 these days, so I don't seem to need ASCII-8BIT anyway), but perhaps the warning could be changed a little. I have no good alternative suggestion how it can be changed, largely because I do not know what it actually means, e. g. what is "historical" about it (but, even then, I'd actually recommend against using the word "historical" because I don't understand what it means; deprecated is easy to understand, historical does not tell me anything). Perhaps it could be expressed somewhat differently and we could get rid of the word "historical" there? Either way, it's a tiny issue so I was not even sure whether to report it. But, from the point of view of other warnings, I believe the term "historical" does not tell the user enough about what the issue is here. (irb):1: warning: historical binary regexp match /.../n against UTF-8 string => ["fox"] -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114911] [Ruby master Feature#19904] Deprecate or warn on multiple regular expression encodings
by tenderlovemaking (Aaron Patterson) 29 Sep '23

29 Sep '23

Issue #19904 has been reported by tenderlovemaking (Aaron Patterson). ---------------------------------------- Feature #19904: Deprecate or warn on multiple regular expression encodings https://bugs.ruby-lang.org/issues/19904 * Author: tenderlovemaking (Aaron Patterson) * Status: Open * Priority: Normal ---------------------------------------- It seems like you can pass multiple encoding flags to regular expression literals, but I think this should be a warning or possibly syntax error. For example: ```ruby x = /foo/nu p x.encoding ``` `n` says the RE should be ASCII-8BIT, and `u` says it should be UTF-8. The last flag wins, so in this case the regular expression gets UTF-8 encoding. However, I think it should be a warning or even a syntax error if you specify multiple encoding options on a regular expression. It seems like a mistake if programmers specify multiple. Thanks! -- https://bugs.ruby-lang.org/

3 5

[ruby-core:114882] [Ruby master Misc#19899] Ruby3.3.-preview2: Behaviour changes of Socket.recv when the sever closes the connection
by jbeschi (jacopo beschi) 28 Sep '23

28 Sep '23

Issue #19899 has been reported by jbeschi (jacopo beschi). ---------------------------------------- Misc #19899: Ruby3.3.-preview2: Behaviour changes of Socket.recv when the sever closes the connection https://bugs.ruby-lang.org/issues/19899 * Author: jbeschi (jacopo beschi) * Status: Open * Priority: Normal ---------------------------------------- I noticed that `Socket#recv` now returns `nil` instead of `""` when the server closes the connection, is it expected? If yes, I suggest mentioning it to the [release page](https://www.ruby-lang.org/en/news/2023/09/14/ruby-3-3-0-preview2-rele…. Example: ``` ruby # server.rb require "socket" port = 3030 server = TCPServer.new("", port) while(conn = server.accept) #serves forever conn.print "hello" conn.close end # client.rb require 'socket' port = 3030 socket = TCPSocket.new("localhost", port) 2.times do res = socket.recv(1012) puts "received:" p res end socket.close ``` ``` jacopo-37s-mb 3.2.0 ~ ruby client.rb received: "hello" received: "" jacopo-37s-mb 3.2.0 ~ rbenv local 3.3.0-preview2 jacopo-37s-mb 3.3.0-preview2 ~ ruby client.rb received: "hello" received: nil ``` Thanks! -- https://bugs.ruby-lang.org/

3 3

[ruby-core:114908] [Ruby master Feature#18515] Add Range#reverse_each implementation for performance
by kyanagi (Kouhei Yanagita) 28 Sep '23

28 Sep '23

Issue #18515 has been updated by kyanagi (Kouhei Yanagita). Let's set aside `#pred` for now to avoid going off track. I have created a new pull request with a simpler implementation and closed the old one. New PR: https://github.com/ruby/ruby/pull/8525 As benchmark results show, huge memory savings are achieved, especially for large `Range`. Execution speed is also increased. ``` prelude: | rf_1 = 0..1 rf_1k = 0..1000 rf_1m = 0..1000000 big = 2**1000 rb_1 = big..big+1 rb_1k = big..big+1000 rb_1m = big..big+1000000 benchmark: "Fixnum 1": rf_1.reverse_each { _1 } "Fixnum 1K": rf_1k.reverse_each { _1 } "Fixnum 1M": rf_1m.reverse_each { _1 } "Bignum 1": rb_1.reverse_each { _1 } "Bignum 1K": rb_1k.reverse_each { _1 } "Bignum 1M": rb_1m.reverse_each { _1 } ``` ### Max resident set size (bytes) | |master |PR |ratio| |:----------|--------:|-------:|----:| |Fixnum 1 | 13.910M| 14.877M|1.069| |Fixnum 1K | 14.778M| 14.762M|0.998| |Fixnum 1M | 28.770M| 14.025M|0.487| |Bignum 1 | 14.729M| 14.189M|0.963| |Bignum 1K | 14.074M| 13.582M|0.965| |Bignum 1M | 224.723M| 15.254M|0.067| ### Iteration per second (i/s) | |master |PR |ratio| |:----------|------:|-------:|----:| |Fixnum 1 | 6.653M| 14.511M|2.181| |Fixnum 1K |27.866k| 45.527k|1.633| |Fixnum 1M | 28.659| 45.667|1.593| |Bignum 1 | 3.534M| 4.635M|1.311| |Bignum 1K | 6.790k| 7.693k|1.132| |Bignum 1M | 5.720| 7.532|1.316| ---------------------------------------- Feature #18515: Add Range#reverse_each implementation for performance https://bugs.ruby-lang.org/issues/18515#change-104776 * Author: kyanagi (Kouhei Yanagita) * Status: Open * Priority: Normal ---------------------------------------- PR is https://github.com/ruby/ruby/pull/5489 Current `Range#reverse_each` uses `Enumerable#reverse_each` which is implemented with `#to_a`. So we are virtually not able to use `reverse_each` for a very large or beginless range, even if few elements are iterated on actually. ``` (1..2**100).reverse_each { |x| p x; break if x.odd? } (..5).reverse_each { |x| p x; break if x == 0 } (1..2**32).reverse_each.lazy.select { |x| Prime.prime?(x) }.take(3).to_a ``` This patch, implements `Range#reverse_each` for Integer elements, enables these examples. I think `#reverse_each` for an endless range should raise an exception. This is a different issue, so I'll create another ticket later. -> posted: https://bugs.ruby-lang.org/issues/18551 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114907] [Ruby master Feature#13933] Add Range#empty?
by Dan0042 (Daniel DeLorme) 27 Sep '23

27 Sep '23

Issue #13933 has been updated by Dan0042 (Daniel DeLorme). akr (Akira Tanaka) wrote in #note-14: > Float::NAN is not comparable with anything else, not only floats. Sorry, maybe I didn't express myself well, I meant that floats are usually comparable with other floats, with the exception of NaN. Whereas one `Object.new` is never comparable with another `Object.new` > I described an idea in https://bugs.ruby-lang.org/issues/19839#note-18 : > `minimum?` method. > > I feel `empty?` can sometimes be a wrong name. > `minimum?` represent the property directly. Yes, `minimum?` is cleaner and "the right way"; `empty?` is more like a hack that works with existing builtin classes with no changes needed. All 3 options you described are perfectly fine. > I think a bug report like my comment will coming someday. > Ruby developers (especially matz) need to respond to it. By "not worth worrying too much" I did not mean we should ignore or postpone it. I just meant we can pick *any* of the options without worrying too much about getting the perfect one. * `empty?` method * `minimum?` method * `MINIMUM` constant * hard code minimum values * document the limitation (even this, while not ideal, is "good enough" imho) * a mix of the above (ex: hardcode for builtins, `minimum?` method for user-defined classes) > Also, this problem can be expanded if we introduce `Range#exclude_begin?` (with neko operator). > https://bugs.ruby-lang.org/issues/12133 > In that case, we need `maximum?` as well. > But we also need a method to check the existence between two objects because `[1] ^..^ [1, ""]` is empty (no objects between `[1]` and `[1, ""]`). Huh, "neko" operator... that's cute 🐱 ---------------------------------------- Feature #13933: Add Range#empty? https://bugs.ruby-lang.org/issues/13933#change-104774 * Author: ted (Ted Johansson) * Status: Open * Priority: Normal ---------------------------------------- Range already responds to #size. It would be nice if it also responded to predicate #empty? :-) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114703] [Ruby master Bug#19875] Ruby 2.7 -> 3.1 Performance regression in String#count
by iz (Illia Zub) 27 Sep '23

27 Sep '23

Issue #19875 has been reported by iz (Illia Zub). ---------------------------------------- Bug #19875: Ruby 2.7 -> 3.1 Performance regression in String#count https://bugs.ruby-lang.org/issues/19875 * Author: iz (Illia Zub) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- `String#count` became slower since Ruby 3.1. Originally found by `@Freaky`: https://github.com/ruby/ruby/pull/4001#issuecomment-1714779781 Compared using the [`benchmark-driver` gem](https://github.com/benchmark-driver/benchmark-driver). ``` $ benchmark-driver tmp/string_count_benchmark_driver.yml --rbenv '3.1.1;3.1.4;2.7.2;3.2.2;3.0.6' Calculating ------------------------------------- 3.1.1 3.1.4 2.7.2 3.2.2 3.0.6 count 465.804 463.741 865.783 462.711 857.395 i/s - 10.000k times in 21.468251s 21.563768s 11.550239s 21.611783s 11.663235s Comparison: count 2.7.2: 865.8 i/s 3.0.6: 857.4 i/s - 1.01x slower 3.1.1: 465.8 i/s - 1.86x slower 3.1.4: 463.7 i/s - 1.87x slower 3.2.2: 462.7 i/s - 1.87x slower ``` Benchmark: ```yml $ cat ./tmp/string_count_benchmark_driver.yml loop_count: 10_000 prelude: | html = "\nruby\n" * 1024 * 1024 benchmark: count: html.count($/) ``` -- https://bugs.ruby-lang.org/

5 17

[ruby-core:114884] [Ruby master Bug#19900] Rename YARP to Prism
by kddnewton (Kevin Newton) 27 Sep '23

27 Sep '23

Issue #19900 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #19900: Rename YARP to Prism https://bugs.ruby-lang.org/issues/19900 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Following up on the discussion from https://bugs.ruby-lang.org/issues/19772#note-29, we would like to rename the YARP project to `Prism`. For the purposes of this ticket specifically, all functionality will remain the same. That is: * Prism will be a bundled gem with a native extension that will require you to `require "prism"` to access it from Ruby code. * Within CRuby, the C library will continue to be used to provide parsing/compiling functionality, under `RubyVM::InstructionSequence.compile_prism`. At the moment, this API is strictly for testing. Going forward, the C symbol prefix will be `pm_` (as opposed to the existing `yp_`). Assuming all of this is okay, I'll begin working on the rename next week. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114902] [Ruby master Feature#13933] Add Range#empty?
by akr (Akira Tanaka) 27 Sep '23

27 Sep '23

Issue #13933 has been updated by akr (Akira Tanaka). Dan0042 (Daniel DeLorme) wrote in #note-13: > akr (Akira Tanaka) wrote in #note-12: > > From experience with `Float::NAN`, I think it is not a good idea to expand such objects. > > It makes the behavior of `Array#<=>` inconsistent. > > I agree but I think this is a bit different from what I was talking about. Float::NAN is a float that is not comparable with other floats. This is not the same case as a value that is not comparable with anything else. > Float::NAN is not comparable with anything else, not only floats. ``` % ruby -e 'p Float::NAN <=> "foo"' nil % ruby -e 'p Float::NAN <=> []' nil ``` > > > Or maybe we can define a range "r" as empty if `r.begin.nil? and r.exclude_end? and r.end.respond_to?(:empty?) and r.end.empty?` > > > > You ignored the user-defined classes I mentioned. > > A user may define a class with a minimum value. > > The minimum value may not have `empty?` method. > > How do you define which is the minimum value? I suggested the above as a *possible* way to define this minimum value. So a custom class would need to have #empty? that returns true in order to define "this is the minimum". But it was just a suggestion. Or maybe each class could have a MINIMUM constant like `String::MINIMUM = ""`, etc. I described an idea in https://bugs.ruby-lang.org/issues/19839#note-18 : `minimum?` method. I feel `empty?` can sometimes be a wrong name. `minimum?` represent the property directly. > More importantly, I think that `(...minimum).empty?` is such an edge case of an edge case that it's not worth worrying too much. Are you suggesting that `Range#empty?` should not exist just because it's not possible to perfectly handle this extreme edge case for every possible class? Would it really be so bad if it returned `true` for user-defined classes? I think a bug report like my comment will coming someday. Ruby developers (especially matz) need to respond to it. So, we compare the usefulness of the method and the difficulties of responding to the bug report. Also, this problem can be expanded if we introduce `Range#exclude_begin?` (with neko operator). https://bugs.ruby-lang.org/issues/12133 In that case, we need `maximum?` as well. But we also need a method to check the existence between two objects because `[1] ^..^ [1, ""]` is empty (no objects between `[1]` and `[1, ""]`). ---------------------------------------- Feature #13933: Add Range#empty? https://bugs.ruby-lang.org/issues/13933#change-104767 * Author: ted (Ted Johansson) * Status: Open * Priority: Normal ---------------------------------------- Range already responds to #size. It would be nice if it also responded to predicate #empty? :-) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114900] [Ruby master Bug#17146] Queue operations are allowed after it is frozen
by jeremyevans0 (Jeremy Evans) 26 Sep '23

26 Sep '23

Issue #17146 has been updated by jeremyevans0 (Jeremy Evans). I submitted a pull request to have `{Queue,SizedQueue}#freeze` raise `TypeError`, as `ENV.freeze` does: https://github.com/ruby/ruby/pull/8515 ---------------------------------------- Bug #17146: Queue operations are allowed after it is frozen https://bugs.ruby-lang.org/issues/17146#change-104765 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal * ruby -v: ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- ``` [1] pry(main)> q = Queue.new => #<Thread::Queue:0x000056263683aee8> [2] pry(main)> q.freeze => #<Thread::Queue:0x000056263683aee8> [3] pry(main)> q << 1 => #<Thread::Queue:0x000056263683aee8> [4] pry(main)> q.pop => 1 [5] pry(main)> q.frozen? => true ``` Found by @ko1 in https://bugs.ruby-lang.org/issues/17100#note-28 I think it's a bug, since those are clear mutations. I guess old Thread::Queue implemented in Ruby did not have this bug. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114895] [Ruby master Feature#13933] Add Range#empty?
by Dan0042 (Daniel DeLorme) 24 Sep '23

24 Sep '23

Issue #13933 has been updated by Dan0042 (Daniel DeLorme). akr (Akira Tanaka) wrote in #note-12: > From experience with `Float::NAN`, I think it is not a good idea to expand such objects. > It makes the behavior of `Array#<=>` inconsistent. I agree but I think this is a bit different from what I was talking about. Float::NAN is a float that is not comparable with other floats. This is not the same case as a value that is not comparable with anything else. > There is no reason to break it. I agree; as I said, I'm not suggesting we change this now. > > Or maybe we can define a range "r" as empty if `r.begin.nil? and r.exclude_end? and r.end.respond_to?(:empty?) and r.end.empty?` > > You ignored the user-defined classes I mentioned. > A user may define a class with a minimum value. > The minimum value may not have `empty?` method. How do you define which is the minimum value? I suggested the above as a *posssible* way to define the minimum value. So a custom class would need to have #empty? that returns true in order to define "this is the minimum". But it was just a suggestion. Or maybe each class could have a MINIMUM constant like `String::MINIMUM = ""`, etc. More importantly, I think that `(...minimum).empty?` is such an edge case of an edge case that it's not worth worrying too much. Are you suggesting that `Range#empty?` should not exist just because it's not possible to perfectly handle this extreme edge case for every possible class? Would it really be so bad if it returned `true` for user-defined classes? ---------------------------------------- Feature #13933: Add Range#empty? https://bugs.ruby-lang.org/issues/13933#change-104753 * Author: ted (Ted Johansson) * Status: Open * Priority: Normal ---------------------------------------- Range already responds to #size. It would be nice if it also responded to predicate #empty? :-) -- https://bugs.ruby-lang.org/

1 0