August 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111526] [Ruby master Bug#19288] Ractor JSON parsing significantly slower than linear parsing

by maciej.mensfeld (Maciej Mensfeld)

Issue #19288 has been reported by maciej.mensfeld (Maciej Mensfeld). ---------------------------------------- Bug #19288: Ractor JSON parsing significantly slower than linear parsing https://bugs.ruby-lang.org/issues/19288 * Author: maciej.mensfeld (Maciej Mensfeld) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- a simple benchmark: ```ruby require 'json' require 'benchmark' CONCURRENT = 5 RACTORS = true ELEMENTS = 100_000 data = CONCURRENT.times.map do ELEMENTS.times.map do { rand => rand, rand => rand, rand => rand, rand => rand }.to_json end end ractors = CONCURRENT.times.map do Ractor.new do Ractor.receive.each { JSON.parse(_1) } end end result = Benchmark.measure do if RACTORS CONCURRENT.times do |i| ractors[i].send(data[i], move: false) end ractors.each(&:take) else # Linear without any threads data.each do |piece| piece.each { JSON.parse(_1) } end end end puts result ``` Gives following results on my 8 core machine: ```shell # without ractors: 2.731748 0.003993 2.735741 ( 2.736349) # with ractors 12.580452 5.089802 17.670254 ( 5.209755) ``` I would expect Ractors not to be two times slower on the CPU intense work. -- https://bugs.ruby-lang.org/

7 months, 3 weeks

6
16
0 0

[ruby-core:114181] [Ruby master Bug#19767] [Not really a bug, but more a not ideal notification] "historical binary regexp match" when using the "n" modifier in a ruby regex

by rubyFeedback (robert heiler)

Issue #19767 has been reported by rubyFeedback (robert heiler). ---------------------------------------- Bug #19767: [Not really a bug, but more a not ideal notification] "historical binary regexp match" when using the "n" modifier in a ruby regex https://bugs.ruby-lang.org/issues/19767 * Author: rubyFeedback (robert heiler) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- To get my knowledge about ruby regexes up-to-date I have been going through this tutorial/book here at: https://learnbyexample.github.io/Ruby_Regexp/unicode.html One example they provide is this, with some odd characters: 'fox:αλεπού'.scan(/\w+/n) This will match the found word ("fox"), but it also reports the following warning: warning: historical binary regexp match /.../n against UTF-8 string Now: this may be obvious to others, but to me personally I am not sure what a "historical" binary regexp match actually is. I assume it may have meant that this was more used in the past, and may be discouraged now? Or is something else meant? What does "historical" mean in this context? I may not be the only one who does not fully understand the term historical. Most of ruby's warnings are fairly easy to understand, but this one seems odd. Right now I do not know whether we can use the "n" modifier in a regex - not that I really have a good use case for it (I am using UTF-8 these days, so I don't seem to need ASCII-8BIT anyway), but perhaps the warning could be changed a little. I have no good alternative suggestion how it can be changed, largely because I do not know what it actually means, e. g. what is "historical" about it (but, even then, I'd actually recommend against using the word "historical" because I don't understand what it means; deprecated is easy to understand, historical does not tell me anything). Perhaps it could be expressed somewhat differently and we could get rid of the word "historical" there? Either way, it's a tiny issue so I was not even sure whether to report it. But, from the point of view of other warnings, I believe the term "historical" does not tell the user enough about what the issue is here. (irb):1: warning: historical binary regexp match /.../n against UTF-8 string => ["fox"] -- https://bugs.ruby-lang.org/

7 months, 3 weeks

3
2
0 0

[ruby-core:114038] [Ruby master Bug#19749] Confirm correct behaviour when attaching private method with `#define_method`

by itarato (Peter Arato)

Issue #19749 has been reported by itarato (Peter Arato). ---------------------------------------- Bug #19749: Confirm correct behaviour when attaching private method with `#define_method` https://bugs.ruby-lang.org/issues/19749 * Author: itarato (Peter Arato) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This issue is a special case of https://bugs.ruby-lang.org/issues/19745: Should dynamically added private methods via `.singleton_class.send(:define_method,...` be accessible publicly? See the following example? ```ruby private def bar; end foo = Object.new foo.singleton_class.send(:define_method, :bar, method(:bar)) foo.bar # No error. ``` The script above runs fine on latest Ruby 3.3. Is this correct to ignore the fact that the added method (method(:bar)) is private? This came up during a TruffleRuby investigation (https://github.com/oracle/truffleruby/issues/3134) where the result for the same script is: `private method 'bar' called for #<Object:0xc8> (NoMethodError)` -- https://bugs.ruby-lang.org/

7 months, 3 weeks

5
13
0 0

[ruby-core:111450] [Ruby master Bug#19268] Mingw64 Build Failure

by cfis (Charlie Savage)

Issue #19268 has been reported by cfis (Charlie Savage). ---------------------------------------- Bug #19268: Mingw64 Build Failure https://bugs.ruby-lang.org/issues/19268 * Author: cfis (Charlie Savage) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x64-mingw-ucrt] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- For both Ruby 3.1.3 and Ruby 3.2.0, building on msys2/ucrt64 fails: ``` c linking miniruby.exe /usr/bin/sh: -c: line 1: syntax error near unexpected token `(' /usr/bin/sh: -c: line 1: `/usr/local/ruby/bin/ruby --disable=gems -n -e BEGIN{version=ARGV.shift;mis=ARGV.dup} -e END{abort "UNICODE version mismatch: #{mis}" unless mis.empty?} -e (mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o 15.0.0 ./enc/unicode/15.0.0/casefold.h ./enc/unicode/15.0.0/name2ctype.h' make: *** [uncommon.mk:878: .rbconfig.time] Error 2 ``` The fix that works for me is changing: ``` c -e '(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o' \ ``` To ``` c -e "(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +\"#{Regexp.quote(version)}\"/o" \ ``` ``` -- https://bugs.ruby-lang.org/

7 months, 3 weeks

1
1
0 0

[ruby-core:114568] [Ruby master Bug#19853] Segfault when calling `system` from a non-blocking fiber

by rmosolgo (Robert Mosolgo)

Issue #19853 has been reported by rmosolgo (Robert Mosolgo). ---------------------------------------- Bug #19853: Segfault when calling `system` from a non-blocking fiber https://bugs.ruby-lang.org/issues/19853 * Author: rmosolgo (Robert Mosolgo) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! I'm trying to improve my Ruby library to support transfer-based fibers (https://github.com/rmosolgo/graphql-ruby/pull/4322). To test parallel I/O, I'm using `system("sleep #{duration}")`, but this causes a segfault sometimes. I'd love to find a way to keep developing this feature but I'm not sure where to go next. I have attached a script which replicates the bug and I have attached the full segfault error dump. I encountered this error by running: ``` ruby fiber_bug.rb 2>out.txt ``` Does that help debug? Please let me know if I can provide anything else that'd be helpful. ---Files-------------------------------- out.txt (121 KB) fiber_bug.rb (624 Bytes) -- https://bugs.ruby-lang.org/

7 months, 4 weeks

3
5
0 0

[ruby-core:114222] [Ruby master Misc#19772] API Naming for YARP compiler

by jemmai (Jemma Issroff)

Issue #19772 has been reported by jemmai (Jemma Issroff). ---------------------------------------- Misc #19772: API Naming for YARP compiler https://bugs.ruby-lang.org/issues/19772 * Author: jemmai (Jemma Issroff) * Status: Open * Priority: Normal ---------------------------------------- We are working on the YARP compiler, and have [the first PR ready](https://github.com/ruby/ruby/pull/8042) which introduces the YARP compile method. Our only outstanding question before merging it is about naming. How should we expose the public API for YARP's compile method? Potential suggestions: 1. YARP.compile 2. RubyVM::InstructionSequence.compile(yarp: true) 3. RubyVM::InstructionSequence.compile_yarp 4. Any of the above options, with a name other than yarp (please suggest an alternative) Regarding option 1, which would mirror `YARP.parse`, is the top level constant `YARP` acceptable? cc @matz @ko1 -- https://bugs.ruby-lang.org/

8 months

9
30
0 0

[ruby-core:114332] [Ruby master Bug#19794] Ruby 3.2.2 fails to build on macOS Sonoma betas

by jhaungs (Jim Haungs)

Issue #19794 has been reported by jhaungs (Jim Haungs). ---------------------------------------- Bug #19794: Ruby 3.2.2 fails to build on macOS Sonoma betas https://bugs.ruby-lang.org/issues/19794 * Author: jhaungs (Jim Haungs) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- With Big Sur, Apple deprecated putting dylibs in /usr/local/lib. In Sonoma (beta 4), this directory has disappeared completely. However, ruby's configure script depends on its existence. So, virtually every ruby installer (RVM, rbenv, asdf, ruby-build, and even building from source tarball) fails. When building ruby 3.2.2 from source, the configure step outputs the irritatingly useless "something wrong with LDFLAGS" error message and fails to build. The solution was to `cd /usr/local; sudo mkdir lib` to create the missing lib directory under /usr/local. It would be nice to remove this dependency from the configure script. -- https://bugs.ruby-lang.org/

8 months

5
6
0 0

[ruby-core:113381] [Ruby master Bug#19624] Backticks - IO object leakage

by pineman

Issue #19624 has been reported by pineman (João Pinheiro). ---------------------------------------- Bug #19624: Backticks - IO object leakage https://bugs.ruby-lang.org/issues/19624 * Author: pineman (João Pinheiro) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi, This code works on ruby 3.0.6: ```ruby `echo` ObjectSpace.each_object(IO) do |io| if ![STDIN, STDOUT, STDERR].include?(io) io.close end end ``` but raises `IOError` on 3.2.2: ``` minimal-repro-case.rb:8:in `close': uninitialized stream (IOError) ``` I found it started failing on ruby 3.1.0 and after, on macOS and Linux. This code is useful for closing unneeded IO objects in forked processes. It looks like backticks is 'leaking' IO objects, waiting for GC, and it didn't used to in 3.1.0. In ruby 3.1.0, inside `rb_f_backquote`, `rb_gc_force_recycle` was removed in favor of `RB_GC_GUARD`. I wonder if this has something to do with the problem. Is this code incorrect since ruby 3.1.0 or is it a bug in ruby? Thanks. ---Files-------------------------------- minimal-repro-case.rb (109 Bytes) -- https://bugs.ruby-lang.org/

8 months

4
9
0 0

[ruby-core:113926] [Ruby master Bug#19735] Add support for UUID version 7

by nevans (Nicholas Evans)

Issue #19735 has been reported by nevans (Nicholas Evans). ---------------------------------------- Bug #19735: Add support for UUID version 7 https://bugs.ruby-lang.org/issues/19735 * Author: nevans (Nicholas Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Although the specification for UUIDv7 is still in draft, the UUIDv7 algorithm has been stable as the RFC progresses to completion. Version 7 UUIDs can be very useful, because they are lexographically sortable, which can improve e.g: database index locality. See section 6.10 of the draft specification for further explanation: https://www.ietf.org/archive/id/draft-ietf-uuidrev-rfc4122bis-06.html ```ruby require 'random/formatter' Random.uuid_v7 # => "0188ca50-fcc0-7881-b5c5-6d55cd8fc373" Random.uuid_v7 # => "0188ca51-0069-7304-be2e-0c3cd908789b" Random.uuid_v7 # => "0188ca51-04aa-7b57-a6ec-c49573412a9d" Random.uuid_v7 # => "0188ca51-0853-7979-ae37-485460e9f4f1" # or prng = Random.new prng.uuid_v7 # => "0188ca51-5e72-7950-a11d-def7ff977c98" ``` PR here: https://github.com/ruby/ruby/pull/7953 -- https://bugs.ruby-lang.org/

8 months

3
4
0 0

[ruby-core:113819] [Ruby master Feature#19720] Warning for non-linear Regexps

by Eregon (Benoit Daloze)

Issue #19720 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #19720: Warning for non-linear Regexps https://bugs.ruby-lang.org/issues/19720 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal ---------------------------------------- I believe the best way to solve ReDoS is to ensure all Regexps used in the process are linear. Using `Regexp.timeout = 5.0` or so does not really prevent ReDoS, given enough requests causing that timeout the servers will still be very unresponsive. To this purpose, we should make it easy to identify non-linear Regexps and fix them. I suggest we either use 1. a performance warning (enabled with `Warning[:performance] = true`, #19538) or 2. a new regexp warning category (enabled with `Warning[:regexp] = true`). I think we should warn only once per non-linear Regexp, to avoid too many such warnings. We could warn as soon as the Regexp is created, or on first match. On first match might makes more sense for Ruby implementations which compile the Regexp lazily (since that is costly during startup), and also avoids warning for Regexps which are never used (which can be good or bad). OTOH, if the warning is enabled, we could always compile the Regexp eagerly (or at least checks whether it's linear), and that would then provide a better way to guarantee that all Regexps created so far are linear. Because warnings are easily customizable, it is also possible to e.g. `raise/abort` on such a warning, if one wants to ensure their application does not use a non-linear Regexp and so cannot be vulnerable to ReDoS: ```ruby Warning.extend Module.new { def warn(message, category: nil, **) raise message if category == :regexp super end } ``` A regexp warning category seems better for that as it makes it easy to filter by category, if a performance warning one would need to match the message which is less clean. As a note, TruffleRuby already has a similar warning, as a command-line option: ``` $ truffleruby --experimental-options --warn-truffle-regex-compile-fallback -e 'Gem' truffleruby-dev/lib/mri/rubygems/version.rb:176: warning: Regexp /\A\s*([0-9]+(?>\.[0-9a-zA-Z]+)*(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?)?\s*\z/ at_start=false encoding=US-ASCII requires backtracking and will not match in linear time truffleruby-dev/lib/mri/rubygems/requirement.rb:105: warning: Regexp /\A\s*(=|!=|>|<|>=|<=|~>)?\s*([0-9]+(?>\.[0-9a-zA-Z]+)*(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?)\s*\z/ at_start=false encoding=US-ASCII requires backtracking and will not match in linear time ``` So the warning message could be like `FILE:LINE: warning: Regexp /REGEXP/ requires backtracking and might not match in linear time and might cause ReDoS` or more concise: `FILE:LINE: warning: Regexp /REGEXP/ requires backtracking and might cause ReDoS` -- https://bugs.ruby-lang.org/

8 months, 1 week

5
10
0 0

2024

2023

2022

ruby-core August 2023