October 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114089] [Ruby master Misc#19758] Statically link ext/json

by MyCo (Maik Menz)

Issue #19758 has been reported by MyCo (Maik Menz). ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/

3 months, 1 week

4
6
0 0

[ruby-core:114997] [Ruby master Bug#19918] Should `a[&b]=c` be syntax valid?

by tompng (tomoya ishida)

Issue #19918 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19918: Should `a[&b]=c` be syntax valid? https://bugs.ruby-lang.org/issues/19918 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-11T04:46:58Z master 40ab7b8c24) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- These codes are syntax valid now. Prism parses it as syntax error. ~~~ruby a[&b]=c a[&b]+=c a[&b]&&=c a[&b]||=c ~~~ Is this syntax intentional or should be error? Issue of Prism https://github.com/ruby/prism/issues/1636 It's added in test_parse.rb https://github.com/ruby/ruby/blob/40ab7b8c244de20007cb45846f41de3a01f7ea0c/… -- https://bugs.ruby-lang.org/

3 months, 3 weeks

5
6
0 0

[ruby-core:111342] [Ruby master Bug#19246] Rebuilding the loaded feature index much slower in Ruby 3.1

by thomthom (Thomas Thomassen)

Issue #19246 has been reported by thomthom (Thomas Thomassen). ---------------------------------------- Bug #19246: Rebuilding the loaded feature index much slower in Ruby 3.1 https://bugs.ruby-lang.org/issues/19246 * Author: thomthom (Thomas Thomassen) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Some background to this issue: (This is a case that is unconventional usage of Ruby, but I hope you bear with me.) We ship the Ruby interpreter with our desktop applications for plugin support in our application (SketchUp). One feature we have had since, at least 2006 (maybe earlier-hard to track history beyond that) is that we had a custom alternate `require` method: `Sketchup.require`. This allows the users of our API to load encrypted Ruby files. This originally used `rb_provide` to add the path to the encrypted file into the list of loaded feature. However, somewhere between Ruby 2.2 and 2.5 there was some string optimisations made and the function `rb_provide` would not use a copy of the string passed to it. Instead it just held on to a pointer reference. In our case that string came from user-land, being passed in from `Sketchup.require` and would eventually be garbage collected and cause access violation crashes. To work around that we changed our custom `Sketchup.require` to push to `$LOADED_FEATURES` directly. There was a small penalty to the index being rebuilt after that, but it was negligible. Recently we tried to upgrade the Ruby interpreter in our application from 2.7 to 3.1 and found a major performance reduction when using our `Sketchup.require. As in, a plugin that would load in half a second would now spend 30 seconds. From https://bugs.ruby-lang.org/issues/18452 it sounds like there is _some_ expected extra penalty due to changes in how the index is built. But should it really be this much? Example minimal repro to simulate the issue: ``` # frozen_string_literal: true require 'benchmark' iterations = 200 foo_files = iterations.times.map { |i| "#{__dir__}/tmp/foo-#{i}.rb" } foo_files.each { |f| File.write(f, "") } bar_files = iterations.times.map { |i| "#{__dir__}/tmp/bar-#{i}.rb" } bar_files.each { |f| File.write(f, "") } biz_files = iterations.times.map { |i| "#{__dir__}/tmp/biz-#{i}.rb" } biz_files.each { |f| File.write(f, "") } Benchmark.bm do |x| x.report('normal') { foo_files.each { |file| require file } } x.report('loaded_features') { foo_files.each { |file| require file $LOADED_FEATURES << "#{file}-fake.rb" } } x.report('normal again') { biz_files.each { |file| require file } } end ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby27.bat ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.000000 0.031000 0.031000 ( 0.078483) loaded_features 0.015000 0.000000 0.015000 ( 0.038759) normal again 0.016000 0.032000 0.048000 ( 0.076940) ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby30.bat ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.000000 0.031000 0.031000 ( 0.074733) loaded_features 0.032000 0.000000 0.032000 ( 0.038898) normal again 0.000000 0.047000 0.047000 ( 0.076343) ``` ``` C:\Users\Thomas\SourceTree\ruby-perf>ruby31.bat ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x64-mingw-ucrt] C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb user system total real normal 0.016000 0.031000 0.047000 ( 0.132633) loaded_features 1.969000 11.500000 13.469000 ( 18.395761) normal again 0.031000 0.125000 0.156000 ( 0.249130) ``` Right now we're exploring options to deal with this. Because the performance degradation is a blocker for us upgrading. We also have 16 years of plugins created by third party developer that makes it impossible for us to drop this feature. Some options as-is, none of which are ideal: 1. We revert to using `rb_provide` but ensure the string passed in is not owned by Ruby, instead building a list of strings that we keep around for the duration of the application process. The problem is that some of our plugin developers have on occasion released plugins that will touch `$LOADED_FEATURES`, and if such a plugin is installed on a user machine it might cause the application to become unresponsive for minutes. The other non-ideal issue with using `rb_provide` is that we're also using that in ways it wasn't really intended (from that I understand). And it's not an official API? 2. We create a separate way for our `Sketchup.require` to keep track of it's loaded features, but then that would diverge even more from the behaviour of `require`. Replicating `require` functionality is not trivial and would be prone to subtle errors and possible diverge. It also doesn't address our issue that there is code out there in existing plugins that touches `$LOADED_FEATURES`. (And it's not something we can just ask people to clean up. From previous experience old versions stick around for a long time and is very hard to purge from circulation.) I have two questions for the Ruby mantainers: 1. Would it be reasonable to see an API for adding/removing/checking `$LOADED_FEATURE` that would allow for a more ideal implementation of a custom `require` functionality? 2. Is the performance difference in rebuilding the loaded feature index really expected to be as high as what we're seeing? An increase of nearly 100 times? Is there something there that might be addressed to make the rebuild to be less expensive against? (This would really help to address our challenges with third party plugins occasionally touching the global.) -- https://bugs.ruby-lang.org/

3 months, 3 weeks

9
21
0 0

[ruby-core:114590] [Ruby master Bug#19857] Eval coverage is reset after each `eval`.

by ioquatix (Samuel Williams)

Issue #19857 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19857: Eval coverage is reset after each `eval`. https://bugs.ruby-lang.org/issues/19857 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: ioquatix (Samuel Williams) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- It seems like `eval` based coverage is reset every time eval is invoked. ```ruby #!/usr/bin/env ruby require 'coverage' def measure(flag) c = Class.new c.class_eval(<<~RUBY, "foo.rb", 1) def foo(flag) if flag puts "foo" else puts "bar" end end RUBY return c.new.foo(flag) end Coverage.start(lines: true, eval: true) # Depending on the order of these two operations, different computation is calculated, because the evaluation of the code is considered different, even if the content/path is the same. measure(false) measure(true) p Coverage.result ``` Further investigation is required. -- https://bugs.ruby-lang.org/

3 months, 3 weeks

3
19
0 0

[ruby-core:115181] [Ruby master Bug#19975] ISeq#to_binary loses hidden local variable indices

by kddnewton (Kevin Newton)

Issue #19975 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #19975: ISeq#to_binary loses hidden local variable indices https://bugs.ruby-lang.org/issues/19975 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When you call `RubyVM::InstructionSequence#to_a`, you get hidden local variables as the index in the stack from the top: ``` c if (rb_id2str(lid)) { rb_ary_push(locals, ID2SYM(lid)); } else { /* hidden variable from id_internal() */ rb_ary_push(locals, ULONG2NUM(iseq_body->local_table_size-i+1)); } ``` ``` rb RubyVM::InstructionSequence.compile("for foo in bar; end").to_a[13][4][2][10] # => [2] ``` When you call `RubyVM::InstructionSequence#to_binary`, it dumps hidden local variables as 0: ``` c if (id == 0 || rb_id2name(id) == NULL) { return 0; } return ibf_dump_object(dump, rb_id2sym(id)); ``` When it reads that back in and then you call `to_a`, you get `:#arg_rest`: ``` ruby RubyVM::InstructionSequence.load_from_binary(RubyVM::InstructionSequence.compile("for foo in bar; end").to_binary).to_a[13][4][2][10] # => [:"#arg_rest"] ``` This means you end up not being able to consistently look at the locals. Instead, when reading back in from binary it could replace it with the index so that it matches up with the value before it is dumped to binary. Could we do that so that `#to_a` is consistent no matter where the iseq came from? -- https://bugs.ruby-lang.org/

3 months, 3 weeks

2
1
0 0

[ruby-core:112091] [Ruby master Bug#19387] Issue with ObjectSpace.each_objects not returning IO objects after starting a ractor

by luke-gru (Luke Gruber)

Issue #19387 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19387: Issue with ObjectSpace.each_objects not returning IO objects after starting a ractor https://bugs.ruby-lang.org/issues/19387 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby r = Ractor.new do receive # block, the problem is not the termination of the ractor but the starting end ObjectSpace.each_object(IO) { |io| p io # we get no objects } ``` -- https://bugs.ruby-lang.org/

3 months, 4 weeks

3
5
0 0

[ruby-core:113788] [Ruby master Bug#19717] `ConditionVariable#signal` is not fair when the wakeup is consistently spurious.

by ioquatix (Samuel Williams)

Issue #19717 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19717: `ConditionVariable#signal` is not fair when the wakeup is consistently spurious. https://bugs.ruby-lang.org/issues/19717 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- For background, see this issue <https://github.com/socketry/async/issues/99>. It looks like `ConditionVariable#signal` is not fair, if the calling thread immediately reacquires the resource. I've given a detailed reproduction here as it's non-trivial: <https://github.com/ioquatix/ruby-condition-variable-timeout>. Because the spurious wakeup occurs, the thread is pushed to the back of the waitq, which means any other waiting thread will acquire the resource, and that thread will perpetually be at the back of the queue. I believe the solution is to change `ConditionVarialbe#signal` should only remove the thread from the waitq if it's possible to acquire the lock. Otherwise it should be left in place, so that the order is retained, this should result in fair scheduling. -- https://bugs.ruby-lang.org/

4 months

3
12
0 0

[ruby-core:115104] [Ruby master Feature#19965] Make the name resolution interruptible

by mame (Yusuke Endoh)

Issue #19965 has been reported by mame (Yusuke Endoh). ---------------------------------------- Feature #19965: Make the name resolution interruptible https://bugs.ruby-lang.org/issues/19965 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- ## Problem Currently, Ruby name resolution is not interruptible. ``` $ cat /etc/resolv.conf nameserver 198.51.100.1 $ ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C^C^C^C ``` If you set a non-responsive IP as the nameserver, you cannot stop `Addrinfo.getaddrinfo` by pressing Ctrl+C. Note that `Timeout.timeout` does not work either. This is because there is no way to cancel `getaddrinfo(3)`. ## Proposal I wrote a patch to make `getaddrinfo(3)` work in a separate pthread. https://github.com/ruby/ruby/pull/8695 Whenever it needs name resolution, it creates a worker pthread, and executes `getaddrinfo(3)` in it. The caller thread waits for the worker to complete. When an interrupt occurs, the caller thread leaves stop waiting and leaves the worker pthread. The detached worker pthread will exit after `getaddrinfo(3)` completes (or name resolution times out). ## Evaluation By applying this patch, name resolution is now interruptible. ``` $ ./local/bin/ruby -rsocket -e 'pp Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C-e:1:in `getaddrinfo': Interrupt from -e:1:in `<main>' ``` As a drawback, name resolution performance will be degraded. ``` 10000.times { Addrinfo.getaddrinfo("www.ruby-lang.org", 80) } # Before patch: 2.3 sec. # After ptach: 3.0 sec. ``` However, I think that name resolution is typically short enough for the application's runtime. For example, the difference is small for the performance of `URI.open`. ``` 100.times { URI.open("https://www.ruby-lang.org").read } # Before patch: 3.36 sec. # After ptach: 3.40 sec. ``` ## Alternative approaches I proposed using c-ares to resolve this issue (#19430). However, there was an opinion that it would be a problem that c-ares does not respect the platform-dependent own name resolution. ## Room for improvement * Currently, this patch works only when pthread is available. * It might be possible to force to stop the worker threads by using `pthread_cancel`. However, `pthread_cancel` with `getaddrinfo(3)` seems still premature; there seems to be a bug in glibc until recently: https://bugzilla.redhat.com/show_bug.cgi?id=1405071 https://sourceware.org/bugzilla/show_bug.cgi?id=20975 * It would be more efficient to pool worker pthreads instead of creating them each time. -- https://bugs.ruby-lang.org/

4 months

3
6
0 0

[ruby-core:114936] [Ruby master Feature#19908] Update to Unicode 15.1

by nobu (Nobuyoshi Nakada)

Issue #19908 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #19908: Update to Unicode 15.1 https://bugs.ruby-lang.org/issues/19908 * Author: nobu (Nobuyoshi Nakada) * Status: Assigned * Priority: Normal * Assignee: duerst (Martin Dürst) * Target version: 3.3 ---------------------------------------- The Unicode 15.1 is released. The current enc-unicode.rb seems to fail because of `Indic_Conjunct_break` properties with values. I'm not sure how these properties should be handled well. `/\p{InCB_Liner}/` or `/\p{InCB=Liner}/` as the comments in that file? https://github.com/nobu/ruby/tree/unicode-15.1 is the former. -- https://bugs.ruby-lang.org/

4 months

4
5
0 0

[ruby-core:112208] [Ruby master Bug#19409] Object's shape is reset after a ractor move

by luke-gru (Luke Gruber)

Issue #19409 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19409: Object's shape is reset after a ractor move https://bugs.ruby-lang.org/issues/19409 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I believe an object should have the same shape after being moved from 1 ractor to another. ```ruby class Obj attr_accessor :a, :b, :c, :d def initialize @a = 1 @b = 2 @c = 3 end end r = Ractor.new do obj = receive #p RubyVM::Shape.of(obj) obj.d = 4 p obj.a, obj.b, obj.c, obj.d # gets wrong values due to object shape id being reset on object end obj = Obj.new #p RubyVM::Shape.of(obj) r.send(obj, move: true) r.take ``` -- https://bugs.ruby-lang.org/

4 months

2
2
0 0

2024

2023

2022

ruby-core October 2023