Issue #19266 has been reported by gareth (Gareth Adams).
----------------------------------------
Bug #19266: URI::Generic should use URI::RFC3986_PARSER instead of URI::DEFAULT_PARSER
https://bugs.ruby-lang.org/issues/19266
* Author: gareth (Gareth Adams)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [arm64-darwin21]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
In June 2014, [`uri/common` was updated][1] to introduce a RFC3986-compliant parser (`URI::RFC3986_PARSER`) as an alternative to the previous RFC2396 parser, and common methods like `URI()` were updated to use that new parser by default. The only methods in `common` not updated were [`URI.extract` and `URI.regexp`][2] which are marked as obsolete. (The old parser was kept in the `DEFAULT_PARSER` constant despite it not being the default for those methods, presumably for backward compatibility.)
However, similar [methods called on `URI::Generic`][3] were never updated to use this new parser. This means that methods like `URI::Generic.build` fail when given input that succeeds normally, and this also affects subclasses like URI::HTTP:
```
$ pry -r uri -r uri/common -r uri/generic
[1] pry(main)> URI::Generic.build(host: "underscore_host.example")
URI::InvalidComponentError: bad component(expected host component): underscore_host.example
from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host'
[2] pry(main)> URI::HTTP.build(host: "underscore_host.example")
URI::InvalidComponentError: bad component(expected host component): underscore_host.example
from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host'
[3] pry(main)> URI("http://underscore_host.example")
=> #<URI::HTTP http://underscore_host.example>
```
`URI::Generic.new` allows a configurable `parser` positional argument to override the class' default parser, but other factory methods like `.build` don't allow this override.
Arguably this doesn't cause problems because at least in the case above, the URI can be built with the polymorphic constructor, but having the option to build URIs from explicit named parts is useful, and leaving the outdated functionality in the `Generic` class is ambiguous. It's possible that the whole Generic class and its subclasses aren't intended to be used directly how I'm intending here, but there's nothing I could see that suggested this is the case.
I'm not aware of the entire list of differences between RFC2396 and RFC3986. The relevant difference here is that in RFC2396 an individual segment of a host ([`domainlabel`s][4]) could only be `alphanum | alphanum *( alphanum | "-" ) alphanum`, whereas RFC3986 allows [hostnames][5] to include any of `ALPHA / DIGIT / "-" / "." / "_" / "~"`. It's possible that other differences might cause issues for developers, but since this has gone over 8 years without anyone else caring about this, this is definitely not especially urgent.
[1]: https://github.com/ruby/ruby/commit/bb83f32dc3e0424d25fa4e55d8ff32b061320e41
[2]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/…
[3]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/…
[4]: https://www.rfc-editor.org/rfc/rfc2396#section-3.2.2
[5]: https://www.rfc-editor.org/rfc/rfc3986#page-13
--
https://bugs.ruby-lang.org/
Issue #19236 has been reported by byroot (Jean Boussier).
----------------------------------------
Feature #19236: Allow to create hashes with a specific capacity from Ruby
https://bugs.ruby-lang.org/issues/19236
* Author: byroot (Jean Boussier)
* Status: Open
* Priority: Normal
* Target version: 3.3
----------------------------------------
Followup on [Feature #18683] which added a C-API for this purpose.
Various protocol parsers such as Redis `RESP3` or `msgpack`, have to create hashes, and they know the size in advance.
For efficiency, it would be preferable if they could directly allocate a Hash of the necessary size, so that large hashes wouldn't cause many re-alloccations and re-hash.
`String` and `Array` both already offer similar APIs:
```ruby
String.new(capacity: XXX)
Array.new(XX) / rb_ary_new_capa(long)
```
However there's no such public API for Hashes in Ruby land.
### Proposal
I think `Hash` should have a way to create a new hash with a `capacity` parameter.
The logical signature of `Hash.new(capacity: 1000)` was deemed too incompatible in [Feature #18683].
@Eregon proposed to add `Hash.create(capacity: 1000)`.
--
https://bugs.ruby-lang.org/
Issue #19161 has been reported by werebus (Matt Moretti).
----------------------------------------
Bug #19161: Cannot compile 3.0.5 or 3.1.3 on Red Hat 7
https://bugs.ruby-lang.org/issues/19161
* Author: werebus (Matt Moretti)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
When attempting to run `make` on either the 3.0.5 or 3.1.3 release, I get the following error (I included the whole output as it's pretty short):
```
BASERUBY = /opt/ruby/bin/ruby --disable=gems
CC = gcc -std=gnu11
LD = ld
LDSHARED = gcc -std=gnu11 -shared
CFLAGS = -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable
XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/13.0.0
CPPFLAGS =
DLDFLAGS = -Wl,--compress-debug-sections=zlib -fstack-protector-strong -pie
SOLIBS = -lz -lpthread -lrt -lrt -ldl -lcrypt -lm
LANG = en_US.UTF-8
LC_ALL =
LC_CTYPE =
MFLAGS =
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
compiling ./main.c
compiling dmydln.c
compiling miniinit.c
In file included from vm_core.h:83:0,
from iseq.h:14,
from mini_builtin.c:3,
from miniinit.c:51:
thread_pthread.h:108:43: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘struct’
RUBY_EXTERN RB_THREAD_LOCAL_SPECIFIER struct rb_execution_context_struct *ruby_current_ec;
^
In file included from iseq.h:14:0,
from mini_builtin.c:3,
from miniinit.c:51:
vm_core.h: In function ‘rb_current_execution_context’:
vm_core.h:1870:34: error: ‘ruby_current_ec’ undeclared (first use in this function)
rb_execution_context_t *ec = ruby_current_ec;
^
vm_core.h:1870:34: note: each undeclared identifier is reported only once for each function it appears in
miniinit.c: At top level:
cc1: warning: unrecognized command line option "-Wno-tautological-compare" [enabled by default]
cc1: warning: unrecognized command line option "-Wno-self-assign" [enabled by default]
cc1: warning: unrecognized command line option "-Wno-parentheses-equality" [enabled by default]
cc1: warning: unrecognized command line option "-Wno-constant-logical-operand" [enabled by default]
cc1: warning: unrecognized command line option "-Wno-cast-function-type" [enabled by default]
make: *** [miniinit.o] Error 1
```
Both Ruby 3.0.4 and 3.1.2 build without error and pass (all but one of) the `make check` tests.
I know RHEL 7 is getting to be pretty old; I suspect a factor here are the ancient build tools available to me. But... EOL for Ruby 2.7 comes before the one for RHEL 7, so I'm trying to prioritize.
--
https://bugs.ruby-lang.org/
Issue #19231 has been reported by andrykonchin (Andrew Konchin).
----------------------------------------
Bug #19231: Integer#step and Float::INFINITY - inconsistent behaviour when called with and without a block
https://bugs.ruby-lang.org/issues/19231
* Author: andrykonchin (Andrew Konchin)
* Status: Open
* Priority: Normal
* ruby -v: 3.1.2
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The initial issue was reported here https://github.com/oracle/truffleruby/issues/2797.
`0.step(Float::INFINITY, 10)` returns:
- `Integers` when called with a block
- `Floats` when called without a block
I would expect `Floats` to be returned in both cases.
Examples:
```ruby
0.step(100.0, 10).take(1).map(&:class)
# => [Float]
```
```ruby
0.step(Float::INFINITY, 10) { |offset| p offset.class; break }
# Integer
```
When `to` argument is a finite `Float` value then calling with a block returns `Floats` as well:
```ruby
0.step(100.0, 10) { |offset| p offset.class; break }
# Float
```
Wondering whether it's intentional behaviour.
I've found a related issue https://bugs.ruby-lang.org/issues/15518.
--
https://bugs.ruby-lang.org/
Issue #19365 has been reported by luke-gru (Luke Gruber).
----------------------------------------
Bug #19365: Ractors can access non-shareable values through enumerators
https://bugs.ruby-lang.org/issues/19365
* Author: luke-gru (Luke Gruber)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
I don't think enumerators should be able to be passed to `Ractor.new`
```ruby
obj = Object.new # unshareable value
p obj
Ractor.new([obj].each) {|f| p f.first }.take
```
--
https://bugs.ruby-lang.org/
Issue #19395 has been reported by luke-gru (Luke Gruber).
----------------------------------------
Bug #19395: Process forking within non-main Ractor creates child stuck in busy loop
https://bugs.ruby-lang.org/issues/19395
* Author: luke-gru (Luke Gruber)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
```ruby
def test_fork_in_ractor
r2 = Ractor.new do
pid = fork do
exit Ractor.count
end
pid
end
pid = r2.take
puts "Process #{Process.pid} waiting for #{pid}"
_pid, status = Process.waitpid2(pid) # stuck forever
if status.exitstatus != 1
raise "status is #{status.exitstatus}"
end
end
test_fork_in_ractor()
```
$ top # shows CPU usage is high for child process
--
https://bugs.ruby-lang.org/
Issue #19246 has been reported by thomthom (Thomas Thomassen).
----------------------------------------
Bug #19246: Rebuilding the loaded feature index much slower in Ruby 3.1
https://bugs.ruby-lang.org/issues/19246
* Author: thomthom (Thomas Thomassen)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Some background to this issue: (This is a case that is unconventional usage of Ruby, but I hope you bear with me.)
We ship the Ruby interpreter with our desktop applications for plugin support in our application (SketchUp).
One feature we have had since, at least 2006 (maybe earlier-hard to track history beyond that) is that we had a custom alternate `require` method: `Sketchup.require`. This allows the users of our API to load encrypted Ruby files.
This originally used `rb_provide` to add the path to the encrypted file into the list of loaded feature. However, somewhere between Ruby 2.2 and 2.5 there was some string optimisations made and the function `rb_provide` would not use a copy of the string passed to it. Instead it just held on to a pointer reference. In our case that string came from user-land, being passed in from `Sketchup.require` and would eventually be garbage collected and cause access violation crashes.
To work around that we changed our custom `Sketchup.require` to push to `$LOADED_FEATURES` directly. There was a small penalty to the index being rebuilt after that, but it was negligible.
Recently we tried to upgrade the Ruby interpreter in our application from 2.7 to 3.1 and found a major performance reduction when using our `Sketchup.require. As in, a plugin that would load in half a second would now spend 30 seconds.
From https://bugs.ruby-lang.org/issues/18452 it sounds like there is _some_ expected extra penalty due to changes in how the index is built. But should it really be this much?
Example minimal repro to simulate the issue:
```
# frozen_string_literal: true
require 'benchmark'
iterations = 200
foo_files = iterations.times.map { |i| "#{__dir__}/tmp/foo-#{i}.rb" }
foo_files.each { |f| File.write(f, "") }
bar_files = iterations.times.map { |i| "#{__dir__}/tmp/bar-#{i}.rb" }
bar_files.each { |f| File.write(f, "") }
biz_files = iterations.times.map { |i| "#{__dir__}/tmp/biz-#{i}.rb" }
biz_files.each { |f| File.write(f, "") }
Benchmark.bm do |x|
x.report('normal') {
foo_files.each { |file|
require file
}
}
x.report('loaded_features') {
foo_files.each { |file|
require file
$LOADED_FEATURES << "#{file}-fake.rb"
}
}
x.report('normal again') {
biz_files.each { |file|
require file
}
}
end
```
```
C:\Users\Thomas\SourceTree\ruby-perf>ruby27.bat
ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32]
C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb
user system total real
normal 0.000000 0.031000 0.031000 ( 0.078483)
loaded_features 0.015000 0.000000 0.015000 ( 0.038759)
normal again 0.016000 0.032000 0.048000 ( 0.076940)
```
```
C:\Users\Thomas\SourceTree\ruby-perf>ruby30.bat
ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x64-mingw32]
C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb
user system total real
normal 0.000000 0.031000 0.031000 ( 0.074733)
loaded_features 0.032000 0.000000 0.032000 ( 0.038898)
normal again 0.000000 0.047000 0.047000 ( 0.076343)
```
```
C:\Users\Thomas\SourceTree\ruby-perf>ruby31.bat
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x64-mingw-ucrt]
C:\Users\Thomas\SourceTree\ruby-perf>ruby test-require.rb
user system total real
normal 0.016000 0.031000 0.047000 ( 0.132633)
loaded_features 1.969000 11.500000 13.469000 ( 18.395761)
normal again 0.031000 0.125000 0.156000 ( 0.249130)
```
Right now we're exploring options to deal with this. Because the performance degradation is a blocker for us upgrading. We also have 16 years of plugins created by third party developer that makes it impossible for us to drop this feature.
Some options as-is, none of which are ideal:
1. We revert to using `rb_provide` but ensure the string passed in is not owned by Ruby, instead building a list of strings that we keep around for the duration of the application process. The problem is that some of our plugin developers have on occasion released plugins that will touch `$LOADED_FEATURES`, and if such a plugin is installed on a user machine it might cause the application to become unresponsive for minutes. The other non-ideal issue with using `rb_provide` is that we're also using that in ways it wasn't really intended (from that I understand). And it's not an official API?
2. We create a separate way for our `Sketchup.require` to keep track of it's loaded features, but then that would diverge even more from the behaviour of `require`. Replicating `require` functionality is not trivial and would be prone to subtle errors and possible diverge. It also doesn't address our issue that there is code out there in existing plugins that touches `$LOADED_FEATURES`. (And it's not something we can just ask people to clean up. From previous experience old versions stick around for a long time and is very hard to purge from circulation.)
I have two questions for the Ruby mantainers:
1. Would it be reasonable to see an API for adding/removing/checking `$LOADED_FEATURE` that would allow for a more ideal implementation of a custom `require` functionality?
2. Is the performance difference in rebuilding the loaded feature index really expected to be as high as what we're seeing? An increase of nearly 100 times? Is there something there that might be addressed to make the rebuild to be less expensive against? (This would really help to address our challenges with third party plugins occasionally touching the global.)
--
https://bugs.ruby-lang.org/
Issue #19387 has been reported by luke-gru (Luke Gruber).
----------------------------------------
Bug #19387: Issue with ObjectSpace.each_objects not returning IO objects after starting a ractor
https://bugs.ruby-lang.org/issues/19387
* Author: luke-gru (Luke Gruber)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
```ruby
r = Ractor.new do
receive # block, the problem is not the termination of the ractor but the starting
end
ObjectSpace.each_object(IO) { |io|
p io # we get no objects
}
```
--
https://bugs.ruby-lang.org/
Issue #19375 has been reported by luke-gru (Luke Gruber).
----------------------------------------
Bug #19375: File objects are currently shareable
https://bugs.ruby-lang.org/issues/19375
* Author: luke-gru (Luke Gruber)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
I don't know the internals of file.c but I don't think files are thread-safe.
--
https://bugs.ruby-lang.org/
Issue #19333 has been reported by ioquatix (Samuel Williams).
----------------------------------------
Feature #19333: Setting (Fiber Local|Thread Local|Fiber Storage) to nil should delete value in order to avoid memory leaks.
https://bugs.ruby-lang.org/issues/19333
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
As it stands, Fiber Locals, Thread Locals and Fiber Storage have no way of deleting key-value associations.
```ruby
100.times do |i|
name = :"variable-#{i}"
Thread.current[name] = 10
end
```
Because of this, dynamically generated associations can leak over time. This is worse for things like Threads that might be pooled (or maybe an argument against user-space pooling).
In any case, having a way to delete those associations would allow application code to at least delete the associations when they no longer make sense.
I propose that assigning `nil` to "locals" or "storage" should effectively delete them.
e.g.
```ruby
100.times do |i|
name = :"variable-#{i}"
Thread.current[name] = 10
Thread.current[name] = nil # delete association
end
```
A more invasive alternative would be to define new interfaces like `Thread::Local`, `Fiber::Local` and `Fiber::Storage::Local` (or something) which correctly clean up on GC.
--
https://bugs.ruby-lang.org/