May 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111566] [Ruby master Bug#19294] Enumerator.product works incorrectly with consuming enumerators
by zverok (Victor Shepelev) 14 May '23

14 May '23

Issue #19294 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #19294: Enumerator.product works incorrectly with consuming enumerators https://bugs.ruby-lang.org/issues/19294 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby s = StringIO.new('abc') Enumerator.product([1, 2, 3], s.each_char).to_a # Expected: => [[1, "a"], [1, "b"], [1, "c"], [2, "a"], [2, "b"], [2, "c"], [3, "a"], [3, "b"], [3, "c"]] # Actual: => [[1, "a"], [1, "b"], [1, "c"]] ``` The implementation consumes the non-first enumerator to produce the first combination. Somewhat related to the dilemma of consuming and non-consuming enumerators (#19061). PS: I noticed I don't understand why it is `Enumerator.product` and not `Enumerable#product`, but probably it is too late to raise the questions :( -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113420] [Ruby master Bug#19632] Disable external iterator for frozen enumerator
by make_now_just (Hiroya Fujinami) 14 May '23

14 May '23

Issue #19632 has been reported by make_now_just (Hiroya Fujinami). ---------------------------------------- Bug #19632: Disable external iterator for frozen enumerator https://bugs.ruby-lang.org/issues/19632 * Author: make_now_just (Hiroya Fujinami) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Currently, methods to manipulate an external iterator like `#next` and `#feed` can be called even if a receiver of an enumerator is frozen. However, these methods change the state of an external iterator in an enumerator. Therefore, it seems a BUG to me, and these methods should raise FrozenError if the receiver is frozen. ```ruby e = 3.times.freeze # Current e.next # => 1 e.next # => 2 # Expected e.next # raise FrozenError ``` Two years ago, this issue was mentioned in [a comment](https://bugs.ruby-lang.org/issues/17195#note-5). I suggest fixing the following methods to raise FrozenError against a frozen enumerator. - `Enumerator#next` - `Enumerator#next_values` - `Enumerator#peek` - `Enumerator#peek_values` - `Enumerator#feed` - `Enumerator#rewind` Also, even if an enumerator is frozen, it does not affect other methods for internal iterators. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:113477] [Ruby master Bug#19639] Escaped newline literals in Regexp are ignored in extended / free-spacing mode
by janosch-x 14 May '23

14 May '23

Issue #19639 has been reported by janosch-x (Janosch Müller). ---------------------------------------- Bug #19639: Escaped newline literals in Regexp are ignored in extended / free-spacing mode https://bugs.ruby-lang.org/issues/19639 * Author: janosch-x (Janosch Müller) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [arm64-darwin21] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When we want to match whitespace with a Regexp that uses the x-flag, we can do that with escaped literal whitespace: ```ruby ' '[/\ /x] # => " " ``` This works for all whitespace - except newlines: ```ruby "\n"[/\ /x] # => "" /\ /x.source # => "" ``` I guess another parsing step eliminates such escaped newlines before the regexp parsing happens? It is probably a rare issue because most people prefer "\n" in Regexps over literal newlines, but at least the relevant statement in the [documentation](https://docs.ruby-lang.org/en/master/regexp_rdoc.html#label-… is a bit too broad as it is: > Use escaped whitespace such as \ , i.e. a space preceded by a backslash. Maybe we want to add a caveat like `(Does not work for newlines.)`? -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113475] [Ruby master Bug#18933] Dir.tmpdir implemented in non-Ractor-safe manner
by headius (Charles Nutter) 14 May '23

14 May '23

Issue #18933 has been updated by headius (Charles Nutter). Why not just put this in a private constant? I believe that would be Ractor-safe and it would make more sense than a lazy class variable or class instance variable (that's not actually lazy). ---------------------------------------- Bug #18933: Dir.tmpdir implemented in non-Ractor-safe manner https://bugs.ruby-lang.org/issues/18933#change-103057 * Author: kreynolds (Kelley Reynolds) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- # Background Inside a Ractor, it is expected that generating a temporary directory using `Dir.tmpdir` will work. The current implementation uses a number of things which are not considered Rector-safe and will require refactoring. # How to reproduce ```ruby Ractor.new { Dir.tmpdir } ``` # Expectation and result The result is expected to be something along the lines of `"/var/folders/xm/y4c00x0s26sgf_zlnqjh_7800000gn/T"` but instead a `Ractor::IsolationError` exception is raised: ```ruby 3.1.2/lib/ruby/3.1.0/tmpdir.rb:23:in `tmpdir': can not access class variables from non-main Ractors (Ractor::IsolationError) ``` # Suggested solutions * Make cached class variable a shareable constant * Modify how random values are generated to be Rector-safe ---Files-------------------------------- tmpdir.diff (1.77 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113474] [Ruby master Feature#18258] Ractor.shareable? can be slow and mutates internal object flags.
by jeremyevans0 (Jeremy Evans) 14 May '23

14 May '23

Issue #18258 has been updated by jeremyevans0 (Jeremy Evans). Tracker changed from Bug to Feature Status changed from Open to Closed ruby -v deleted (3.0.2) Backport deleted (2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN) @headius and I discussed this and we agree it is not a bug, though potentially the situation could be improved. You could avoid the object flag mutation using a temporary hash per call to shareable, but that would likely make it even slower. It doesn't seem possible to cache that an object is not shareable without creating significant cache invalidation issues. Having String#freeze set the shareable flag if the string has no instance variables seems like a good idea, but that is more of a feature request than a bug fix. Changing `Ractor.shareable?` to only do a shallow check and not a full check seems like a large semantic change, not a bug fix, and would make Ractor.shareable? operate differently than Ractor.make_shareable. Maybe a new Ractor.flagged_shareable? method could be added to do a shallow check for the shareable flag, but I'm not sure how useful such a method would be. ---------------------------------------- Feature #18258: Ractor.shareable? can be slow and mutates internal object flags. https://bugs.ruby-lang.org/issues/18258#change-103055 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) ---------------------------------------- On my computer, even with a relatively small object graph,`Ractor.shareable?` can be quite slow (around 1-2ms). The following example creates an object graph with ~40k objects as an example, and on my computer takes around 20ms to execute `Ractor.shareable?`. Because the object cannot be marked as `RB_FL_SHAREABLE` because it contains mutable state, every time we check `Ractor.shareable?` it will perform the same object traversal which is the slow path. ``` ruby require 'benchmark' class Borked def freeze end end class Nested def initialize(count, top = true) if count > 0 @nested = count.times.map{Nested.new(count - 1, false).freeze}.freeze end if top @borked = Borked.new end end attr :nested attr :borked end def test(n) puts "Creating nested object of size N=#{n}" nested = Nested.new(n).freeze shareable = false result = Benchmark.measure do shareable = Ractor.shareable?(nested) end pp result: result, shareable: shareable end test(8) ``` I propose we change `Ractor.shareable?` to only check `RB_FL_SHAREABLE` which gives (1) predictable and fast performance in every case and (2) avoids mutating internal object flags when performing what looks like a read-only operation. I respect that one way of looking at `Ractor.shareable?` is as a cache for object state. But this kind of cache can lead to unpredictable performance. As a result, something like `String#freeze` would not create objects that can be shared with Ractor. However, I believe we can mitigate this by tweaking `String#freeze` to also set `RB_FL_SHAREABLE` if possible. I believe we should apply this to more objects. It will lead to more predictable performance for Ruby. Since there are few real-world examples of Ractor, it's hard to find real world example of the problem. However, I believe such an issue will prevent Ractor usage as even relatively small object graphs (~1000 objects) can cause 1-2ms of latency, and this particular operation does not release the GVL either which means it stalls the entire VM. This issue came from discussion regarding https://bugs.ruby-lang.org/issues/18035 where we are considering using `RB_FL_SHAREABLE` as a flag for immutability. By fixing this issue, we make it easier to implement model for immutability because we don't need to introduce new flags and can instead reuse existing flags. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113473] [Ruby master Bug#17680] tab completion no longer works on irb3.0
by jeremyevans0 (Jeremy Evans) 14 May '23

14 May '23

Issue #17680 has been updated by jeremyevans0 (Jeremy Evans). Status changed from Open to Closed This change from 2.5 is expected due to differences between reline and readline. As readline has been removed from stdlib, reline's behavior is now the only available behavior. Behavior is the same on JRuby. It's possible a pull request to reline to have tab completion behavior more compatible to readline may be considered, but that would be up to the reline maintainer. ---------------------------------------- Bug #17680: tab completion no longer works on irb3.0 https://bugs.ruby-lang.org/issues/17680#change-103054 * Author: simondedeo (Simon DeDeo) * Status: Closed * Priority: Normal * ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin16] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- in irb2.5: rb(main):001:0> source 'li [then I hit tab...] linking.rb list_load.rb list_load_OLD.rb irb(main):001:0> source 'lis [then I hit tab again...] list_load.rb list_load_OLD.rb irb(main):001:0> source 'list_load.rb' [then I hit tab again...] in irb3.0, hitting tab after typing "source 'li" gives. irb(main):001:0' source 'li' Here are my version numbers: ayerie:simon$ irb3.0 -v irb 1.3.0 (2020-12-25) ayerie:CHLOE simon$ ruby3.0 -v ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin16] ayerie:simon$ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113472] [Ruby master Bug#16959] Weakmap has specs and third-party usage despite being a private API
by headius (Charles Nutter) 14 May '23

14 May '23

Issue #16959 has been updated by headius (Charles Nutter). Status changed from Open to Rejected WeakMap has now been in the wild with documentation and specs for several years, so I think it has become a de-facto public API. I'm fine closing this, though I still would like to see the other features I have described added to Weakref. ---------------------------------------- Bug #16959: Weakmap has specs and third-party usage despite being a private API https://bugs.ruby-lang.org/issues/16959#change-103052 * Author: headius (Charles Nutter) * Status: Rejected * Priority: Normal * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- Weakmap is still described as an internal API, and the documentation points users at WeakRef as the official public API: https://github.com/ruby/ruby/blob/1fb16dbb6e28b9f32f92554d29e646e088b21a98/… However there are now specs for its current set of features, even though those features have never been discussed or approved as a public API: https://github.com/ruby/spec/tree/dd8437628a6f2de5b74b338d4960682bb1590a60/… And we are starting to see it being used by the community: * https://github.com/jruby/jruby/issues/6267 * https://github.com/rsim/oracle-enhanced/issues/2027 * https://github.com/rails/rails/pull/39121 One of two things needs to happen: * Weakmap is made a public API after some discussion. It would be an official public feature only in 2.8/3.0 or higher. * ~~The specs are be removed and Weakmap remains a private API not to be used by the community. I suspect the addition of the specs led to folks starting to use this private API.~~ (edit: The Rails PR was merged after the specs, but the change is actually a year old, as mentioned below. In any case there's plenty of in-the-wild uses of WeakMap that go back even further.) Personally, I'm in much more in favor of making WeakRef support all the features necessary to implement Weakmap in pure Ruby, rather than the other way around: https://bugs.ruby-lang.org/issues/6309 But whatever happens it needs to happen soon, since this use case is now a merged feature in Rails master. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113470] [Ruby master Bug#5179] Complex#rationalize and to_r with approximate zeros
by kjtsanaktsidis (KJ Tsanaktsidis) 14 May '23

14 May '23

Issue #5179 has been updated by kjtsanaktsidis (KJ Tsanaktsidis). I've explored the behaviour here a bit, and I think I do believe that `0.0` really does represent the concept of "zero". Specifically, it's the number that satisfies the following properties, for all possible non-NaN floats N: * `abs(0.0 * N) == 0.0` * `0.0 + N == N` You might obtain a bit pattern for 0.0 from the result of a calculation which might have otherwise produced a non-zero result if there was more precision; however that doesn't change the fact that the bit pattern you have really does satisfy the mathematical properties of zero. Additionally, I think the distinction that complex.c currently draws between floating-point and non-floating-point zero also results in some other surprising results: ``` irb(main):030:0> Complex(3, 7) ** 0.0 => (1.0+0.0i) irb(main):031:0> Complex(3, 7) ** 0 => (1+0i) irb(main):032:0> Complex(3.2, 7) ** 0 => (1+0i) irb(main):033:0> Complex(3.2, 7) ** 0.0 => (1.0+0.0i) ``` Why does the type of `A ** B` depend on whether B is a float or not, but not A? ``` irb(main):034:0> Complex(3.2, 7) ** Complex(1, 0.0) => (3.200000000000001+6.999999999999999i) irb(main):035:0> Complex(3.2, 7) ** Complex(1, 0) => (3.2+7i) ``` This doesn't _need_ to accumulate floating point error - the code in `rb_complex_pow` special-cases `(a + bi) ** (c + 0i) -->(a + bi) ** c`, but the special-casing is skipped if `0i` is `0.0i` instead. Also, `#to_i` and `#to_f` have the same issue as `#to_r`: ``` irb(main):037:0> Complex(3, 0.0).to_i (irb):37:in `to_i': can't convert 3+0.0i into Integer (RangeError) irb(main):038:0> Complex(3, 0.0).to_f (irb):38:in `to_f': can't convert 3+0.0i into Float (RangeError) ``` This is especially odd for `#to_i` because it of course has no problems truncating away the real part: ``` irb(main):040:0> Complex(3.1, 0).to_i => 3 ``` I think we should change all the checks for `k_exact_zero_p` in `complex.c` to `f_zero_p`; i.e. in all the places where `complex.c` special-cases integer zero, also make it special-case floating-point zero. If people agree with this (especially people who actually use Complex in their code - I have pretty much never used it!) I can send a pretty simple patch to cloe this out. ---------------------------------------- Bug #5179: Complex#rationalize and to_r with approximate zeros https://bugs.ruby-lang.org/issues/5179#change-103049 * Author: marcandre (Marc-Andre Lafortune) * Status: Assigned * Priority: Normal * Assignee: mrkn (Kenta Murata) * ruby -v: r32354 ---------------------------------------- Currently, Complex#rationalize and Complex#to_r raise a RangeError if the imaginary part is nonzero *or is a Float*. Note that a BigDecimal(0) is accepted, though: Complex(1, 0).to_r # => Rational(1,1) Complex(1, BigDecimal("0.0")).to_r # => Rational(1,1) Complex(1, 0.0).to_r # => RangeError This is inconsistent. I recommend not raising an error for 0.0 (Float or BigDecimal). Any objection? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113464] [Ruby master Misc#16512] Improving `www.ruby-lang.org` reference by merging with `rubyreferences.github.io`
by hsbt (Hiroshi SHIBATA) 13 May '23

13 May '23

Issue #16512 has been updated by hsbt (Hiroshi SHIBATA). Assignee changed from zverok (Victor Shepelev) to hsbt (Hiroshi SHIBATA) I discussed this with @ufuk. We will work this to forward. ---------------------------------------- Misc #16512: Improving `www.ruby-lang.org` reference by merging with `rubyreferences.github.io` https://bugs.ruby-lang.org/issues/16512#change-103043 * Author: matz (Yukihiro Matsumoto) * Status: Assigned * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- @zverok prepared better-looking reference pages at `rubyreferences.github.io`. I think there's room for improvement of reference pages on `www.ruby-lang.org`. @hsbt Is there a chance to work with? Matz. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113463] [Ruby master Feature#10602] Support multithreaded profiling
by ivoanjo (Ivo Anjo) 12 May '23

12 May '23

Issue #10602 has been updated by ivoanjo (Ivo Anjo). PR to implement this being discussed in https://github.com/ruby/ruby/pull/7784 ---------------------------------------- Feature #10602: Support multithreaded profiling https://bugs.ruby-lang.org/issues/10602#change-103042 * Author: mperham (Mike Perham) * Status: Open * Priority: Normal ---------------------------------------- The current `rb_profile_frames` captures the frame for whatever thread is current. This makes profiling a multithreaded system impossible. I'd like a `rb_thread_profile_frames` which captures a given thread. It seems like it would be a very simple change, something like this: ~~~ int rb_profile_frames(int start, int limit, VALUE *buff, int *lines) { rb_profile_frames(start, limit, buff, lines, GET_THREAD()) } int rb_thread_profile_frames(int start, int limit, VALUE *buff, int *lines, rb_thread_t *th) { int i; rb_control_frame_t *cfp = th->cfp, *end_cfp = RUBY_VM_END_CONTROL_FRAME(th); ... ~~~ This way profiling gems could lock to a specific thread. -- https://bugs.ruby-lang.org/

1 0