August 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118952] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by nobu (Nobuyoshi Nakada) 25 Aug '24

25 Aug '24

Issue #19057 has been updated by nobu (Nobuyoshi Nakada). OK, unless `FMODE_EXTERNAL` is given `descriptor` will be closed when a new object could not be allocated. I thought about pty extension, and it seems to work at least for pty. ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-109525 * Author: ioquatix (Samuel Williams) * Status: Assigned * Assignee: ioquatix (Samuel Williams) * Target version: 3.4 ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118951] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by nobu (Nobuyoshi Nakada) 25 Aug '24

25 Aug '24

Issue #19057 has been updated by nobu (Nobuyoshi Nakada). I found `rb_io_open_descriptor` is a bad design, it can results in FD-leaks, because a file descriptor isn't a Ruby object with finalizer. It should be "allocate then open & set" instead of "open then allocate & initialize". ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-109524 * Author: ioquatix (Samuel Williams) * Status: Assigned * Assignee: ioquatix (Samuel Williams) * Target version: 3.4 ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118933] [Ruby master Bug#20694] "try to mark T_NONE object" error for very large arrays
by ngan (Ngan Pham) 25 Aug '24

25 Aug '24

Issue #20694 has been reported by ngan (Ngan Pham). ---------------------------------------- Bug #20694: "try to mark T_NONE object" error for very large arrays https://bugs.ruby-lang.org/issues/20694 * Author: ngan (Ngan Pham) * Status: Open * ruby -v: 3.3.4 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- In CI, we've been occasionally seeing `[BUG] try to mark T_NONE object`. I don't have any way to reproduce it but I'm going to put some information here in case anyone has any ideas. One thing I've noticed is that it happens for very large arrays: ``` <OBJ_INFO:gc_mark_ptr@gc.c:7072> 0x00007f587d04fc90 [0 M ] T_NONE /usr/local/bundle/ruby/3.3.0/gems/simpleidn-0.2.2/lib/simpleidn/uts46mapping.rb: [BUG] try to mark T_NONE object ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] ``` https://github.com/mmriis/simpleidn/blob/master/lib/simpleidn/uts46mapping.… and... ``` <OBJ_INFO:gc_mark_ptr@gc.c:7072> 0x00007f7ec426fdc8 [0 M ] T_NONE /usr/local/bundle/ruby/3.3.0/gems/domain_name-0.6.20240107/lib/domain_name/etld_data.rb: [BUG] try to mark T_NONE object ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux] ``` https://github.com/knu/ruby-domain_name/blob/master/lib/domain_name/etld_da… ---Files-------------------------------- backtrace.txt (9.12 KB) -- https://bugs.ruby-lang.org/

3 6

[ruby-core:118535] [Ruby master Feature#20625] Object#chain_of
by matheusrich (Matheus Richard) 24 Aug '24

24 Aug '24

Issue #20625 has been reported by matheusrich (Matheus Richard). ---------------------------------------- Feature #20625: Object#chain_of https://bugs.ruby-lang.org/issues/20625 * Author: matheusrich (Matheus Richard) * Status: Open ---------------------------------------- ## Motivation It's often common to traverse a tree/list-like structure in order to get a chain of elements. This proposal is to add a method to `Object` that allows collecting a chain of elements by applying a block to each element. It doesn't require the root element to be an instance of a specific class or respond to a specific protocol. I think this method could be useful in many cases, since hierarchies like this are common in codebases (e.g. file systems, organizations structures, commit histories, breadcrumbs in web apps, configuration hierarchies, etc.). Here are some examples extracted from real codebases (simplified for the sake of the example/privacy): ```rb # Given a file system structure, get the breadcrumbs of a file or directory def breadcrumbs(root) crumbs = [] current = root while current crumbs << current current = current.parent_dir end crumbs end # Given an employee, get the hierarchy of managers def hierarchy(employee) hierarchy = [] current = employee while current hierarchy << current current = current.manager end hierarchy end ``` ## Implementation The implementation in Ruby could look like this: ```rb class Object def chain_of(&block) chain = [] current = self while current chain << current current = block.call(current) end chain end end ``` Here's an example use: ```rb class ListNode attr_accessor :value, :parent def initialize(value, parent = nil) @value = value @parent = parent end def ancestors chain_of(&:parent).shift end end root = ListNode.new("root") child1 = ListNode.new("child1", root) child2 = ListNode.new("child2", child1) puts child2.ancestors.map(&:value) # => ["child1", "root"] ``` The examples from the motivation section could be rewritten as: ```rb breadcrumbs = root.chain_of(&:parent_directory) hierarchy = employee.chain_of(&:manager) ``` ## Considerations - While I'm including the object by default in the chain, it could be more intuitive to exclude it. In any case, it's easy to remove or add it with `shift`/`unshift`. - On a different note, the method could be named differently (I do like `chain_of`, though). Some alternatives I've considered are `map_chain`, `traverse`, and `trace_path`. - The method assumes that the traversal will finish at some point. If the user has a cyclic structure, it will loop indefinitely. We could stop looping if we find the same element twice. I don't think it's worth the extra complexity. - I'm not sure `Object` is the best place for this method. While it's very general, I think it gives power to the user to decide how to traverse a chain like this without having to rely on a specific class. Maybe a mixin (`Traversable`/`Chainable`) would be more appropriate? Could this fit in `Enumerable`, somehow? Of course, I'm open to suggestions and feedback. Thanks for reading! -- https://bugs.ruby-lang.org/

3 4

[ruby-core:118941] [Ruby master Bug#20697] `Rational("1r")` fails while `Complex("1i")` succeeds
by nobu (Nobuyoshi Nakada) 23 Aug '24

23 Aug '24

Issue #20697 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20697: `Rational("1r")` fails while `Complex("1i")` succeeds https://bugs.ruby-lang.org/issues/20697 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- @mame reported: ```ruby Rational("1r") #=> invalid value for convert(): "1r" (ArgumentError) ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118940] [Ruby master Bug#20696] `IO.read` ignores `err:` option
by nobu (Nobuyoshi Nakada) 23 Aug '24

23 Aug '24

Issue #20696 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20696: `IO.read` ignores `err:` option https://bugs.ruby-lang.org/issues/20696 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I expected the following code captures the error message, but `err:` option is ignored. ```shell-session $ ruby -e 'p IO.read("|ruby -e raise", err: %i[child out])' -e:1:in '<main>': unhandled exception "" ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118939] [Ruby master Bug#20695] Elevated GC allocations in parse.y parser
by alanwu (Alan Wu) 23 Aug '24

23 Aug '24

Issue #20695 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20695: Elevated GC allocations in parse.y parser https://bugs.ruby-lang.org/issues/20695 * Author: alanwu (Alan Wu) * Status: Open * ruby -v: ruby 3.4.0dev (2024-08-23T14:49:27Z master 3f6be01bfc) [arm64-darwin23] * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONTNEED ---------------------------------------- Testing on the `lobsters` benchmark from [`yjit-bench`](https://github.com/shopify/yjit-bench), the latest `master` parser is allocating more objects compared to commit:98eeadc932 ("Development of 3.4.0 started."). The following patch shows allocation count after code loading: ```diff diff --git a/benchmarks/lobsters/benchmark.rb b/benchmarks/lobsters/benchmark.rb index 240c50c..6cdd0ac 100644 --- a/benchmarks/lobsters/benchmark.rb +++ b/benchmarks/lobsters/benchmark.rb @@ -7,6 +7,8 @@ Dir.chdir __dir__ use_gemfile require_relative 'config/environment' +printf "allocated_after_load=%d\n", GC.stat(:total_allocated_objects) +exit require_relative "route_generator" # For an in-mem DB, we need to load all data on every boot ``` ``` $ ruby benchmarks/lobsters/benchmark.rb ruby 3.4.0dev (2023-12-25T09:13:40Z master 98eeadc932) [arm64-darwin23] <snip> allocated_after_load=1747084 $ chruby master $ ruby benchmarks/lobsters/benchmark.rb ruby 3.4.0dev (2024-08-23T14:49:27Z master 3f6be01bfc) [arm64-darwin23] <snip> allocated_after_load=2163031 $ ruby --parser=prism benchmarks/lobsters/benchmark.rb ruby 3.4.0dev (2024-08-23T14:49:27Z master 3f6be01bfc) +PRISM [arm64-darwin23] <snip> allocated_after_load=932571 $ math 2163031 / 1747084 1.238081 ``` --- Profiling shows extra GC allocations coming from set_number_literal(); set_yylval_node() expands to a call to rb_enc_str_new(). So this issue seems related to #20659. A surprising side effect of these extra allocations is that they seem to impact the speed of the benchmark, not only the loading speed, even though the body of benchmark *does not do any Ruby parsing* after warmup iterations. The speed impact seem to be due to poorer data locality, as perf(1) shows elevated levels of `CYCLE_ACTIVITY.STALLS_L2_MISS` and related events. Using Prism speeds up benchmark iterations, probably since it allocates a lot fewer during code loading, leaving the heap in a better layout in terms of locality. With Prism, fewer cycles stall on data cache misses. ``` $ ruby run_benchmarks.rb --chruby 'master;mstr+p::master --parser=prism' lobsters <snip> master: ruby 3.4.0dev (2024-08-23T14:49:27Z master 3f6be01bfc) [arm64-darwin23] mstr+p: ruby 3.4.0dev (2024-08-23T14:49:27Z master 3f6be01bfc) +PRISM [arm64-darwin23] -------- ----------- ---------- ----------- ---------- -------------- ------------- bench master (ms) stddev (%) mstr+p (ms) stddev (%) mstr+p 1st itr master/mstr+p lobsters 567.9 0.7 557.0 0.8 1.00 1.02 -------- ----------- ---------- ----------- ---------- -------------- ------------- Legend: - mstr+p 1st itr: ratio of master/mstr+p time for the first benchmarking iteration. - master/mstr+p: ratio of master/mstr+p time. Higher is better for mstr+p. Above 1 represents a speedup. ``` In hindsight, it's clear how that the parser's allocation pattern can have a significant impact on heap layout, since typical app boot with Kernel#require alternates between running app code and running the parser. Parser allocations essentially end up as gaps between app objects. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118929] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by Dan0042 (Daniel DeLorme) 22 Aug '24

22 Aug '24

Issue #18368 has been updated by Dan0042 (Daniel DeLorme). matz (Yukihiro Matsumoto) wrote in #note-31: > It seems this change breaks step over string ranges (e.g. `"a".."z").step(3)`). We need to handle string ranges specifically. Thank you Matz, I am truly overjoyed the one in charge still cares about backward compatibility. knu (Akinori MUSHA) wrote in #note-33: > If we take this compatibility as important and go with it, should we give up `("a".."aaaa").step("a")` or support it? I'd say definitely support it. That way Range#step behavior remains largely consistent, with just one little special extra for compat. BTW not sure if it's important but `(:a .. :z).step(3)` is also supported. ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-109498 * Author: zverok (Victor Shepelev) * Status: Open ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118910] [Ruby master Bug#20688] Use-after-free in WeakMap and WeakKeyMap
by peterzhu2118 (Peter Zhu) 22 Aug '24

22 Aug '24

Issue #20688 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20688: Use-after-free in WeakMap and WeakKeyMap https://bugs.ruby-lang.org/issues/20688 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/11421 We cannot free the weakmap_entry before the ST_DELETE because it could hash the key which would read the weakmap_entry and would cause a use-after-free. Instead, we store the entry and free it on the next iteration. For example, the following script triggers a use-after-free in Valgrind: ```ruby weakmap = ObjectSpace::WeakMap.new 10_000.times { weakmap[Object.new] = Object.new } ``` ``` ==25795== Invalid read of size 8 ==25795== at 0x462297: wmap_cmp (weakmap.c:165) ==25795== by 0x3A2B1C: find_table_bin_ind (st.c:930) ==25795== by 0x3A5EAA: st_general_foreach (st.c:1599) ==25795== by 0x3A5EAA: rb_st_foreach (st.c:1640) ==25795== by 0x25C991: gc_mark_children (default.c:4870) ==25795== by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565) ==25795== by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557) ==25795== by 0x25C991: rgengc_rememberset_mark (default.c:6233) ==25795== by 0x25C991: gc_marks_start (default.c:6057) ==25795== by 0x25C991: gc_marks (default.c:6077) ==25795== by 0x25C991: gc_start (default.c:6723) ==25795== by 0x260F96: heap_prepare (default.c:2282) ==25795== by 0x260F96: heap_next_free_page (default.c:2489) ==25795== by 0x260F96: newobj_cache_miss (default.c:2598) ==25795== by 0x26197F: newobj_alloc (default.c:2622) ==25795== by 0x26197F: rb_gc_impl_new_obj (default.c:2701) ==25795== by 0x26197F: newobj_of (gc.c:890) ==25795== by 0x26197F: rb_wb_protected_newobj_of (gc.c:917) ==25795== by 0x2DEA88: rb_class_allocate_instance (object.c:131) ==25795== by 0x2E3B18: class_call_alloc_func (object.c:2141) ==25795== by 0x2E3B18: rb_class_alloc (object.c:2113) ==25795== by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172) ==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786) ==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953) ==25795== by 0x44B08D: vm_exec_core (insns.def:898) ==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564) ==25795== by 0x234914: rb_ec_exec_node (eval.c:281) ==25795== Address 0x21603710 is 0 bytes inside a block of size 16 free'd ==25795== at 0x4849B2C: free (vg_replace_malloc.c:989) ==25795== by 0x249651: rb_gc_impl_free (default.c:8527) ==25795== by 0x249651: rb_gc_impl_free (default.c:8508) ==25795== by 0x249651: ruby_sized_xfree.constprop.0 (gc.c:4178) ==25795== by 0x4626EC: ruby_sized_xfree_inlined (gc.h:277) ==25795== by 0x4626EC: wmap_free_entry (weakmap.c:45) ==25795== by 0x4626EC: wmap_mark_weak_table_i (weakmap.c:61) ==25795== by 0x3A5CEF: apply_functor (st.c:1633) ==25795== by 0x3A5CEF: st_general_foreach (st.c:1543) ==25795== by 0x3A5CEF: rb_st_foreach (st.c:1640) ==25795== by 0x25C991: gc_mark_children (default.c:4870) ==25795== by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565) ==25795== by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557) ==25795== by 0x25C991: rgengc_rememberset_mark (default.c:6233) ==25795== by 0x25C991: gc_marks_start (default.c:6057) ==25795== by 0x25C991: gc_marks (default.c:6077) ==25795== by 0x25C991: gc_start (default.c:6723) ==25795== by 0x260F96: heap_prepare (default.c:2282) ==25795== by 0x260F96: heap_next_free_page (default.c:2489) ==25795== by 0x260F96: newobj_cache_miss (default.c:2598) ==25795== by 0x26197F: newobj_alloc (default.c:2622) ==25795== by 0x26197F: rb_gc_impl_new_obj (default.c:2701) ==25795== by 0x26197F: newobj_of (gc.c:890) ==25795== by 0x26197F: rb_wb_protected_newobj_of (gc.c:917) ==25795== by 0x2DEA88: rb_class_allocate_instance (object.c:131) ==25795== by 0x2E3B18: class_call_alloc_func (object.c:2141) ==25795== by 0x2E3B18: rb_class_alloc (object.c:2113) ==25795== by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172) ==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786) ==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953) ==25795== by 0x44B08D: vm_exec_core (insns.def:898) ==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564) ==25795== Block was alloc'd at ==25795== at 0x484680F: malloc (vg_replace_malloc.c:446) ==25795== by 0x25CE9E: rb_gc_impl_malloc (default.c:8542) ==25795== by 0x462A39: wmap_aset_replace (weakmap.c:423) ==25795== by 0x3A5542: rb_st_update (st.c:1487) ==25795== by 0x462B8E: wmap_aset (weakmap.c:452) ==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786) ==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953) ==25795== by 0x44B08D: vm_exec_core (insns.def:898) ==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564) ==25795== by 0x234914: rb_ec_exec_node (eval.c:281) ==25795== by 0x2369B8: ruby_run_node (eval.c:319) ==25795== by 0x15D675: rb_main (main.c:43) ==25795== by 0x15D675: main (main.c:62) ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:118926] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by zverok (Victor Shepelev) 22 Aug '24

22 Aug '24

Issue #18368 has been updated by zverok (Victor Shepelev). @mame It doesn’t take long, but I’ll have time for that on the weekend (most probably Sunday). I hope that some clarifications might happen in a few days before that. ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-109495 * Author: zverok (Victor Shepelev) * Status: Open ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0