December 2022 - ruby-core - ml.ruby-lang.org

[ruby-core:111249] [Ruby master Feature#19104] Introduce the cache-based optimization for Regexp matching
by mmizutani (Minoru Mizutani) 10 Dec '22

10 Dec '22

Issue #19104 has been updated by mmizutani (Minoru Mizutani). Regex fuzzing encountered an edge-case regression: ```bash $ ruby --version ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] $ time ruby -e 'puts /あ(?~い)/.match?("あ")' true 0.04s user 0.04s system 74% cpu 0.118 total ``` ```bash $ ruby --version ruby 3.2.0rc1 (2022-12-10) [x86_64-linux] $ time ruby -e 'puts /あ(?~い)/.match?("あ")' # This regex matching involving an absent operator and non-ASCII characters runs indefinitely and consumes gigabytes of memory, eventually killed by the OS. ``` ---------------------------------------- Feature #19104: Introduce the cache-based optimization for Regexp matching https://bugs.ruby-lang.org/issues/19104#change-100542 * Author: make_now_just (Hiroya Fujinami) * Status: Open * Priority: Normal ---------------------------------------- Regexp matching causes a time-complexity explosion problem as known as ReDoS ([https://en.wikipedia.org/wiki/ReDoS](https://en.wikipedia.org/wiki/ReDoS)) ReDoS has become serious vulnerability in many places in recent years, and Ruby is no exception. The following is the incomplete list of such vulnerability reports: - [https://github.com/sinatra/sinatra/pull/1823](https://github.com/sinatra/si… - [https://github.com/lostisland/faraday-net_http/pull/27](https://github.com/… These reports have been addressed by fixing the library/software implementation. But, if the language’s Regexp implementation becomes safe, the vulnerability is fundamentally archived. For a few months, Ruby has implemented a Regexp matching timeout ([https://bugs.ruby-lang.org/issues/17837](https://bugs.ruby-lang.org/issues/…) It is one of the useful methods for preventing ReDoS vulnerability, but it is a problem that setting a correct timeout value is hard. This value is depending on input length, environment, network status, system load, etc. When the value is too small, a system may be broken, but when the value is too large, it is not useful for preventing ReDoS. Therefore, as a new way to prevent ReDoS, we propose to introduce cache-based optimization for Regexp matching. As CS fundamental knowledge, automaton matching result depends on the position of input and state. In addition, matching time explosion is caused for repeating to arrive at the same position and state many times. Then, ReDoS can be prevented when pairs of position, and state arrived once is recorded (cached). In fact, under such an optimization, it is known as the matching time complexity is linear against input size [1]. [1]: Davis, James C., Francisco Servant, and Dongyoon Lee. "Using selective memoization to defeat regular expression denial of service (ReDoS)." *2021 IEEE symposium on security and privacy (SP)*. IEEE, 2021. [https://www.computer.org/csdl/proceedings-article/sp/2021/893400a543/1oak98… See the following example. ```bash $ ruby --version ruby 3.2.0preview2 (2022-09-09 master 35cfc9a3bb) [arm64-darwin21] $ time ruby -e '/^(a|a)*$/ =~ "a" * 28 + "b"' ruby -e '/^(a|a)*$/ =~ "a" * 28 + "b"' 8.49s user 0.04s system 98% cpu 8.663 total $ ./miniruby --version ruby 3.2.0dev (2022-10-27T13:39:56Z recache bc59b7cc1e) [arm64-darwin21] $ time ./miniruby -e '/^(a|a)*$/ =~ "a" * 28 + "b"' ./miniruby -e '/^(a|a)*$/ =~ "a" * 28 + "b"' 0.01s user 0.01s system 8% cpu 0.310 total ``` In this example, using ruby v3.2.0-preview2, matching `/^(a|a)*$/` against `"a" * 28 + "b"` takes 8.6 seconds because matching time of this Regexp takes exponentially against `"a" * N + "b"` form string. But, using the patched version of ruby, it takes 0.01 seconds. Incredibly it is 860 times faster because matching is done in linear time. By this optimization, the matching time is linear to the input size. It sounds secure and good. Unfortunately, when Regexp uses some extension (e.g. look-around, back-reference, subexpression call), the optimization is not applied. Also, the optimization needs a bit more memory for caching. However, we have already investigated that they are not so the major problems (See the "Limitation" section). ## Implementation The basic cache implementation is complete at this time and can be found in the following Pull Request. [https://github.com/ruby/ruby/pull/6486](https://github.com/ruby/ruby/pull/6… Some tests seem to be failed, but it is no problem! Because the failed tests are for Regexp timeout, optimization works correctly and so they failed as expected. Of course, we need to fix these tests before merging. Implementation notes: - A null-check on loop causes non-linear behavior, so the field to index the latest null-check item on the stack is added to OnigStackType. ([https://github.com/ruby/ruby/pull/6486/files#diff-4347460e379cd970ba0b88b4a…) - When the loop is null and this loop has a capture, matching behaves as a monomaniac. To reproduce this behavior, caches in the loop is cleared as necessary. ([https://github.com/ruby/ruby/pull/6486/files#diff-c3cfe19efff0cc51813847413…) Like a flip-flop operator, we hope to drop this if possible. But it still remains backward compatibility. ### Limitation Cache-based optimization is not applied in the following cases: 1. Regexp using some extensions (back-reference and subexpression call, look-around, atomic, absent operators) is not optimized because it is impossible or hard. However, it may be possible for look-around operators. 2. A bounded or fixed times repetition nesting in another repetition (e.g. `/(a{2,3})*/`). It is an implementation issue entirely, but we believe it is hard to support this case correctly. 3. Bounded or fixed times repetition is too large (e.g. `/(a|b){100000,200000}/`). The cache table size is proportional to the product of the number of cache points of regex and input size. In this case, since the number of cache points becomes too large, the optimization cannot be applied. Experiments were conducted to investigate how these limitations are problematic in practice. We used ObjectSpace to collect Regexps and investigate whether they could be optimized and the number of cache points. Regexps were collected from the standard library, Webrick, and Rails. See the following gist for the details ([https://gist.github.com/makenowjust/83e1e75a2d7de8b956e93bdac004a06b](https…) The experiments result is shown in the following table. | Collected from | # Regexp | # non-optimizable | Maximum number of cache points | | --- | --- | --- | --- | | stdlib | 1009 | 86 (8.52%) | 81 | | Webrick | 356 | 44 (12.36%) | 20 | | Rails | 759 | 74 (7.75%) | 27 | | Total <br> (Duplications are reduced) | 1506 | 118 (7.84%) | 81 | This result shows that the percentage of non-optimizable Regexp is less than 10%, and the amount of memory used for optimization is about 10 times the length of the string (81/8, for a bit array) at worst in this case. It is considered that a sufficient number of Regexp can be optimized in practice. ## Specification The detailed specification has been fixed yet. We have some ideas and we would like to discuss them. - When is optimization enabled? Currently, it turns on when the backtrack causes as many as the input length. - How the number of cache points is allowed, and how memory can be allocated? It is not determined for now. - If the above parameters can be specified by users, how are they specified? (via command-line flags, or static / instance parameters like `Regexp#.timeout=` and `Regexp#timeout=`) - Unless the input size is too large, the availability of optimization can be determined on compile-time. So, we would like to add a new flag to Regexp to determine whether a cache is available. It becomes one of the criteria for whether Regexp is efficiently executable or not. We believe it helps users. Thus, which letter is preferred for this purpose? `l` (linear) or `r` (regular) sounds good, but I am not sure which is the best. Thank you. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111246] [Ruby master Bug#19087] String#to_c supports multiple "_"
by nagachika (Tomoyuki Chikanaga) 09 Dec '22

09 Dec '22

Issue #19087 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONTNEED I agree that the new behavior is consistent and preferable, but I don't think it should be fixed on the stable branches. If there are any troubles on the real world applications with this issue, please let me know. ---------------------------------------- Bug #19087: String#to_c supports multiple "_" https://bugs.ruby-lang.org/issues/19087#change-100537 * Author: andrykonchin (Andrew Konchin) * Status: Closed * Priority: Normal * Backport: 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONTNEED ---------------------------------------- I've noticed a minor inconsistent behaviour of `String#to_c` method. It does not treat a sequence of `_` character as an end of meaningful characters: ```ruby "123".to_c # => (123+0i) "12_3".to_c # => (123+0i) "12__3".to_c # => (123+0i) "12___3".to_c # => (123+0i) ``` Similar methods `#to_i` and `#to_r` treat multiple `_` in a different way - they just ignore characters after it: ```ruby "12__3".to_i # => 12 "12__3".to_r # => (12/1) ``` I would expect `#to_c` to behave similarly. --- ``` $ ruby -v ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin21] ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111244] [Ruby master Feature#19036] Provide a way to set path for File instances created with for_fd
by headius (Charles Nutter) 09 Dec '22

09 Dec '22

Issue #19036 has been updated by headius (Charles Nutter). Eregon (Benoit Daloze) wrote in #note-7: > > - Should we introduce `IO#path=`? > > I think not, AFAIK there is no need for this to be mutable. Agree. Also briefly brainstormed some possible security issues with being able to mutate the path after init. Don't want to go there and don't really see a need. > > - Should we introduce `IO#dup(..., path:)`? > > Definitely not worth the complexity, and `dup` doesn't have this general contract anyway. Agree. Additional question: why is rb_io_path not part of the public C API? Luckily, rb_file_path wasn't either, but it seems like it would be useful to expose rb_io_path now. ---------------------------------------- Feature #19036: Provide a way to set path for File instances created with for_fd https://bugs.ruby-lang.org/issues/19036#change-100535 * Author: headius (Charles Nutter) * Status: Closed * Priority: Normal ---------------------------------------- Ruby provides `IO.for_fd` to instantiate an IO object from an existing file descriptor value. The logic for this simply calls the base `IO.new` logic, which for all IO and subtypes simply wraps the given file descriptor. When called against File, or other subtypes of IO, this has the side effect of creating an IO instance with that type, e.g. `File.for_fd` will behave identically to `IO.for_fd` except that the class of the resulting object will be File. Unfortunately, this results in a File object that does not have any `path` associated with it: ``` 3.1.2 :001 > f = File.open('README.md') => #<File:README.md> 3.1.2 :002 > f.path => "README.md" 3.1.2 :003 > f2 = File.for_fd(f.fileno) => #<File:fd 5> 3.1.2 :004 > f2.path (irb):4:in `path': File is unnamed (TMPFILE?) (IOError) from (irb):4:in `<main>' from /home/headius/.rvm/rubies/ruby-3.1.2/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `load' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `<main>' ``` I propose that there should be a way, via an extra parameter or a keyword argument, to provide a path when constructing a new File via `for_fd`. Possible forms: * `File.for_fd(fileno, "my/path")` * `File.for_fd(fileno, path: "my/path")` This would necessitate a separate implementation for `File.for_fd` unless we want to make it possible to set a path for all `for_fd` calls (which may not make sense for many of them). This came up while trying to implement a pure-Ruby (plus FFI) version of the "pty" library. Without overriding the `path` function, it is not possible for the File object returned by `PTY.open` to gain the "masterpty:<slavename>" filename, and therefore it does not clearly indicate it is from a PTY. See https://github.com/jruby/jruby/pull/7391, an attempt to match inspect output for these return values using `define_singleton_method`. Providing a way to set the path would make this automatic without the singleton definition. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111243] [Ruby master Feature#19000] Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object]
by Eregon (Benoit Daloze) 08 Dec '22

08 Dec '22

Issue #19000 has been updated by Eregon (Benoit Daloze). I also agree the 0-kwargs case should be supported for general use like `data.with(**changes)`, whether `changes` is empty or not. And that reads fine of course. It's also like `Hash#merge` with no arguments. I think `#with` with empty kwargs should do the same as `dup`. So this depends on what `Data#dup` does. If `Data#dup` returns `self` like immutable objects (Fixnum/Symbol/true/false/nil/etc) then let's do that too, if not let's dup in that case. Note that `Data` is not truly immutable (nor always Ractor-shareable) because it can refer to mutable objects. Currently `Data#dup` creates a new instance. ---------------------------------------- Feature #19000: Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object] https://bugs.ruby-lang.org/issues/19000#change-100533 * Author: RubyBugs (A Nonymous) * Status: Open * Priority: Normal ---------------------------------------- *As requested: extracted a follow-up to #16122 Data: simple immutable value object from [this comment](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/109815)* # Proposal: Add a "Copy with changes" method to Data Assume the proposed `Data.define` exists. Seeing examples from the [[Values gem]](https://github.com/ms-ati/Values): ```ruby require "values" # A new class Point = Value.new(:x, :y) # An immutable instance Origin = Point.with(x: 0, y: 0) # Q: How do we make copies that change 1 or more values? right = Origin.with(x: 1.0) up = Origin.with(y: 1.0) up_and_right = right.with(y: up.y) # In loops movements = [ [ :x, +0.5 ], [ :x, +0.5 ], [ :y, -1.0 ], [ :x, +0.5 ], ] # position = Point(x: 1.5, y: -1.0) position = movements.inject(Origin) do |p, (field, delta)| p.with(field => p.send(field) + delta) end ``` ## Proposed detail: Call this method: `#with` ```ruby Money = Data.define(:amount, :currency) account = Money.new(amount: 100, currency: 'USD') transactions = [+10, -5, +15] account = transactions.inject(account) { |a, t| a.with(amount: a.amount + t) } #=> Money(amount: 120, currency: "USD") ``` ## Why add this "Copy with changes" method to the Data simple immutable value class? Called on an instance, it returns a new instance with only the provided parameters changed. This API affordance is now **widely adopted across many languages** for its usefulness. Why is it so useful? Because copying immutable value object instances, with 1 or more discrete changes to specific fields, is the proper and ubiquitous pattern that takes the place of mutation when working with immutable value objects. **Other languages** C# Records: “immutable record structs — Non-destructive mutation” — is called `with { ... }` https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-t… Scala Case Classes — is called `#copy` https://docs.scala-lang.org/tour/case-classes.html Java 14+ Records — Brian Goetz at Oracle is working on adding a with copy constructor inspired by C# above as we speak, likely to be called `#with` https://mail.openjdk.org/pipermail/amber-spec-experts/2022-June/003461.html Rust “Struct Update Syntax” via `..` syntax in constructor https://doc.rust-lang.org/book/ch05-01-defining-structs.html#creating-insta… ## Alternatives Without a copy-with-changes method, one must construct entirely new instances using the constructor. This can either be (a) fully spelled out as boilerplate code, or (b) use a symmetrical `#to_h` to feed the keyword-args constructor. **(a) Boilerplate using constructor** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(x: Origin.x, y: Origin.y, **change) ``` **(b) Using a separately proposed `#to_h` method and constructor symmetry** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(**(Origin.to_h.merge(change))) ``` Notice that the above are not ergonomic -- leading so many of our peer language communities to adopt the `#with` method to copy an instance with discrete changes. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111242] [Ruby master Feature#19036] Provide a way to set path for File instances created with for_fd
by Eregon (Benoit Daloze) 08 Dec '22

08 Dec '22

Issue #19036 has been updated by Eregon (Benoit Daloze). ioquatix (Samuel Williams) wrote in #note-6: > Implemented in https://github.com/ruby/ruby/pull/6867. Thanks! > Follow up discussion: > > - Should we introduce `IO#path=`? I think not, AFAIK there is no need for this to be mutable. > - Should we introduce `IO#dup(..., path:)`? Definitely not worth the complexity, and `dup` doesn't have this general contract anyway. ---------------------------------------- Feature #19036: Provide a way to set path for File instances created with for_fd https://bugs.ruby-lang.org/issues/19036#change-100532 * Author: headius (Charles Nutter) * Status: Closed * Priority: Normal ---------------------------------------- Ruby provides `IO.for_fd` to instantiate an IO object from an existing file descriptor value. The logic for this simply calls the base `IO.new` logic, which for all IO and subtypes simply wraps the given file descriptor. When called against File, or other subtypes of IO, this has the side effect of creating an IO instance with that type, e.g. `File.for_fd` will behave identically to `IO.for_fd` except that the class of the resulting object will be File. Unfortunately, this results in a File object that does not have any `path` associated with it: ``` 3.1.2 :001 > f = File.open('README.md') => #<File:README.md> 3.1.2 :002 > f.path => "README.md" 3.1.2 :003 > f2 = File.for_fd(f.fileno) => #<File:fd 5> 3.1.2 :004 > f2.path (irb):4:in `path': File is unnamed (TMPFILE?) (IOError) from (irb):4:in `<main>' from /home/headius/.rvm/rubies/ruby-3.1.2/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `load' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `<main>' ``` I propose that there should be a way, via an extra parameter or a keyword argument, to provide a path when constructing a new File via `for_fd`. Possible forms: * `File.for_fd(fileno, "my/path")` * `File.for_fd(fileno, path: "my/path")` This would necessitate a separate implementation for `File.for_fd` unless we want to make it possible to set a path for all `for_fd` calls (which may not make sense for many of them). This came up while trying to implement a pure-Ruby (plus FFI) version of the "pty" library. Without overriding the `path` function, it is not possible for the File object returned by `PTY.open` to gain the "masterpty:<slavename>" filename, and therefore it does not clearly indicate it is from a PTY. See https://github.com/jruby/jruby/pull/7391, an attempt to match inspect output for these return values using `define_singleton_method`. Providing a way to set the path would make this automatic without the singleton definition. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111241] [Ruby master Misc#18976] [ANN] blade.nagaokaut.ac.jp is down
by hsbt (Hiroshi SHIBATA) 08 Dec '22

08 Dec '22

Issue #18976 has been updated by hsbt (Hiroshi SHIBATA). I restored some of mojibake messages of Japanese lists that are `ruby-list` and `ruby-dev`. ---------------------------------------- Misc #18976: [ANN] blade.nagaokaut.ac.jp is down https://bugs.ruby-lang.org/issues/18976#change-100531 * Author: hsbt (Hiroshi SHIBATA) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- The mail archive server named `blade.nagaokaut.ac.jp` is down now. blade had some hardware issues. Prof. Hara tries to salvage mail data and rebuild blade. But It's difficult status. I have a plan to migrate Ruby mail server includes mailing-list to Google workspace. I also rebuild mail archives using Google groups or others. Sorry for the inconvenient experience. FYI: original announce in Japanese https://github.com/ruby-no-kai/official/issues/306#issuecomment-1207819210 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111135] [Ruby master Bug#19167] Object#inspect does not correctly show NilClass TrueClass and FalseClass stored in instance variables
by tompng (tomoya ishida) 08 Dec '22

08 Dec '22

Issue #19167 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19167: Object#inspect does not correctly show NilClass TrueClass and FalseClass stored in instance variables https://bugs.ruby-lang.org/issues/19167 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- ~~~ruby Object.new.instance_eval do @a = nil @b = NilClass @c = true @d = TrueClass @e = [nil, NilClass, true, TrueClass] puts self.inspect end # actual # => #<Object:0x2f15e3c8 @a=nil, @b=nil, @c=true, @d=true, @e=[nil, NilClass, true, TrueClass]> # expected # => #<Object:0x2f15e3c8 @a=nil, @b=NilClass, @c=true, @d=TrueClass, @e=[nil, NilClass, true, TrueClass]> ~~~ -- https://bugs.ruby-lang.org/

4 6

[ruby-core:111239] [Ruby master Misc#18976] [ANN] blade.nagaokaut.ac.jp is down
by hsbt (Hiroshi SHIBATA) 08 Dec '22

08 Dec '22

Issue #18976 has been updated by hsbt (Hiroshi SHIBATA). Status changed from Assigned to Closed >I'll build the simple txt archives like https://blade.ruby-lang.org/ruby-core/xxxxxx. >I restore mailing-list data to it with ruby-core, ruby-dev, ruby-list, ruby-ext and ruby-math. >Change archive link of bugs.ruby-lang.org to them. done. You can access the above archive like https://blade.ruby-lang.org/ruby-core/1 , https://blade.ruby-lang.org/ruby-dev/1 ... >I'll create the mirror program for the new mail for ruby-core and ruby-dev. also done. https://github.com/ruby/b.r-l.o/pull/170 I didn't confirm all of mail data are restored. Because I only have the archives provided from Prof. Hara, ko1 and shugo. If you find a missing messages, please share to me. I'll restore it manually. ---------------------------------------- Misc #18976: [ANN] blade.nagaokaut.ac.jp is down https://bugs.ruby-lang.org/issues/18976#change-100529 * Author: hsbt (Hiroshi SHIBATA) * Status: Closed * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- The mail archive server named `blade.nagaokaut.ac.jp` is down now. blade had some hardware issues. Prof. Hara tries to salvage mail data and rebuild blade. But It's difficult status. I have a plan to migrate Ruby mail server includes mailing-list to Google workspace. I also rebuild mail archives using Google groups or others. Sorry for the inconvenient experience. FYI: original announce in Japanese https://github.com/ruby-no-kai/official/issues/306#issuecomment-1207819210 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111238] [Ruby master Feature#19036] Provide a way to set path for File instances created with for_fd
by ioquatix (Samuel Williams) 08 Dec '22

08 Dec '22

Issue #19036 has been updated by ioquatix (Samuel Williams). Status changed from Open to Closed Implemented in https://github.com/ruby/ruby/pull/6867. Follow up discussion: - Should we introduce `IO#path=`? - Should we introduce `IO#dup(..., path:)`? ---------------------------------------- Feature #19036: Provide a way to set path for File instances created with for_fd https://bugs.ruby-lang.org/issues/19036#change-100527 * Author: headius (Charles Nutter) * Status: Closed * Priority: Normal ---------------------------------------- Ruby provides `IO.for_fd` to instantiate an IO object from an existing file descriptor value. The logic for this simply calls the base `IO.new` logic, which for all IO and subtypes simply wraps the given file descriptor. When called against File, or other subtypes of IO, this has the side effect of creating an IO instance with that type, e.g. `File.for_fd` will behave identically to `IO.for_fd` except that the class of the resulting object will be File. Unfortunately, this results in a File object that does not have any `path` associated with it: ``` 3.1.2 :001 > f = File.open('README.md') => #<File:README.md> 3.1.2 :002 > f.path => "README.md" 3.1.2 :003 > f2 = File.for_fd(f.fileno) => #<File:fd 5> 3.1.2 :004 > f2.path (irb):4:in `path': File is unnamed (TMPFILE?) (IOError) from (irb):4:in `<main>' from /home/headius/.rvm/rubies/ruby-3.1.2/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `load' from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `<main>' ``` I propose that there should be a way, via an extra parameter or a keyword argument, to provide a path when constructing a new File via `for_fd`. Possible forms: * `File.for_fd(fileno, "my/path")` * `File.for_fd(fileno, path: "my/path")` This would necessitate a separate implementation for `File.for_fd` unless we want to make it possible to set a path for all `for_fd` calls (which may not make sense for many of them). This came up while trying to implement a pure-Ruby (plus FFI) version of the "pty" library. Without overriding the `path` function, it is not possible for the File object returned by `PTY.open` to gain the "masterpty:<slavename>" filename, and therefore it does not clearly indicate it is from a PTY. See https://github.com/jruby/jruby/pull/7391, an attempt to match inspect output for these return values using `define_singleton_method`. Providing a way to set the path would make this automatic without the singleton definition. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111235] [Ruby master Feature#19000] Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object]
by RubyBugs (A Nonymous) 07 Dec '22

07 Dec '22

Issue #19000 has been updated by RubyBugs (A Nonymous). Hi @mame! Thank you for your questions. mame (Yusuke Endoh) wrote in #note-24: > Thanks for the update. > > Now I have a question. Do you really want to write `p.with(field => p.send(field) + delta)`? I don't think it is very elegant. It is not very convincing (at least, to me) as a first motivation example. > The challenge is that this operation "make a copy of an immutable value object with 0 or more fields changed" is so fundamental, it's hard to find examples that strip away every other concern. Staying with the `Point` example, would **translate** and **invert** work better as examples that change multiple fields? ```ruby ## # Example of proposed Data#with # # To try this out: # # ruby-install ruby-3.2.0-preview3 # ## Point3d = Data.define(:x, :y, :z) do # Example only, too slow due to allocations def with(**args) raise ArgumentError unless args.keys.all? { |k| members.include? k } self.class.new(**(to_h.merge(args))) end def translate_2d(dx: 0, dy: 0) with(x: x + dx, y: y + dy) end def invert_2d with(x: -x, y: -y) end end Origin3d = Point3d.new(x: 0, y: 0, z:0) north = Origin3d.translate_2d(dy: 1.0) south = north.invert_2d east = Origin3d.translate_2d(dx: 1.0) west = east.invert_2d west == Point3d.new(x: -1.0, y: 0, z: 0) # => true mountain_height = 2.0 western_mountain = west.with(z: mountain_height) # => #<data Point3d x=-1.0, y=0, z=2.0> ``` > Also, do you need the ability to update multiple fields at once? Both motivation examples only update a single field. This may be the result of simplifying the motivation example, though. > Yes! Definitely need to update multiple fields at once > Looking at these motivation examples, there may be room to consider an API like `p.with(field) {|old_value| old_value + delta }` or something. Agreed that this probably comes out of a motivating example which is trying to calculate the new value for a **single** field based on **only** the previous value of that field. ---------------------------------------- Feature #19000: Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object] https://bugs.ruby-lang.org/issues/19000#change-100524 * Author: RubyBugs (A Nonymous) * Status: Open * Priority: Normal ---------------------------------------- *As requested: extracted a follow-up to #16122 Data: simple immutable value object from [this comment](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/109815)* # Proposal: Add a "Copy with changes" method to Data Assume the proposed `Data.define` exists. Seeing examples from the [[Values gem]](https://github.com/ms-ati/Values): ```ruby require "values" # A new class Point = Value.new(:x, :y) # An immutable instance Origin = Point.with(x: 0, y: 0) # Q: How do we make copies that change 1 or more values? right = Origin.with(x: 1.0) up = Origin.with(y: 1.0) up_and_right = right.with(y: up.y) # In loops movements = [ [ :x, +0.5 ], [ :x, +0.5 ], [ :y, -1.0 ], [ :x, +0.5 ], ] # position = Point(x: 1.5, y: -1.0) position = movements.inject(Origin) do |p, (field, delta)| p.with(field => p.send(field) + delta) end ``` ## Proposed detail: Call this method: `#with` ```ruby Money = Data.define(:amount, :currency) account = Money.new(amount: 100, currency: 'USD') transactions = [+10, -5, +15] account = transactions.inject(account) { |a, t| a.with(amount: a.amount + t) } #=> Money(amount: 120, currency: "USD") ``` ## Why add this "Copy with changes" method to the Data simple immutable value class? Called on an instance, it returns a new instance with only the provided parameters changed. This API affordance is now **widely adopted across many languages** for its usefulness. Why is it so useful? Because copying immutable value object instances, with 1 or more discrete changes to specific fields, is the proper and ubiquitous pattern that takes the place of mutation when working with immutable value objects. **Other languages** C# Records: “immutable record structs — Non-destructive mutation” — is called `with { ... }` https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-t… Scala Case Classes — is called `#copy` https://docs.scala-lang.org/tour/case-classes.html Java 14+ Records — Brian Goetz at Oracle is working on adding a with copy constructor inspired by C# above as we speak, likely to be called `#with` https://mail.openjdk.org/pipermail/amber-spec-experts/2022-June/003461.html Rust “Struct Update Syntax” via `..` syntax in constructor https://doc.rust-lang.org/book/ch05-01-defining-structs.html#creating-insta… ## Alternatives Without a copy-with-changes method, one must construct entirely new instances using the constructor. This can either be (a) fully spelled out as boilerplate code, or (b) use a symmetrical `#to_h` to feed the keyword-args constructor. **(a) Boilerplate using constructor** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(x: Origin.x, y: Origin.y, **change) ``` **(b) Using a separately proposed `#to_h` method and constructor symmetry** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(**(Origin.to_h.merge(change))) ``` Notice that the above are not ergonomic -- leading so many of our peer language communities to adopt the `#with` method to copy an instance with discrete changes. -- https://bugs.ruby-lang.org/

1 0