December 2022 - ruby-core - ml.ruby-lang.org

[ruby-core:111163] [Ruby master Feature#17325] Adds Fiber#cancel, which forces a Fiber to break/return
by nevans (Nicholas Evans) 02 Dec '22

02 Dec '22

Issue #17325 has been updated by nevans (Nicholas Evans). This should be closed, in favor of a (TBD) configurable structured concurrency system, built on `Fiber.current.storage` (#19078). :) ---------------------------------------- Feature #17325: Adds Fiber#cancel, which forces a Fiber to break/return https://bugs.ruby-lang.org/issues/17325#change-100448 * Author: nevans (Nicholas Evans) * Status: Open * Priority: Normal ---------------------------------------- Calling `Fiber#cancel` will force a fiber to return, skipping rescue and catch blocks but running all ensure blocks. It behaves as if a `break` or `return` were used to jump from the last suspension point to the top frame of the fiber. Control will be transferred to the canceled fiber so it can run its ensure blocks. ## Propagation from resuming to resumed fibers Any non-root living fiber can be canceled and cancellation will propagate to child (resumed) fibers. In this way, a suspended task can be canceled even if it is e.g. resuming into an enumerator, and the enumerator will be canceled as well. Transfer of control should match #17221's *(much improved)* transfer/resume semantics. After the cancellation propagates all the way to the bottom of the fiber resume stack, the last fiber in the chain will then be resumed. Resuming fibers will not run until they are yielded back into. ## Suspension of canceled fibers Canceled fibers can still transfer control with `resume`, `yield`, and `transfer`, which may be necessary in order to release resources from `ensure` blocks. For simplicity, subsequent cancels will behave similarly to calling `break` or `return` inside an `ensure` block, and the last cancellation reason will overwrite earlier reasons. ## Alternatives `Fiber#raise` could be used, but: * Can only raise on resumable fibers. * Cannot propagate cancellation down to resumed fibers. * Exceptions are bigger and slower than `break`. * `#raise` can't (and shouldn't) be sent to resuming fibers. (It can't propagate.) * Exceptions can be caught. This might be desirable, but that should be at the discretion of the calling fiber. Catch/Throw could be used (with an anonymous `Object.new`), but: * We would need to add `Fiber#throw` (or wrap/intercept `Fiber.yield`). * A hypothetical `Fiber#throw` should probably have similar semantics to `#resume` and thus only be allowed on resumable fibers. * In that case, it wouldn't propagate down to resumed fibers. * `catch` adds an extra stack frame. We could use go-style "Context" objects that contain a "done?" queue/future. * These would need to be explicitly passed around. * Although their usage could be enforced via linters like rubocop, I think that placing it off to the side will give developers the impression that it is optional Some sort of cancel propagation mechanism is not optional for structured concurrency. * It should built into any task-scheduler library, which would allow application code to use it explicitly. * But this suffers the same problem as current Fiber wrappers: it works fine if your code uses the wrapper, but code that uses fibers without the wrapper can be incompatible and introduce bugs (e.g. fibers that are released without running their `ensure` blocks). * This make sense for a language like go which doesn't have exceptions but does have a convention of returning an "error" value. It feels out of place in ruby, IMO. Letting the fiber-task-scheduler mitigates that... for code that uses the fiber-task-scheduler. We could add a keyword option to `Fiber#raise` that gives it similar propagation semantics to this. * IMO, the simplicity of `Fiber#raise` simply being a specialized version of `Fiber#resume` is worth preserving. * The propagation changes alone are enough of a semantic difference to warrant a new method. We could implement `Fiber#cancel` by using `fiber.raise(FiberCancellationError)` on the bottom fiber and catching that exception during termination of the canceled fiber. * This would have the "benefit" that the exception could be rescued. * I might be wrong, but I think that doing this would mostly duplicate my PR, but with some added complexity around exception construction and catching. * It might be a good keyword option? e.g. `Fiber#cancel(with_exception: [true,#exception,#to_str])` Just let the task-fiber-scheduler library handle this. * That's what I'm already doing now. It's mostly fine. It works in my code. * Putting it into ruby core should lead to a small performance boost on very commonly repeated code. * There's probably a better way to store the `cancel_reason` that doesn't require the overhead of adding another `VALUE` to `rb_fiber_struct`. Maybe it can be placed directly into `errinfo`? * Although the common cases can be handled via a trampoline fiber or #17221, there can still be situations where your application's fiber-scheduler library might not know about fibers created by other libraries. This adds interoperability to a common scenario. * Coroutine cancellation is IMO a core feature. It's important to have something like this for all applications and libraries to use as a baseline for interoperability. Implementation: https://github.com/ruby/ruby/pull/3766 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111162] [Ruby master Feature#19000] Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object]
by ufuk (Ufuk Kayserilioglu) 02 Dec '22

02 Dec '22

Issue #19000 has been updated by ufuk (Ufuk Kayserilioglu). Thanks for the discussion and the name suggestion. I updated the PR https://github.com/ruby/ruby/pull/6766 to use `Data#with`. I would be grateful if it could get a review. However, that raises a question: How should `Data#with` behave if it is passed no arguments? Semantically, that should mean "create a copy of the current data object with no fields changed", but then that ends up being identical to what `Kernel#dup` does. Also, `with` with no arguments does not read that well. @matz and @mame Should it be an error? Example: ```ruby Point = Data.define(:x, :y) origin = Point.new(x: 0, y: 0) new_origin = origin.with # should this be an error? ``` ---------------------------------------- Feature #19000: Data: Add "Copy with changes method" [Follow-on to #16122 Data: simple immutable value object] https://bugs.ruby-lang.org/issues/19000#change-100445 * Author: RubyBugs (A Nonymous) * Status: Open * Priority: Normal ---------------------------------------- *As requested: extracted a follow-up to #16122 Data: simple immutable value object from [this comment](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/109815)* # Proposal: Add a "Copy with changes" method to Data Assume the proposed `Data.define` exists. Seeing examples from the [[Values gem]](https://github.com/ms-ati/Values): ```ruby require "values" # A new class Point = Value.new(:x, :y) # An immutable instance Origin = Point.with(x: 0, y: 0) # Q: How do we make copies that change 1 or more values? right = Origin.with(x: 1.0) up = Origin.with(y: 1.0) up_and_right = right.with(y: up.y) # In loops movements = [ [ :x, +0.5 ], [ :x, +0.5 ], [ :y, -1.0 ], [ :x, +0.5 ], ] # position = Point(x: 1.5, y: -1.0) position = movements.inject(Origin) do |p, (field, delta)| p.with(field => p.send(field) + delta) end ``` ## Proposed detail: Call this method: `#with` ```ruby Money = Data.define(:amount, :currency) account = Money.new(amount: 100, currency: 'USD') transactions = [+10, -5, +15] account = transactions.inject(account) { |a, t| a.with(amount: a.amount + t) } #=> Money(amount: 120, currency: "USD") ``` ## Why add this "Copy with changes" method to the Data simple immutable value class? Called on an instance, it returns a new instance with only the provided parameters changed. This API affordance is now **widely adopted across many languages** for its usefulness. Why is it so useful? Because copying immutable value object instances, with 1 or more discrete changes to specific fields, is the proper and ubiquitous pattern that takes the place of mutation when working with immutable value objects. **Other languages** C# Records: “immutable record structs — Non-destructive mutation” — is called `with { ... }` https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-t… Scala Case Classes — is called `#copy` https://docs.scala-lang.org/tour/case-classes.html Java 14+ Records — Brian Goetz at Oracle is working on adding a with copy constructor inspired by C# above as we speak, likely to be called `#with` https://mail.openjdk.org/pipermail/amber-spec-experts/2022-June/003461.html Rust “Struct Update Syntax” via `..` syntax in constructor https://doc.rust-lang.org/book/ch05-01-defining-structs.html#creating-insta… ## Alternatives Without a copy-with-changes method, one must construct entirely new instances using the constructor. This can either be (a) fully spelled out as boilerplate code, or (b) use a symmetrical `#to_h` to feed the keyword-args constructor. **(a) Boilerplate using constructor** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(x: Origin.x, y: Origin.y, **change) ``` **(b) Using a separately proposed `#to_h` method and constructor symmetry** ```ruby Point = Data.define(:x, :y, :z) Origin = Point.new(x: 0.0, y: 0.0, z: 0.0) change = { z: -1.5 } # Have to use full constructor -- does this even work? point = Point.new(**(Origin.to_h.merge(change))) ``` Notice that the above are not ergonomic -- leading so many of our peer language communities to adopt the `#with` method to copy an instance with discrete changes. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111161] [Ruby master Bug#19175] Ripper does not recognize some pattern matching defining local variable
by tompng (tomoya ishida) 02 Dec '22

02 Dec '22

Issue #19175 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19175: Ripper does not recognize some pattern matching defining local variable https://bugs.ruby-lang.org/issues/19175 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Ripepr does not recognize hshptn and aryptn splat defining local variables. ~~~ruby Ripper.sexp('a in [*x]; x') # => [:program, [[:case, [:vcall, [:@ident, "a", [1, 0]]], [:in, [:aryptn, nil, nil, [:var_field, [:@ident, "x", [1, 7]]], nil], nil, nil]], [:vcall, [:@ident, "x", [1, 11]]]]] Ripper.sexp('a in {x:}; x') # => [:program, [[:case, [:vcall, [:@ident, "a", [1, 0]]], [:in, [:hshptn, nil, [[[:@label, "x:", [1, 6]], nil]], nil], nil, nil]], [:vcall, [:@ident, "x", [1, 11]]]]] ~~~ RubyVM::AbstractSyntaxTree returns `(LVAR@1:11-1:12 :x)` I think `[:vcall, [:@ident, "x", [1, 11]]]` should be `[:var_ref, [:@ident, "x", [1, 11]]]` Other pattern seems to be OK. ~~~ruby Ripper.sexp('a in [x, y => z]; x; y; z') # => [:program, [[:case, [:vcall, [:@ident, "a", [1, 0]]], [:in, [:aryptn, nil, [[:var_field, [:@ident, "x", [1, 6]]], [:binary, [:var_field, [:@ident, "y", [1, 9]]], :"=>", [:var_field, [:@ident, "z", [1, 14]]]]], nil, nil], nil, nil]], [:var_ref, [:@ident, "x", [1, 18]]], [:var_ref, [:@ident, "y", [1, 21]]], [:var_ref, [:@ident, "z", [1, 24]]]]] ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111157] [Ruby master Bug#19174] YJIT configure "checking whether rustc works for YJIT... no" in rustc 1.65.0 on x86_64
by jaruga (Jun Aruga) 02 Dec '22

02 Dec '22

Issue #19174 has been reported by jaruga (Jun Aruga). ---------------------------------------- Bug #19174: YJIT configure "checking whether rustc works for YJIT... no" in rustc 1.65.0 on x86_64 https://bugs.ruby-lang.org/issues/19174 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- On the latest master branch `11871e49c4fe493d6b958046969bc863f7fb6627` on Fedora 36, ``` $ uname -m x86_64 $ which rustc /bin/rustc $ rustc --version rustc 1.65.0 (Fedora 1.65.0-1.fc37) ``` I am seeing the "checking whether rustc works for YJIT... no". ``` $ ./autogen.sh $ ./configure --enable-yjit 2>&1 | tee configure.log ... checking for rustc... rustc checking whether rustc works for YJIT... no ... * YJIT support: yes ... ``` Seeing the code <https://github.com/ruby/ruby/blob/11871e49c4fe493d6b958046969bc863f7fb6627/…>, it is to check if the rustc is >= 1.58.0. And the used rustc is 1.65.0. ``` dnl check if rustc is recent enough to build YJIT (rustc >= 1.58.0) ``` I suspect that the checking logic might be something wrong. I checked why the checking logic fails with the modification below. ``` $ git diff diff --git a/configure.ac b/configure.ac index 79167a9c67..33a1b9540d 100644 --- a/configure.ac +++ b/configure.ac @@ -3753,7 +3753,7 @@ AS_IF([test "$RUSTC" != "no"], dnl Fails in case rustc target doesn't match ruby target. dnl Can happen on Rosetta, for example. AS_IF([echo "#[cfg(target_arch = \"$YJIT_TARGET_ARCH\")]\n fn main() { let x = 1; format!(\"{x}\"); }" | - $RUSTC - --emit asm=/dev/null 2>/dev/null], + $RUSTC - --emit asm=/dev/null], [YJIT_RUSTC_OK=yes] ) AC_MSG_RESULT($YJIT_RUSTC_OK) ``` ``` $ ./autogen.sh $ ./configure --enable-yjit 2>&1 | tee configure_debug.log ... checking for rustc... rustc checking whether rustc works for YJIT... error: unknown start of token: \ --> <anon>:1:31 | 1 | #[cfg(target_arch = "x86_64")]\n fn main() { let x = 1; format!("{x}"); } | ^ error: expected one of `!` or `::`, found keyword `fn` --> <anon>:1:34 | 1 | #[cfg(target_arch = "x86_64")]\n fn main() { let x = 1; format!("{x}"); } | ^^ expected one of `!` or `::` error: aborting due to 2 previous errors no ... * YJIT support: yes ... ``` Is it intended behavior? Actually after running the `configure`, the `make` works. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111159] [Ruby master Feature#19078] Introduce `Fiber#storage` for inheritable fiber-scoped variables.
by nevans (Nicholas Evans) 02 Dec '22

02 Dec '22

Issue #19078 has been updated by nevans (Nicholas Evans). @Dan0042 (Daniel DeLorme) wrote in #note-21: > What about when we want per-ractor storage... a fourth type? FWIW, `Ractor#[]` has existed since 3.0: https://docs.ruby-lang.org/en/3.0/Ractor.html#method-i-5B-5D :) > Per-ractor inheritable storage... a fifth type? In my opinion, absolutely not. The following is just a hypothetical--if and when I have an actual proposal, I'll post it in another ticket. :) This feature is brand-new so it has some "rough edges", but I think should be updated to automatically share all ractor-sharable values. Of course, the big question is: what to do for non-ractor-sharable values? Hypothetically: 1) By default, simple raise an exception from Ractor.new, just like it does any other time you accidentally attempt to send non-sharable state into a new Ractor. 2) Add a kwarg to Ractor.new, and a method to the fiber storage, both of which will simply drop all non-ractor sharable storage. Either way: non-inheritance should always be an explicit opt-out. 3) Add an API for more fine-grained control at certain points, so that library authors can designate different behaviors via callback. Because often the desired behavior is more complicated than simply dropping or inheriting a variable: e.g. rather than inherit the current task id, a new fiber/thread/ractor might automatically create a brand new task which is a child of the current task. For inspiration here, we might look to Kotlin, which delegates a great deal of functionality to the current `coroutineContext[CoroutineDispatcher]` (this would be handled entirely by library code, and invisible to most users), to `coroutineContext[Job]` (which combines with `CoroutineScope` and a few others to handle almost everything related to structured concurrency), and any [ThreadContextElement] that's been added to the context. That last one is mostly used for backward compatibility with libraries that expect thread-locals. Similarly, go contexts delegate internally to `context.Value(cancelContextKey)`, where `cancelContextKey` is a private package var which is used purely for comparison-by-identity. This is essentially the `@key = Object.new` trick demonstrated by @ioquatix in #note-10, and it's the recommended usage pattern for context keys in go. [ThreadContextElement]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotli… Anyway: my goal here isn't to propose any particular approach, but to show this feature might be used as a foundation for other future features. IMO, fine-grained inheritance control might be out-of-scope for 3.2 (at least, any API we put together now would need to be marked "experimental"). I simply wanted to show how other languages build on a similar foundation to handle the problems you mentioned. ---------------------------------------- Feature #19078: Introduce `Fiber#storage` for inheritable fiber-scoped variables. https://bugs.ruby-lang.org/issues/19078#change-100443 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- Pull Request: https://github.com/ruby/ruby/pull/6612 This is an evolution of the previous ideas: - https://bugs.ruby-lang.org/issues/19058 - https://bugs.ruby-lang.org/issues/19062 This PR introduces fiber scoped variables, and is a solution for problems like <https://github.com/ioquatix/ioquatix/discussions/17>. The main interface is: ```ruby Fiber[key] = value Fiber[key] # => value ``` The variables are scoped (local to) a fiber and inherited into child fibers and threads. ```ruby Fiber[:request_id] = SecureRandom.hex(16) Fiber.new do p Fiber[:request_id] # prints the above request id end ``` The fiber scoped variables are stored and can be accessed: ```ruby Fiber.current.storage # => returns a Hash (copy) of the internal storage. Fiber.current.storage= # => assigns a Hash (copy) to the internal storage. ``` Fiber itself has one new keyword argument: ``` Fiber.new(..., storage: hash, false, undef, nil) ``` This can control how the fiber variables are setup in a child context. To minimise the performance overhead of some of the implementation choices, we are also simultaneously implementing <https://bugs.ruby-lang.org/issues/19077>. ## Examples ### Request loop ```ruby Thread.new do while request = queue.pop Fiber.new(storage: {id: SecureRandom.hex(16)}) do handle_request.call(request) end end end ``` OR ```ruby Thread.new do while request = queue.pop Fiber.current.storage = {id: SecureRandom.hex(16)} handle_request.call(request) end end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111158] [Ruby master Feature#19078] Introduce `Fiber#storage` for inheritable fiber-scoped variables.
by nevans (Nicholas Evans) 02 Dec '22

02 Dec '22

Issue #19078 has been updated by nevans (Nicholas Evans). I'm also a bit late to this. While I do have some nitpicks, I'll leave them for another ticket (if I ever get around to it). But, to answer a couple of @Dan0042's questions (and maybe others): As mentioned by @eregon above, one of the most important use-cases for this is structured concurrency. If you try to implement structured concurrency, you will eventually wind up needing and implementing some version of this feature. If you look at the documentation and API for go's and kotlin's versions of this feature, it's overwhelmingly about structured concurrency. @eregon included a good link describing structured concurrency, above. If I remember correctly, this page is also a good introduction to the idea: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-con… ---- Here's a short list of the same basic feature, as implemented in several other languages (I copied a couple of these links from the other tickets linked in the description): * [go's context package](https://pkg.go.dev/context#pkg-overview) * Maybe the simplest implementation of the idea? * [Kotlin's Coroutine Contexts](https://kotlinlang.org/docs/coroutine-context-and-dispatchers.htm… * [python's contextvars](https://docs.python.org/3/library/contextvars.html) (integrated with `asyncio`, etc) * See [PEP 567](https://peps.python.org/pep-0567/) (and PEP-550 which preceded it) for more info on design details, etc. * [JEP 429: Extent-Local Variables](https://openjdk.org/jeps/429) This ticket is implemented a little bit differently than these, but is basically in the same category and has many of the same motivations, etc. ---------------------------------------- Feature #19078: Introduce `Fiber#storage` for inheritable fiber-scoped variables. https://bugs.ruby-lang.org/issues/19078#change-100441 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- Pull Request: https://github.com/ruby/ruby/pull/6612 This is an evolution of the previous ideas: - https://bugs.ruby-lang.org/issues/19058 - https://bugs.ruby-lang.org/issues/19062 This PR introduces fiber scoped variables, and is a solution for problems like <https://github.com/ioquatix/ioquatix/discussions/17>. The main interface is: ```ruby Fiber[key] = value Fiber[key] # => value ``` The variables are scoped (local to) a fiber and inherited into child fibers and threads. ```ruby Fiber[:request_id] = SecureRandom.hex(16) Fiber.new do p Fiber[:request_id] # prints the above request id end ``` The fiber scoped variables are stored and can be accessed: ```ruby Fiber.current.storage # => returns a Hash (copy) of the internal storage. Fiber.current.storage= # => assigns a Hash (copy) to the internal storage. ``` Fiber itself has one new keyword argument: ``` Fiber.new(..., storage: hash, false, undef, nil) ``` This can control how the fiber variables are setup in a child context. To minimise the performance overhead of some of the implementation choices, we are also simultaneously implementing <https://bugs.ruby-lang.org/issues/19077>. ## Examples ### Request loop ```ruby Thread.new do while request = queue.pop Fiber.new(storage: {id: SecureRandom.hex(16)}) do handle_request.call(request) end end end ``` OR ```ruby Thread.new do while request = queue.pop Fiber.current.storage = {id: SecureRandom.hex(16)} handle_request.call(request) end end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111036] [Ruby master Bug#19156] ObjectSpace.dump_all segfault during string inspection
by mk 02 Dec '22

02 Dec '22

Issue #19156 has been reported by mk (Matthias Käppler). ---------------------------------------- Bug #19156: ObjectSpace.dump_all segfault during string inspection https://bugs.ruby-lang.org/issues/19156 * Author: mk (Matthias Käppler) * Status: Open * Priority: Normal * ruby -v: ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- I am working on a feature that would allow our application to capture heap dumps during shutdown for later inspection. These heap dumps are captured via `ObjectSpace.dump_all(output: io)`. While walking the object space, MRI occasionally segfaults while inspect string object in `search_nonascii` of `string.c`: ``` /usr/local/lib/ruby/3.0.0/objspace.rb:87: [BUG] Segmentation fault at 0x00007efee4201000 ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux] ... -- Control frame information ----------------------------------------------- c:0053 p:---- s:0312 e:000311 CFUNC :_dump_all c:0052 p:0130 s:0305 e:000304 METHOD /usr/local/lib/ruby/3.0.0/objspace.rb:87 c:0051 p:0023 s:0295 e:000294 METHOD /home/git/gitlab/lib/gitlab/memory/reports/heap_dump.rb:26 ... -- C level backtrace information ------------------------------------------- /usr/local/lib/libruby.so.3.0(rb_print_backtrace+0x11) [0x7efee4ad0c5e] vm_dump.c:758 /usr/local/lib/libruby.so.3.0(rb_vm_bugreport) vm_dump.c:998 /usr/local/lib/libruby.so.3.0(rb_bug_for_fatal_signal+0xf8) [0x7efee48d0b08] error.c:787 /usr/local/lib/libruby.so.3.0(sigsegv+0x55) [0x7efee4a23db5] signal.c:963 /lib/x86_64-linux-gnu/libpthread.so.0(__restore_rt+0x0) [0x7efee4f12140] ../sysdeps/pthread/funlockfile.c:28 /usr/local/lib/libruby.so.3.0(search_nonascii+0x30) [0x7efee4a3ca60] string.c:552 /usr/local/lib/libruby.so.3.0(coderange_scan) string.c:585 /usr/local/lib/libruby.so.3.0(enc_coderange_scan+0x1b) [0x7efee4a3e28a] string.c:709 /usr/local/lib/libruby.so.3.0(rb_enc_str_coderange) string.c:727 /usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(is_broken_string+0x8) [0x7efeced9c304] ../../internal/string.h:116 /usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(dump_object) objspace_dump.c:388 /usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(heap_i+0x39) [0x7efeced9caa9] objspace_dump.c:521 /usr/local/lib/libruby.so.3.0(objspace_each_objects_without_setup+0xaf) [0x7efee48e878f] gc.c:3232 /usr/local/lib/libruby.so.3.0(objspace_each_objects_protected+0x14) [0x7efee48e87c4] gc.c:3242 /usr/local/lib/libruby.so.3.0(rb_ensure+0x12a) [0x7efee48d96aa] eval.c:1162 /usr/local/lib/libruby.so.3.0(objspace_each_objects+0x28) [0x7efee48fb458] gc.c:3310 /usr/local/lib/libruby.so.3.0(rb_objspace_each_objects) gc.c:3298 /usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(objspace_dump_all+0x88) [0x7efeced9b068] objspace_dump.c:616 ... ``` Unfortunately I couldn't get my hands on that memory region to see which strings are causing this since this doesn't always happen. I suspect this is also a problem with MRI master since the code looks unchanged from 3.0.4. -- https://bugs.ruby-lang.org/

2 24

[ruby-core:111153] [Ruby master Bug#19012] BasicSocket#recv* methods return an empty packet instead of nil on closed connections
by byroot (Jean Boussier) 02 Dec '22

02 Dec '22

Issue #19012 has been updated by byroot (Jean Boussier). Apparently we can get this via `getsockopt(fd, SOL_SOCKET, SO_TYPE, &type, &length);`, the question being how portable it is. ---------------------------------------- Bug #19012: BasicSocket#recv* methods return an empty packet instead of nil on closed connections https://bugs.ruby-lang.org/issues/19012#change-100437 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- `man recvmsg(2)` states: > Return Value > These calls return the number of bytes received, or -1 if an error occurred. The return value will be 0 when the peer has performed an orderly shutdown. But somehow the entire `receiv` family of methods in Ruby seem to interpret `0` as empty string instead of "EOF". ```ruby require 'socket' puts "=== pipes ===" r, w = IO.pipe r.read_nonblock(1, exception: false) # => :wait_readable w.close r.read_nonblock(1, exception: false) # => nil (EOF) puts "=== sockets ====" r, w = UNIXSocket.socketpair r.read_nonblock(1, exception: false) # => :wait_readable r.recvmsg_nonblock(1, exception: false) # => :wait_readable r.recv_nonblock(1, exception: false) # => :wait_readable w.close r.read_nonblock(1, exception: false) # => nil (EOF) r.recvmsg_nonblock(1, exception: false) # => ["", #<Addrinfo: empty-sockaddr SOCK_STREAM>, 128]] r.recvmsg # => ["", #<Addrinfo: empty-sockaddr SOCK_STREAM>, 0]] r.recv_nonblock(1, exception: false) # => "" ``` ### Expected behavior I would expect `recvmsg_nonblock`, `recvmsg`, `recv_nonblock` and `recv` to return `nil` when the connection is closed. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111152] [Ruby master Bug#19012] BasicSocket#recv* methods return an empty packet instead of nil on closed connections
by byroot (Jean Boussier) 02 Dec '22

02 Dec '22

Issue #19012 has been updated by byroot (Jean Boussier). > I am not sure if there is a portable way to determine if the file descriptor behind an IO object is stream or datagram. Yeah, me neither. I'll try to dig more. ---------------------------------------- Bug #19012: BasicSocket#recv* methods return an empty packet instead of nil on closed connections https://bugs.ruby-lang.org/issues/19012#change-100436 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- `man recvmsg(2)` states: > Return Value > These calls return the number of bytes received, or -1 if an error occurred. The return value will be 0 when the peer has performed an orderly shutdown. But somehow the entire `receiv` family of methods in Ruby seem to interpret `0` as empty string instead of "EOF". ```ruby require 'socket' puts "=== pipes ===" r, w = IO.pipe r.read_nonblock(1, exception: false) # => :wait_readable w.close r.read_nonblock(1, exception: false) # => nil (EOF) puts "=== sockets ====" r, w = UNIXSocket.socketpair r.read_nonblock(1, exception: false) # => :wait_readable r.recvmsg_nonblock(1, exception: false) # => :wait_readable r.recv_nonblock(1, exception: false) # => :wait_readable w.close r.read_nonblock(1, exception: false) # => nil (EOF) r.recvmsg_nonblock(1, exception: false) # => ["", #<Addrinfo: empty-sockaddr SOCK_STREAM>, 128]] r.recvmsg # => ["", #<Addrinfo: empty-sockaddr SOCK_STREAM>, 0]] r.recv_nonblock(1, exception: false) # => "" ``` ### Expected behavior I would expect `recvmsg_nonblock`, `recvmsg`, `recv_nonblock` and `recv` to return `nil` when the connection is closed. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111147] [Ruby master Feature#19078] Introduce `Fiber#storage` for inheritable fiber-scoped variables.
by ioquatix (Samuel Williams) 02 Dec '22

02 Dec '22

Issue #19078 has been updated by ioquatix (Samuel Williams). I made a compatibility shim for older Rubies. It's not 100% perfect, but it's close enough to be useful. https://github.com/ioquatix/fiber-storage ---------------------------------------- Feature #19078: Introduce `Fiber#storage` for inheritable fiber-scoped variables. https://bugs.ruby-lang.org/issues/19078#change-100431 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- Pull Request: https://github.com/ruby/ruby/pull/6612 This is an evolution of the previous ideas: - https://bugs.ruby-lang.org/issues/19058 - https://bugs.ruby-lang.org/issues/19062 This PR introduces fiber scoped variables, and is a solution for problems like <https://github.com/ioquatix/ioquatix/discussions/17>. The main interface is: ```ruby Fiber[key] = value Fiber[key] # => value ``` The variables are scoped (local to) a fiber and inherited into child fibers and threads. ```ruby Fiber[:request_id] = SecureRandom.hex(16) Fiber.new do p Fiber[:request_id] # prints the above request id end ``` The fiber scoped variables are stored and can be accessed: ```ruby Fiber.current.storage # => returns a Hash (copy) of the internal storage. Fiber.current.storage= # => assigns a Hash (copy) to the internal storage. ``` Fiber itself has one new keyword argument: ``` Fiber.new(..., storage: hash, false, undef, nil) ``` This can control how the fiber variables are setup in a child context. To minimise the performance overhead of some of the implementation choices, we are also simultaneously implementing <https://bugs.ruby-lang.org/issues/19077>. ## Examples ### Request loop ```ruby Thread.new do while request = queue.pop Fiber.new(storage: {id: SecureRandom.hex(16)}) do handle_request.call(request) end end end ``` OR ```ruby Thread.new do while request = queue.pop Fiber.current.storage = {id: SecureRandom.hex(16)} handle_request.call(request) end end ``` -- https://bugs.ruby-lang.org/

1 0