Issue #19276 has been reported by graywolf (Gray Wolf).
----------------------------------------
Misc #19276: It is not possible to reply to emails from rubymine
https://bugs.ruby-lang.org/issues/19276
* Author: graywolf (Gray Wolf)
* Status: Open
* Priority: Normal
----------------------------------------
It is no possible to reply to email from about issue update in order to add further comments to the tickets. Is that something that could be turned on? I think it would be user friendly to be able to react directly from mail client, without having to switch to the browser and navigate to the issue and comment there.
--
https://bugs.ruby-lang.org/
Issue #19189 has been reported by mistydemeo (Misty De Meo).
----------------------------------------
Bug #19189: Ruby 3.1.3/3.2.x can no longer find pkg-config if not present at buildtime
https://bugs.ruby-lang.org/issues/19189
* Author: mistydemeo (Misty De Meo)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Starting with Ruby 3.1.3, mkmf.rb calls to `pkg-config` will fail if `pkg-config` wasn't present on the system at the time it was built. Prior to that version, even if Ruby itself wasn't configured with `pkg-config`, mkmf.rb would be able to correctly pick it up from the `PATH`. This was introduced by https://github.com/ruby/ruby/commit/7f1ca666424849134990d022266bcd4d6636465f and its Ruby 3.1.3 equivalent, https://github.com/ruby/ruby/commit/3914581213e3aea138b02941b41e62c15461af27. I've confirmed that reverting this specific change fixes the problem.
This broke a Ruby deployment which is built in a minimal Docker environment with no `pkg-config`, but which is later copied into a system with `pkg-config` present.
--
https://bugs.ruby-lang.org/
Issue #19158 has been reported by deivid (David Rodríguez).
----------------------------------------
Bug #19158: Ruby 3.1.3 installs wrong gemspec for debug gem
https://bugs.ruby-lang.org/issues/19158
* Author: deivid (David Rodríguez)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [arm64-darwin22]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
A pristine installation of Ruby 3.1.3 shows an installed `debug-1.6.3.gemspec` file that claims that debug-1.6.3 has no dependencies. This is incorrect, and causes issues for Bundler as reported at https://github.com/rubygems/rubygems/issues/6082.
An issue workaround is to manually reinstall the gem with `gem install debug:1.6.3`. That will reinstall the gem, including a correct gemspec file.
This is the file diff before and after reinstalling the gem
```diff
--- old.gemspec 2022-11-29 13:54:36
+++ /Users/deivid/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/specifications/debug-1.6.3.gemspec 2022-11-29 13:55:25
@@ -24,4 +24,16 @@
s.summary = "Debugging functionality for Ruby".freeze
s.installed_by_version = "3.3.26" if s.respond_to? :installed_by_version
+
+ if s.respond_to? :specification_version then
+ s.specification_version = 4
+ end
+
+ if s.respond_to? :add_runtime_dependency then
+ s.add_runtime_dependency(%q<irb>.freeze, [">= 1.3.6"])
+ s.add_runtime_dependency(%q<reline>.freeze, [">= 0.3.1"])
+ else
+ s.add_dependency(%q<irb>.freeze, [">= 1.3.6"])
+ s.add_dependency(%q<reline>.freeze, [">= 0.3.1"])
+ end
end
```
In general, the files installed by ruby-core for bundled gems should be identical to the ones installed by explicit `gem install`.
This issue is also present in master and I'm guessing it was introduced by https://github.com/ruby/ruby/commit/a2c66f52f402cb58372e271226f3341065561e53 (backported at https://github.com/ruby/ruby/commit/2d26e45135af8b427d9ccc6d47082c21be8b9c74).
--
https://bugs.ruby-lang.org/
Issue #19153 has been reported by cpinto (Celso Pinto).
----------------------------------------
Bug #19153: Since 2.7.7 CGI::Cookie raises ArgumentError when cookie domains is prefixed with a dot
https://bugs.ruby-lang.org/issues/19153
* Author: cpinto (Celso Pinto)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [arm64-darwin22]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The rspec tests of our Rails app started failing with an ArgumentError after upgrading to 2.7.7. On inspection, the issue seems to be caused by CGI::Cookie.domain=:
```
def domain=(str)
if str and ((str = str.b).bytesize > 255 or !DOMAIN_VALUE_RE.match?(str))
raise ArgumentError, "invalid domain: #{str.dump}"
end
@domain = str
end
```
Setting a breakpoint:
0> str
=> ".example.com"
0> DOMAIN_VALUE_RE
=> /\A(?<label>(?!-)[-A-Za-z0-9]+(?<!-))(?:\.\g<label>)*\z/
0> DOMAIN_VALUE_RE.match?(str)
=> false
0> DOMAIN_VALUE_RE.match?('example.com')
=> true
--
https://bugs.ruby-lang.org/
Issue #19353 has been reported by shyouhei (Shyouhei Urabe).
----------------------------------------
Misc #19353: Drop gcc <= 6 and clang <= 9
https://bugs.ruby-lang.org/issues/19353
* Author: shyouhei (Shyouhei Urabe)
* Status: Open
* Priority: Normal
----------------------------------------
We test those old compilers using Ubuntu bionic. This OS is reaching its EOL.
Making them available would become harder.
I would like to drop supporting them. Any opinions?
--
https://bugs.ruby-lang.org/
Issue #19366 has been reported by zverok (Victor Shepelev).
----------------------------------------
Feature #19366: Rename/alias Refinedment#refined_class => #refined_module
https://bugs.ruby-lang.org/issues/19366
* Author: zverok (Victor Shepelev)
* Status: Open
* Priority: Normal
----------------------------------------
In #12737, `Refinement#refined_class` is introduced.
As "module" is more generic concept than "class", the name misleadingly implies that either this method doesn't returns refined _modules_, or modules can't be refined. This is obviously not true and trivially disproved:
```ruby
module Refs
refine Enumerable do
def foo = puts 'foo'
end
end
Refs.refinements[0].refined_class
#=> Enumerable. Which is, well, not a class.
# The refinement is usable, so it is not a mute concept:
using Refs
[1, 2, 3].foo # prints "foo" successfully
```
I believe we refer to "modules" when some feature applies to modules and classes.
Unless there is some deeper consideration for the current naming (I don't see justification in #12737, but I might miss something), the method should be renamed or aliased.
PS: Sorry for a huge amount of "nitpicky" issues after the version release. Typically, those concerns emerge while I am working on the [changelog](https://rubyreferences.github.io/rubychanges/), and I am usually trying to start the work at least a month before the release. Unfortunately, due to you-know-what, I was unable to do it in time this year.
After the 3.2 changelog, I plan to switch to the "master changelog" approach (following the development with the "current unreleased changes" document) this year. Unless I'll be unable to due to reasons not in my control.
--
https://bugs.ruby-lang.org/
Issue #19322 has been reported by kjtsanaktsidis (KJ Tsanaktsidis).
----------------------------------------
Feature #19322: Support spawning "private" child processes
https://bugs.ruby-lang.org/issues/19322
* Author: kjtsanaktsidis (KJ Tsanaktsidis)
* Status: Open
* Priority: Normal
----------------------------------------
## Background
The traditional Unix process APIs (`fork` etc) are poorly isolated. If a library spawns a child process, this is not transparent to the program using the library. Any signal handler for `SIGCHLD` in the program will be called when the spawned process exits, and even worse, if the parent calls `Process.waitpid2(-1)`, it will consume the returned status code, stealing it from the library!
Unfortunately, the practice of responding to `SIGCHLD` by calling `waitpid2(-1)` in a loop is a pretty common unixism. For example, Unicorn does it [here](https://yhbt.net/unicorn.git/tree/lib/unicorn/http_server.rb#n401). In short, there is no reliable way for a gem to spawn a child process in a way that can’t (unintentionally) be interfered with by other parts of the program.
## Existing solutions in OS’s
Several operating systems provide an improved API for spawning child processes which are fully isolated; that is, they do not generate `SIGCHLD` signals in the program, and are invisible to calls to `waitpid(2)`
* On Linux, such invisible processes can be made by calling `clone(2)` with a zero value in the low byte of `flags`. If the CLONE_PIDFD flag is also provided, then a file descriptor representing the process is also returned; this can be used to wait for and signal the process in a race-free way.
* On FreeBSD, the `pdfork(2)` syscall makes a process that does not signal SIGCHLD and is ignored by `waitpid(2)` calls that do not explicitly specify the pid (i.e. it is ignored when -1 is passed). It also returns a file descriptor representing the process.
Both of these APIs center around the idea of a process file descriptor. Rather than managing a child process using the old process-global wait/signal mechanisms, they return a file descriptor representing the process. Such a file descriptor can uniquely identify the spawned process, be used to wait on the process and get the status, send signals, and even participate in `poll(2)`. They also protect against pid-reuse race conditions; after a process has terminated and been reaped, the pidfd becomes invalid, and can’t randomly begin to refer to a different process.
## Proposed Ruby APIs
I think we should make a new API `Process.spawn_handle`, which accepts all the same parameters as `Process.spawn`. However, it does _not_ return a pid like `Process.spawn`, but rather a new type `Process::Handle`.
`Process::Handle` would identify a single spawned process, using a durable OS-supplied handle not subject to re-use risks (e.g. a pidfd). It would provide the following methods:
* `#pid` - get the pid that the handle is for.
* `#send_signal(signal)` - send a signal to the wrapped process (where "signal" is a symbol, string, or number with the same meaning as in `Process.kill`.
* `#wait` - blocks waiting for the program to exit, and then returns a `Process::Status` object representing e.g. the exit code. Like calling `waitpid`.
* `#wait_nonblock` - Returns a `Process::Status` object for the child process. If the child has not exited, it will be a status object for which `#exited?` is false. Does not block. Like calling `waitpid(WNOHANG)`.
Finally, the `Open3` family of methods would be extended to accept `handle:` as an additional keyword argument. When set to true, `Process.spawn_handle` will be used to start the child, and `Process::Handle` objects will be returned in place of pids.
Modifying backticks, `Kernel#system` and other process-creating methods which don't return pids to use `spawn_handle` internally would also be possible, but out of scope for an initial implementation of this ticket.
## OS compatibility
For this API to be useful to gem authors, it has to be widely available on the systems that they and their users care about. As discussed, the `clone(2)` syscall and `CLONE_PIDFD` flag can be used on Linux 5.2+ to implement `Process::Handle`. FreeBSD has `pdfork(2)` since v9.
I haven’t investigated Windows _deeply_, but I think Windows doesn’t really have the notion of process-global `waitpid` or `SIGCHLD` anyway. The `CreateProcess` function returns a `PROCESS_INFORMATION` struct, which returns a `HANDLE` for the child process, which seems analogous to a process FD.
However this does leave a large chunk of operating systems which don’t have this functionality built-in. Off the top of my head:
* MacOS, NetBSD, and OpenBSD have nothing. I stared pretty hard at the Darwin XNU source and couldn’t find a race-free way to convince it not to dispatch `SIGCHLD` for a particular process or stop it from being reaped by process-wide `wait4` calls.
* Linux < 5.2 is in some probably-pretty-widely-deployed-still distros - it’s the release kernel in Ubuntu 18.04 for example.
I have two ideas for how the semantics of `Process::Handle` could be emulated on such systems. However I recognise that they aren’t amazing so if anybody has some better ideas I would dearly love to hear them.
### Long-lived proxy
The first time `Process.spawn_handle` is used, we would fork/exec a long-lived “fork-helper” program. This could be a separate helper binary we compile with the build system, or perhaps just a re-invocation of the ruby interpreter with something like `ruby -e “Process._fork_helper”`. There would be a unix socketpair shared between the parent process & the helper.
Instead of actually forking when we’re calling `Process.spawn_handle`, we would instead send a message on this socket asking the helper to, _itself_, fork & exec the specified program. Any file descriptors etc needed in the child could also be sent over this socket. All of the `Process::Handle` methods would be proxies which called through to the helper binary.
This way, the ruby process is never actually the parent of the spawned child, so we would never get any SIGCHLD etc from it. The fork-helper program might generate a SIGCHLD, but it should persist until the ruby process exits; we would only generate a SIGCHLD signal if it crashed abnormally.
### Forward misdirected waits
With this approach, `Process.spawn_handle` would just `fork(2)`/`exec(2)` or `posix_spawn(2)` processes like normal. We would however keep a table of pids -> `Process::Handle` instances.
When Ruby’s C-level SIGCHLD handler is invoked, we would inspect that table and see if the pid has an associated `Process::Handle`. If so, we would skip calling any registered Ruby SIGCHLD handler; instead, we would call `waitpid` ourselves, update the status info on the handle object, and unblock anybody waiting on `Process::Handle#wait`.
Likewise, in the C-side implementation of `Process.waitpid2` etc, we would check the returned pid from the syscall against the handle table. If it matched, we would perform the same work as in the SIGCHLD case, and then re-start the original call to `Process.waitpid2`.
This approach keeps the process tree correct and involves less silly proxying, but it won’t hide the process from any callers to the raw `waitpid` library functions in C extensions. Doing that seems like a silly idea anyway though, so maybe that’s OK?
## Motivation
My use-case for this is that I’m working on a perf-based profiling tool for Ruby. To get around some Linux capability issues, I want my profiler gem (or CRuby patch, whatever it winds up being!) to fork a privileged helper binary to do some eBPF twiddling. But, if you’re profiling e.g. a Unicorn master process, the result of that binary exiting might be caught by Unicorn itself, rather than my (gem | interpreter feature).
In my case, I'm so deep in linux specific stuff that just calling `clone(2)` from my extension is probably fine, but I had enough of a look at this process management stuff I thought it would be worth asking the question if this might be useful to other, more normal, gems.
--
https://bugs.ruby-lang.org/
Issue #19179 has been reported by kjtsanaktsidis (KJ Tsanaktsidis).
----------------------------------------
Feature #19179: Support parsing SCM_CRED(ENTIALS) messages from ancillary messages
https://bugs.ruby-lang.org/issues/19179
* Author: kjtsanaktsidis (KJ Tsanaktsidis)
* Status: Open
* Priority: Normal
----------------------------------------
## Background
Linux and FreeBSD support processes at either end of a unix socket identifying themselves to the other party by passing an ancillary message of type `SCM_CREDENTIALS` (Linux) or `SCM_CREDS` (FreeBSD). The socket library contains code to parse these ancillary messages, but the only way this is exposed into Ruby code is by the `Socket::AncillaryData#inspect` method - e.g.
```
# On Linux
irb(main):002:0> s1, s2 = UNIXSocket.pair
=> [#<UNIXSocket:fd 5>, #<UNIXSocket:fd 6>]
irb(main):004:0> s2.setsockopt Socket::SOL_SOCKET, Socket::SO_PASSCRED, 1
=> 0
# struct ucred on Linux is (32-bit signed) pid_t, followed by (32-bit unsigned) uid_t, followed by
# (32-bit unsigned) gid_t
irb(main):008:0> ancdata = [Process.pid, Process.uid, Process.gid].pack("lLL")
=> "\x1ET\x05\x00\xE8\x03\x00\x00\xE8\x03\x00\x00"
# Socket::AncillaryData knows how to unmarshal the data into struct ucred
irb(main):010:0> ancmsg = Socket::AncillaryData.new(Socket::AF_UNIX, Socket::SOL_SOCKET, Socket::SCM_CRE
DENTIALS, ancdata)
=> #<Socket::AncillaryData: UNIX SOCKET CREDENTIALS pid=349214 uid=1000 gid=1000 (ucred)>
irb(main):011:0> s1.sendmsg "hi", 0, nil, ancmsg
=> 2
# ancillary message can be passed through
irb(main):012:0> _, _, _, recvanc = s2.recvmsg; recvanc
=> #<Socket::AncillaryData: UNIX SOCKET CREDENTIALS pid=349214 uid=1000 gid=1000 (ucred)>
```
On Linux, at least, a suitably privileged process can send any value through for the pid, uid, or gid, but the kernel will reject attempts by unprivileged processes to forge credentials in this way. So SCM_CREDENTIALS messages can be useful for certain systems programming tasks.
A somewhat wider array of operating systems support querying the identity of the other side of a socket using a socket option, variously `SO_PEERCRED` (Linux, OpenBSD) or `LOCAL_PEERCRED` (FreeBSD, MacOS). Again, the socket library is able to unmarshal the socket data into the correct structure on these various systems, but it's only exposed to Ruby code via `#inspect` - e.g.
```
irb(main):002:0> s1, s2 = UNIXSocket.pair
=> [#<UNIXSocket:fd 5>, #<UNIXSocket:fd 6>]
irb(main):014:0> s1.getsockopt Socket::SOL_SOCKET, Socket::SO_PEERCRED
=> #<Socket::Option: UNIX SOCKET PEERCRED pid=349214 euid=1000 egid=1000 (ucred)>
```
Ruby _does_ however support e.g. `BasicSocket#getpeereid`, which could use `SO_PEERCRED` etc under the hood - so getting the uid/gid data is not totally impossible. I believe getting the pid is though.
```
irb(main):016:0> s1.getpeereid
=> [1000, 1000]
```
## My proposal
I believe we should implement the following:
* `Socket::Credentials` - this would be a struct which can contain all the various platform-specific pieces of credential info that can be transferred over a socket, such as uid, gid, pid, euid, egid, and group list.
* `Socket::AncillaryData#credentials` - this would parse an `SCM_CREDS` or `SCM_CREDENTIALS` ancillary data message into the appropriate platform-specific struct, and return a `Socket::Credentials` instance containing that data. This would be analogous to `Socket::AncillaryData#int`; a method for interpreting the ancillary data in a certain form.
* `Socket::Option#credentials` - This would parse a `SO_PEERCRED` or `LOCAL_PEERCRED` socket option response into the appropriate platform-specific struct, and return a `Socket::Credentials` instance containing that data. Again, this would be analogous to `Socket::Option#int`.
The existing `struct ucred`/`struct xucred`/`struct sockpeercred`/`struct cmsgcred` parsing code (used only for `#inspect` output) would be moved into `Socket::Credentials`, and `Socket::AncillaryData#inspect`/`Socket::Option#inspect` would be implemented in terms of `Socket::Credentials`.
This would nicely wrap a lot of parsing work that Ruby is already doing, into an API which allows Ruby code to take advantage of it.
## Use-cases
My motivation for designing this feature came about whilst I was experimenting with some ideas for Ruby profilers. I wanted to allow a CLI tool to ask a Ruby process to start profiling itself by sending a message on a unix socket. Alongside the message, it would send a file descriptor which was the result of calling `perf_event_open(2)` in the CLI tool. In order to call `perf_event_open(2)`, the CLI tool would need to be privileged. I also wanted the Ruby process to authenticate the request and make sure it came from the same UID that it was running as. Calling `BasicSocket#getpeereuid` would reveal the remote process to be running as UID 0, (or perhaps even some other UID, with sufficient ambient capabilities to call `perf_event_open`). Instead, I decided to make the CLI tool send a `SCM_CREDENTIALS` message containing the uid of the process to be profiled; that way, the kernel does all the policy checking on whether or not this is actually allowed, and the Ruby process receiving the message just needs to check if `uid == Process.getuid`.
I think, on Linux at least, that this feature will be useful for any kind of communication/authentication scheme between privileged & unprivileged processes over unix sockets.
## My implementation
I have an implementation of roughly this in this pull request: https://github.com/ruby/ruby/pull/6822
Thanks!
--
https://bugs.ruby-lang.org/
Issue #19306 has been reported by ioquatix (Samuel Williams).
----------------------------------------
Feature #19306: Expand zlib interface
https://bugs.ruby-lang.org/issues/19306
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
Introduce some extra methods like `window_size`, `level` (compression/decompression), enhance `inspect` and various other quality of life improvements.
--
https://bugs.ruby-lang.org/