[ruby-core:120458] [Ruby master Misc#20995] exception escapes block given to IO.popen("-") in child process
Issue #20995 has been reported by martin.dorey@hds.com (Martin Dorey). ---------------------------------------- Misc #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Open ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by nobu (Nobuyoshi Nakada). Tracker changed from Misc to Bug Backport set to 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED It seems a bug. ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111244 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Open * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by martin.dorey@hds.com (Martin Dorey). Domo arigato, Nobu-san, for agreeing that the previous behavior wasn't right and for such a quick fix. I really like the idea of expressing how the code should behave with a unit test. I was surprised by a couple of aspects of your test. This is how I expected it to behave: ``` diff martind@stormy:~/download/ruby$ git diff ... diff --git a/test/ruby/test_process.rb b/test/ruby/test_process.rb index 9c1836dffb..781fe77b7d 100644 --- a/test/ruby/test_process.rb +++ b/test/ruby/test_process.rb @@ -939,9 +939,10 @@ def test_popen_fork_ensure STDERR.reopen(STDOUT) raise "fooo" else - assert_empty io.read + assert_include(io.read, "fooo") end end + assert_not_predicate($?, :success?) rescue RuntimeError abort "[Bug #20995] should not reach here" end martind@stormy:~/download/ruby$ ``` I have very little idea whether this is safe but, with the following change, the above test passes: ``` diff martind@stormy:~/download/ruby$ git diff diff --git a/io.c b/io.c index a78936488f..612c9caa40 100644 --- a/io.c +++ b/io.c @@ -8038,10 +8038,11 @@ popen_finish(VALUE port, VALUE klass) if (NIL_P(port)) { /* child */ if (rb_block_given_p()) { - rb_protect(rb_yield, Qnil, NULL); + int state; + rb_protect(rb_yield, Qnil, &state); rb_io_flush(rb_ractor_stdout()); rb_io_flush(rb_ractor_stderr()); - _exit(0); + _exit(ruby_cleanup(state)); } return Qnil; } ``` With that change, these ancestral behaviors: ``` martind@stormy:~/download/ruby$ ../ruby-3.3.4/ruby --disable-gems -we 'IO.popen("-") { |io| raise("a fuss") unless io; }; puts($?.inspect())' -e:1:in `block in <main>': a fuss (RuntimeError) from -e:1:in `popen' from -e:1:in `<main>' #<Process::Status: pid 120178 exit 1> martind@stormy:~/download/ruby$ ``` ... which I think have positive value, are preserved: ``` martind@stormy:~/download/ruby$ ./ruby --disable-gems -we 'IO.popen("-") { |io| raise("a fuss") unless io; }; puts($?.inspect())' -e:1:in 'block in <main>': a fuss (RuntimeError) from -e:1:in 'IO.popen' from -e:1:in '<main>' #<Process::Status: pid 120135 exit 1> martind@stormy:~/download/ruby$ ``` ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111247 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by martin.dorey@hds.com (Martin Dorey). Sorry, I meant to add that, if I revert io.c to how it is in git, my one-liner says: ``` martind@stormy:~/download/ruby$ ./ruby --disable-gems -we 'IO.popen("-") { |io| raise("a fuss") unless io; }; puts($?.inspect())' #<Process::Status: pid 126700 exit 0> martind@stormy:~/download/ruby$ ``` ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111248 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED, 3.4: REQUIRED ruby_3_2 commit:f150d67b7d389eb88e0cd13694d3529895d55579 merged revision(s) commit:8034e9c3d001ca3dff124ab42972684eac8af2ae. ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111440 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED, 3.4: REQUIRED ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by k0kubun (Takashi Kokubun). Backport changed from 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED, 3.4: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED ruby_3_3 commit:12a0807965624a0be37dc79371a69b5d787cc8d1 merged revision(s) commit:8034e9c3d001ca3dff124ab42972684eac8af2ae. ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111513 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
Issue #20995 has been updated by k0kubun (Takashi Kokubun). Backport changed from 3.1: REQUIRED, 3.2: DONE, 3.3: DONE, 3.4: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: DONE, 3.4: DONE ruby_3_4 commit:b65cea74295358265dfabc9e1f4d107b21e58e58 merged revision(s) commit:8034e9c3d001ca3dff124ab42972684eac8af2ae. ---------------------------------------- Bug #20995: exception escapes block given to IO.popen("-") in child process https://bugs.ruby-lang.org/issues/20995#change-111921 * Author: martin.dorey@hds.com (Martin Dorey) * Status: Closed * Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: DONE, 3.4: DONE ---------------------------------------- I was surprised by the "ensure" being reached in the child process here: ``` ruby martind@stormy:~/tmp/D161730$ cat repro.rb #!/usr/bin/ruby -w parent = Process.pid() [false, true].each() { |nauseous| $stderr.puts("#{nauseous ? "" : "not "}raising exception from child:") begin IO.popen("-") { |io| unless io if nauseous raise("childish fit") end end } ensure $stderr.puts("in finalization block from #{Process.pid() == parent ? "parent" : "child"}") $stderr.puts() end } martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby --version ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux] martind@stormy:~/tmp/D161730$ ~/download/ruby-3.3.4/ruby -w repro.rb not raising exception from child: in finalization block from parent raising exception from child: in finalization block from child repro.rb:11:in `block (2 levels) in <main>': childish fit (RuntimeError) from repro.rb:7:in `popen' from repro.rb:7:in `block in <main>' from repro.rb:3:in `each' from repro.rb:3:in `<main>' in finalization block from parent martind@stormy:~/tmp/D161730$ ``` In the first iteration of the loop, we don't see the surprise - the child process exits when the block is completed. Only when an exception is thrown, in the second iteration, does the control flow escape from the block. It does so even for the SystemExit exception raised by exit(0) but not, as we see above, from just leaving the block (in which case perhaps we _exit(0) at https://github.com/ruby/ruby/blob/master/io.c#L8044). My expectation was that the block would be executed in the child process much like the "main" part of a program, with the usual unhandled exception reporting, which we see demonstrated above, being invoked as soon as the exception propagated out the block, rather than first unwinding the callers. I think the behavior is unchanged since the oldest version I was conveniently able to test, which was: ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux] ... so I didn't file this report as a Bug in case users have (accidentally?) come to depend on the behavior. I didn't find any mention of this at eg https://ruby-doc.org/3.4.1/IO.html#method-c-popen but I didn't file this report as a Feature request for the documentation in case the documentation deliberately leaves open such possibilities. I didn't even find anyone else puzzling over this behavior on eg StackOverflow, but I felt that I should report it for the benefit of the next poor sap to run into it, especially in case that's Future Me. Could it be a case where a warning would be worth more than the trouble it causes? -- https://bugs.ruby-lang.org/
participants (4)
-
k0kubun (Takashi Kokubun) -
martin.dorey＠hds.com (Martin Dorey)
-
nagachika (Tomoyuki Chikanaga)
-
nobu (Nobuyoshi Nakada)