Issue #22239 has been updated by YO4 (Yoshinao Muramatsu). GH PR created. https://github.com/ruby/ruby/pull/18276 ---------------------------------------- Bug #22239: Multiple buffer clear errors in IO https://bugs.ruby-lang.org/issues/22239#change-118487 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- While testing my changes on the Windows platform to use an encoding converter for newline conversion for "r", I discovered a failure in test_reopen. This corresponds to case (3) below. I asked an LLM to investigate similar cases and related code that raised concerns, and was able to identify several issues, so I am reporting them here. I'll submit a PR later. I used Hy3 (High) via opencode GO to perform this survey and create the test code. (For transparency—this is not an advertisement.) case(1) In io.c, free_io_buffer only clears ptr and left off, len and capa as they were. Since clear_readconv releases the character buffer through it, READ_CHAR_PENDING() kept reporting pending characters after the buffer had been released. As a result, every operation that clears the code converter (IO#rewind, IO#seek, IO#pos=, IO#flush, IO#binmode, IO#tell, IO#set_encoding, ...) left the IO in a state where byte oriented reads raise IOError. Also IO#eof? answers false at the end of file and IO#sysseek refuses to work. ``` File.binwrite("t", "foo") open("t", "rt") do |f| f.ungetc(f.getc) f.rewind f.getbyte # => 'IO#getbyte': byte oriented read for character buffered IO (IOError) end ``` case(2) IO#reopen(io) only flushes the write buffer when the IO is writable, and never unreads the read buffer. The buffer happens to be dropped by the io_seek after dup2, but only when the other stream is readable and seekable, so a read-write IO keeps stale bytes otherwise. ``` File.binwrite("t", "0123456789") f = File.open("t", "r+b") f.getbyte IO.pipe do |r, w| w.write("ABC"); w.close f.reopen(r) f.read # => "123456789ABC" instead of "ABC" end ``` case(3) IO#reopen(name) discards the byte buffer but keeps the code converter and the character buffer, so characters left over from the previous stream leak into the reopened one. ``` File.binwrite("t", "foo") open(File::NULL, "rt") do |f| f.ungetc("a") f.reopen("t") f.read # => "afoo" instead of "foo" end ``` case(4) IO#reopen(io) relies on flush_before_seek to drop the buffers, but io_unread returns at once when rbuf is empty and therefore never reaches clear_codeconv. With characters pending in cbuf and nothing in rbuf they leak into the reopened stream. ``` File.write("t", "foo\n") open(File::NULL, "rt") do |f| f.ungetc("a") open("t", "rt") {|f2| f.reopen(f2)} f.gets # => "afoo\n" instead of "foo\n" end ``` case(5) Unlike IO#rewind, IO#seek and IO#pos= do not clear the character buffer. As io_unread returns at once when rbuf is empty, an ungotten character survives the repositioning while it does not in binary mode. ``` File.write("t", "foo\n") open("t", "rt") do |f| f.ungetc("a") f.seek(2, :SET) f.getc # => "a" instead of "o" end ``` -- https://bugs.ruby-lang.org/