Issue #21765 has been updated by YO4 (Yoshinao Muramatsu). The current io.c code for windows is intended to avoid performance limitations of the encoding converter. Therefore, performance is important. Here are some benchmarks before and after the patch. (some empty lines removed manually) ``` ** generate fixture **

c:\ruby34-x64\bin\ruby -e "open('long.txt', 'wb') { |f| 8.times { f.write 'x'*1024*1024 } }" c:\ruby34-x64\bin\ruby -e "open('medium.txt', 'w') { |f| 65536.times { f.puts 'x'*80 } }" c:\ruby34-x64\bin\ruby -e "open('dense.txt', 'w') { |f| (65536*8).times { f.puts '' } }"

** IO#read with no line breaks **

master\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 128.times { f.read; f.rewind }; puts Time.now - t }" 1.5247651 patched\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 128.times { f.read; f.rewind }; puts Time.now - t }" 1.260006

** IO#read with moderate line breaks **

master\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 128.times { f.read; f.rewind }; puts Time.now - t }" 0.8651427 patched\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 128.times { f.read; f.rewind }; puts Time.now - t }" 1.0337131

** IO#read with dense line breaks **

master\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 1024.times { f.read; f.rewind }; puts Time.now - t }" 1.2129726 patched\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 1024.times { f.read; f.rewind }; puts Time.now - t }" 1.3608866

** IO#readlines with no line breaks **

master\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { f.readlines; f.rewind }; puts Time.now - t }" 0.9637844 patched\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { f.readlines; f.rewind }; puts Time.now - t }" 0.9633248

** IO#readlines with moderate line breaks **

master\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 64.times { f.readlines; f.rewind }; puts Time.now - t }" 1.2625658 patched\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 64.times { f.readlines; f.rewind }; puts Time.now - t }" 1.2737932

** IO#readlines with dense line breaks **

master\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 32.times { f.readlines; f.rewind }; puts Time.now - t }" 1.8690925 patched\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 32.times { f.readlines; f.rewind }; puts Time.now - t }" 1.7237312

** IO#gets with no line breaks **

master\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { while f.gets do end; f.rewind }; puts Time.now - t }" 0.9798694 patched\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { while f.gets do end; f.rewind }; puts Time.now - t }" 1.0012916

** IO#gets with moderate line breaks **

master\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 64.times { while f.gets do end; f.rewind }; puts Time.now - t }" 1.3087455 patched\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 64.times { while f.gets do end; f.rewind }; puts Time.now - t }" 1.3204693

** IO#gets with dense line breaks **

master\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 16.times { while f.gets do end; f.rewind }; puts Time.now - t }" 1.2261222 patched\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 16.times { while f.gets do end; f.rewind }; puts Time.now - t }" 1.1575592

** IO#getc with no line breaks **

master\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { while f.getc do end; f.rewind }; puts Time.now - t }" 0.8297943 patched\ucrt64\bin\ruby -e "open('long.txt', 'r') { |f| t = Time.now; 1.times { while f.getc do end; f.rewind }; puts Time.now - t }" 0.767664

** IO#getc with moderate line breaks **

master\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 2.times { while f.getc do end; f.rewind }; puts Time.now - t }" 1.0667623 patched\ucrt64\bin\ruby -e "open('medium.txt', 'r') { |f| t = Time.now; 2.times { while f.getc do end; f.rewind }; puts Time.now - t }" 0.9615283

** IO#getc with dense line breaks **

master\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 16.times { while f.getc do end; f.rewind }; puts Time.now - t }" 0.9435967 patched\ucrt64\bin\ruby -e "open('dense.txt', 'r') { |f| t = Time.now; 16.times { while f.getc do end; f.rewind }; puts Time.now - t }" 0.8731051

** ruby versions **

master\ucrt64\bin\ruby -v ruby 4.0.0dev (2025-12-04T09:08:23Z master 8099e9d2d1) +PRISM [x64-mingw-ucrt] patched\ucrt64\bin\ruby -v ruby 4.0.0dev (2025-12-04T12:54:48Z remove_crt_read a72e32e5e4) +PRISM [x64-mingw-ucrt]

The variation in IO#read results after the patch is due to the characteristics of memchr.
For other results, it is thought to be a balance between increased overhead and the reduction in interlocks used internally by set_mode().

----------------------------------------
Bug #21765: stop using the C runtime _read() on Windows
https://bugs.ruby-lang.org/issues/21765#change-115489

* Author: YO4 (Yoshinao Muramatsu)
* Status: Open
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
When creating an IO instance in Windows, the default data mode is text mode.
In reality, the IO encoding conversion mechanism is not used when encoding conversion is not performed. Instead, the CRLF conversion provided by the C runtime's `_read()` is used.
This is explicitly for speed.
https://bugs.ruby-lang.org/issues/6401#note-4

As a trade-off, `SET_BINARY_MODE(fptr)` and `SET_BINARY_MODE_WITH_SEEK_CUR(fptr)` are used in various places within io.c, altering the state of the file descriptor.
This made the flow of operations difficult to understand and changes hard to implement, especially for developers on other platforms.

Additionally, the issues I recently reported were discovered while verifying the impact of modifying the CRLF conversion to utilize the encoding conversion mechanism.
#21691	On Windows some of binary read functions of IO are not functional
#21687	IO＃pos goes wrong after EOF character(ctrl-z) met
#21634	Combining read(1) with eof? causes dropout of results unexpectedly on Windows.

These issues arise because data read into the rbuf does not match the stream due to newline conversion, or because the buffer end and file position do not align when CTRLZ is detected.
As a fix for Bug #21687, I created PR #15216. However, this relies on the internal behavior of the C runtime's `_read()` function, and it seems there is no way to avoid this dependency.

**I propose removing the use of C runtime _read().**

Reason for Proposal
- The mismatch between rbuf and stream contents complicates io_unread() and makes maintenance difficult.
- Changing the O_BINARY/O_TEXT state of the file descriptor in various places hinders understanding of the behavior and makes modifications difficult.

Two methods to remove C runtime _read() while maintaining current behavior
1. Interpret CRLF and CTRLZ when reading rbuf within io.c.
1. Interpret CRLF and CTRLZ within the encoding conversion framework.

My initial idea was to implement the second, using encoding conversion.
However, this internally changes the read operation from rbuf to cbuf, resulting in a change to the behavior of ungetc.
The proposal in Bug #21682 attempted to generalize this change to minimize its impact.
https://bugs.ruby-lang.org/issues/21682

**This issue proposes the first method, crlf conversion during rbuf read.**

Problems caused by inconsistencies between the rbuf and stream contents are avoided, and io_unread() becomes the same as on other platforms.

Compared to implementing it as an encoding conversion, the advantage is that there is no change in behavior.
On the other hand, since each read method in io.c requires individual handling, using encoding conversion results in more localized changes.




-- 
https://bugs.ruby-lang.org/

Issue #21765 has been updated by YO4 (Yoshinao Muramatsu). I now consider [GH-PR#15408](https://github.com/ruby/ruby/pull/15408) to be a too invasive change. If we can make the encoding converter for CRLF conversion fast enough for practical use, the code that needs to be changed will be sufficiently isolated. However, this means that on Windows, `open(name, "r")` will return a `#<IO>` with encoding conversion enabled, which introduces incompatibility with the behavior of `ungetc`. Regarding this incompatibility, since a test recently added to ruby/spec was related to it, I raised https://github.com/ruby/spec/issues/1387. Regardless, whether to accept the incompatibility and proceed with the change will be discussed and decided within CRuby. Would it be acceptable for `open(name, "r")` on Windows to return a `#<IO>` with encoding conversion enabled? For reference, using `ruby -Eiso-8859-1:utf-8` will result in this incompatibility on other platforms as well. We are already facing this incompatibility. ---------------------------------------- Bug #21765: stop using the C runtime _read() on Windows https://bugs.ruby-lang.org/issues/21765#change-118631 * Author: YO4 (Yoshinao Muramatsu) * Status: Assigned * Assignee: windows * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- When creating an IO instance in Windows, the default data mode is text mode. In reality, the IO encoding conversion mechanism is not used when encoding conversion is not performed. Instead, the CRLF conversion provided by the C runtime's `_read()` is used. This is explicitly for speed. https://bugs.ruby-lang.org/issues/6401#note-4 As a trade-off, `SET_BINARY_MODE(fptr)` and `SET_BINARY_MODE_WITH_SEEK_CUR(fptr)` are used in various places within io.c, altering the state of the file descriptor. This made the flow of operations difficult to understand and changes hard to implement, especially for developers on other platforms. Additionally, the issues I recently reported were discovered while verifying the impact of modifying the CRLF conversion to utilize the encoding conversion mechanism. #21691 On Windows some of binary read functions of IO are not functional #21687 IO＃pos goes wrong after EOF character(ctrl-z) met #21634 Combining read(1) with eof? causes dropout of results unexpectedly on Windows. These issues arise because data read into the rbuf does not match the stream due to newline conversion, or because the buffer end and file position do not align when CTRLZ is detected. As a fix for Bug #21687, I created PR #15216. However, this relies on the internal behavior of the C runtime's `_read()` function, and it seems there is no way to avoid this dependency. **I propose removing the use of C runtime _read().** Reason for Proposal - The mismatch between rbuf and stream contents complicates io_unread() and makes maintenance difficult. - Changing the O_BINARY/O_TEXT state of the file descriptor in various places hinders understanding of the behavior and makes modifications difficult. Two methods to remove C runtime _read() while maintaining current behavior 1. Interpret CRLF and CTRLZ when reading rbuf within io.c. 1. Interpret CRLF and CTRLZ within the encoding conversion framework. My initial idea was to implement the second, using encoding conversion. However, this internally changes the read operation from rbuf to cbuf, resulting in a change to the behavior of ungetc. The proposal in Bug #21682 attempted to generalize this change to minimize its impact. https://bugs.ruby-lang.org/issues/21682 **This issue proposes the first method, crlf conversion during rbuf read.** Problems caused by inconsistencies between the rbuf and stream contents are avoided, and io_unread() becomes the same as on other platforms. Compared to implementing it as an encoding conversion, the advantage is that there is no change in behavior. On the other hand, since each read method in io.c requires individual handling, using encoding conversion results in more localized changes. -- https://bugs.ruby-lang.org/