"ioquatix (Samuel Williams) via ruby-core" <ruby-core@ml.ruby-lang.org> wrote:
``` irb(main):001:0> $stderr.sync => true irb(main):002:0> $stdout.sync => true ```
It looks like `$stdout` and `$stderr` are both buffered internally.
IO#sync==true means unbuffered.
I think Ruby should guarantee buffered writes will be atomic up to PIPE_BUF.
PIPE_BUF is only relevant for pipes/FIFOs (not regular files, sockets, etc..).
Unbuffered writes should have no atomicity guarantees.
Each write/writev syscall to regular files is atomic with O_APPEND. It's how multi-process servers can write to the same log file without interleaving lines. The limit is not PIPE_BUF, but (IIRC) roughly INT_MAX/SSIZE_MAX; It may be FS-dependent, but it's large enough to not matter on reasonable local FSes. Side notes: Also, any writev benchmarks you'd do should account for the common pattern of giant bodies prefixed with a small header. (e.g. HTTP/1.1 chunking, tnetstrings(*), etc). The 128 bytes you use in the benchmark is tiny and unrealistic. writev has high overhead with many small chunks (each iovec is 16 bytes), and most I/O size is far larger than 128 bytes. (*) https://tnetstrings.info/ I understand that writev is slow for the kernel, but (alloc + memcpy + GC/free) w/ giant strings is slow in userspace, too. Stuff like: io.write("#{buf.bytesize.to_s(16)}\r\n", buf, -"\r\n") When buf is a gigantic string (multiple KB or MB) is where I expect writev to avoid copies and garbage. Adding a buffer_offset (tentative name) arg to write_nonblock would make retrying non-blocking I/O much easier: n = 0 tot = ary.map(&:bytesize).sum while n < tot case w = io.write_nonblock(*ary, buffer_offset: n, exception: false) when :wait_readable, :wait_writable break # or schedule or whatever else n += w end end Perl's syswrite/read/sysread ops have the OFFSET arg for decades.