- ruby-dev - ml.ruby-lang.org

[ruby-dev:52247] [Ruby Feature#22405] Run-Length Methods for String Bit Operations
by hasumikin (hitoshi hasumi) 07 Oct '26

07 Oct '26

Issue #22405 has been reported by hasumikin (hitoshi hasumi). ---------------------------------------- Feature #22405: Run-Length Methods for String Bit Operations https://bugs.ruby-lang.org/issues/22405 * Author: hasumikin (hitoshi hasumi) * Status: Open ---------------------------------------- PR: (WIP) ### What this adds This ticket adds the run-length methods of the String bit operation API. A run is a maximal sequence of consecutive bits with the same value. The methods build on the region arguments of #22279 and on the iterator conventions of #22399: ``` String#bit_run_length(bit, offset, lsb_first: true) -> Integer | nil String#each_bit_run(lsb_first: true) {|bit, offset, length| ... } -> self String#each_bit_run(offset, length, lsb_first: true) {|bit, offset, length| ... } -> self String#each_bit_run(range, lsb_first: true) {|bit, offset, length| ... } -> self String#each_bit_run(lsb_first: true) -> Enumerator String#each_bit_run(offset, length, lsb_first: true) -> Enumerator String#each_bit_run(range, lsb_first: true) -> Enumerator String#bit_runs(lsb_first: true) -> Array String#bit_runs(offset, length, lsb_first: true) -> Array String#bit_runs(range, lsb_first: true) -> Array String#bit_runs(lsb_first: true) {|bit, offset, length| ... } -> self String#bit_runs(offset, length, lsb_first: true) {|bit, offset, length| ... } -> self String#bit_runs(range, lsb_first: true) {|bit, offset, length| ... } -> self ``` `each_bit_run` and `bit_runs` visit every run of the region as `[bit, offset, length]`. The runs come in increasing order of offset. `bit_run_length` returns the length of the run of `bit` that begins at `offset`. ```ruby data = "\x0F\x01".b data.bit_runs # => [[1, 0, 4], [0, 4, 4], [1, 8, 1], [0, 9, 7]] data.bit_runs(8..) # => [[1, 8, 1], [0, 9, 7]] data.bit_run_length(1, 0) # => 4 data.bit_run_length(0, 0) # => 0 data.bit_run_length(1, 16) # => nil data.each_bit_run.size # => 4 ``` The pairing follows `each_bit` / `bits`. The `each_*` form returns `self` with a block. Without a block, it returns an Enumerator. The Array form returns an Array. With a block, it behaves like the `each_*` form. ### Use cases #### Bitstream parsing (MSB-first) - **Rice codes for sensor logs.** A battery device logs the differences between consecutive samples. Small differences are frequent, so a Rice code stores each value as a unary quotient followed by `k` remainder bits. The unary part is a run of set bits, so `bit_run_length(1, pos, lsb_first: false)` reads it in one call. The region form of `bits` then reads the remainder from the position where the run ended. ```ruby # Rice code, MSB-first. A value n is stored as: # q = n >> k set bits, one cleared bit, then the k low bits of n # bs is the bitstream (a binary String) and pos is a bit offset in it. # Returns the value and the bit offset of the next code. def read_rice(bs, pos, k) q = bs.bit_run_length(1, pos, lsb_first: false) # count the set bits pos += q + 1 # skip them and the cleared bit # Read k bits and build an Integer from them, MSB first r = bs.bits(pos, k, lsb_first: false).inject(0) {|acc, b| acc * 2 + b } [(q << k) | r, pos + k] end # "\x79\x14" = 0111 1001 0001 0100, decoded with k = 2: # pos unary q remainder value # 0 0 0 11 0 << 2 | 3 = 3 # 3 110 2 01 2 << 2 | 1 = 9 # 8 0 0 00 0 << 2 | 0 = 0 # 11 10 1 10 1 << 2 | 2 = 6 # (bit 15 is padding) bs = "\x79\x14".b pos = 0 values = [] 4.times do value, pos = read_rice(bs, pos, 2) # pass the returned offset to the next call values << value end values # => [3, 9, 0, 6] pos # => 15 ``` #### 1 bpp frame buffers - **E-paper partial refresh.** An e-paper panel refreshes a small window much faster than the full screen, and each refresh costs power. A driver keeps the previous frame and sends only the pixels that changed. `bitwise_xor` of the previous row and the new row marks the changed pixels. `bit_runs` of the result gives the changed spans. Most panel controllers accept a window only at byte boundaries. Thus the driver widens each span to a multiple of 8 and refreshes one window per span. The same pattern applies to a monochrome LCD with a page or row buffer. ```ruby # 32 pixels, MSB-first (the high bit of each byte is the left pixel), 1 = black old_row = "\xFF\x00\xF0\x0F".b new_row = "\xFF\x3C\xF0\x00".b # XOR sets a bit only where the pixel changed: # pixel: 0 8 16 24 # old_row: 1111 1111 0000 0000 1111 0000 0000 1111 # new_row: 1111 1111 0011 1100 1111 0000 0000 0000 # changed: 0000 0000 0011 1100 0000 0000 0000 1111 # ^^^^ ^^^^ # 10..13 28..31 changed = old_row.bitwise_xor(new_row) # bit_runs gives [bit, offset, length] for each run: # [[0, 0, 10], [1, 10, 4], [0, 14, 14], [1, 28, 4]] # Keep the runs of 1 (the changed spans) and convert them to Ranges. spans = changed.bit_runs(lsb_first: false).select {|bit, _, _| bit == 1 } .map {|_, x, width| x...(x + width) } spans # => [10...14, 28...32] # Widen each span to byte boundaries for the panel controller: # round the begin down and the end up to a multiple of 8. # 10...14 -> (10 / 8 * 8)...((14 + 7) / 8 * 8) = 8...16 # 28...32 -> (28 / 8 * 8)...((32 + 7) / 8 * 8) = 24...32 spans.map {|r| (r.begin / 8 * 8)...((r.end + 7) / 8 * 8) } # => [8...16, 24...32] ``` #### Columnar data (LSB-first) - **Bulk copies over valid ranges.** An Apache Arrow validity bitmap marks the null elements. Where the valid elements form long runs, a kernel copies each run as one slice and does not test every element. `bit_runs(0, length)` gives the runs within the column length. `bit_run_length(1, i)` tells how many valid elements start at `i`, for a loop that advances by run. ```ruby validity = "\xFC\x0F".b # 12 elements, elements 0 and 1 are null validity.bit_runs(0, 12).select {|bit, _, _| bit == 1 } .map {|_, off, len| off...(off + len) } # => [2...12] ``` #### Microcontrollers - **Block allocators.** A memory allocator or a page map keeps a bitmap of the used blocks. A first-fit allocation for `n` blocks is the first run of cleared bits with a sufficient length. Such an allocator is written in Ruby in two settings. One is a program assuming Spinel compiles AOT to native code, where a bitmap walk in Ruby becomes a plain loop. The other is a microcontroller program that manages a pool of slots. In both cases `each_bit_run` walks the bitmap and allocates nothing. ```ruby bitmap = "\xFF\x0F\xC0\x00".b # 32 blocks, 1 = in use def first_fit(bitmap, n) bitmap.each_bit_run do |bit, off, len| return off if bit.zero? && len >= n end nil end first_fit(bitmap, 8) # => 12 first_fit(bitmap, 20) # => nil ``` - **Pulse widths from a sampled input.** A program samples a GPIO pin at a fixed rate into a bitmap. This turns a signal into runs. `bit_runs` converts the capture into level and width pairs. These pairs are the input of an IR remote decoder (NEC and similar protocols distinguish bits by pulse width) or of a logic analyzer view. ```ruby capture = "\x01\x00\xFF\x0F".b # one sample per bit, LSB-first capture.bit_runs.map {|level, _, width| [level, width] } # => [[1, 1], [0, 15], [1, 12], [0, 4]] ``` ### API Contracts The three methods follow the rules of the earlier tickets. `each_bit_run` and `bit_runs` take the region arguments of #22279 and share every contract of `each_bit` in #22399. In particular, reads clamp, offsets are absolute, errors are raised without a block, the Enumerator validates again on use, and frozen receivers work. `bit_run_length` takes a lone `offset` like `bit_get` in #22118 and a `bit` argument like `each_bit_offset` in #22399. Encoding, `lsb_first:` and the out-of-range errors are as in #22118 and #22279. The contracts below are the ones specific to runs. 1. **Runs are cut at the region boundaries.** The first run starts at the region start, even if the same bit continues before it. The last run ends at the region end. 2. **Each run is one Array** `[bit, offset, length]`, with `bit` as `0` or `1`. A block with three parameters receives the three values. A block with one parameter, or a lambda, receives the Array, as with `Hash#each`. `to_a` and `bit_runs` collect the Arrays. 3. **`bit_run_length` returns `nil`, `0` or a count.** It returns `nil` when `offset` is at or beyond the end of the string. It returns `0` when the bit at `offset` differs from `bit`. Otherwise it counts up to the first differing bit or the end of the string. 4. **Enumerator#size** is the number of runs in the region, computed on demand from the current contents of the receiver. 5. **The block can modify the receiver.** The iteration reads each run from the current contents of the string when it visits the run. It clamps the length of the run to the current end and stops there. A longer receiver does not extend the iteration beyond the original region. ### Performance The script below gives the times. Each time is the median of 5 calls on a 1 MiB buffer. The machine is an AMD Ryzen 5 5600X with Ruby 4.1.0dev (master at 2db3072cfb) and this patch, gcc 13.3.0 `-O3`, YJIT off. {{collapse(Benchmark script) ```ruby few = ("\xFF".b * 4096 + "\x00".b * 4096) * 128 # 1 MiB, 256 runs many = "\xAA".b * (1 << 20) # 1 MiB, 8 Mi runs # The alternatives build the same [bit, offset, length] Arrays as bit_runs. def by_chunk_while(s) off = 0 s.bits.chunk_while {|a, b| a == b }.map {|c| [c[0], off, c.size].tap { off += c.size } } end def by_scan(s) off = 0 s.unpack1("b*").scan(/0+|1+/).map {|m| [m.ord - 48, off, m.size].tap { off += m.size } } end def median(n = 5) Array.new(n) { t = Process.clock_gettime(Process::CLOCK_MONOTONIC) yield Process.clock_gettime(Process::CLOCK_MONOTONIC) - t }.sort[n / 2] end { few:, many: }.each do |name, s| raise unless by_chunk_while(s) == s.bit_runs && by_scan(s) == s.bit_runs printf "%-4s chunk_while %.4f\n", name, median { by_chunk_while(s) } printf "%-4s scan %.4f\n", name, median { by_scan(s) } printf "%-4s bit_runs %.4f\n", name, median { s.bit_runs } printf "%-4s size %.4f\n", name, median { s.each_bit_run.size } end printf "index %.6f\n", median { few.unpack1("b*").index("0") } printf "bit_run_length %.6f\n", median { few.bit_run_length(1, 0) } ``` }} | | Few runs | Many runs | |--------------------------|---------:|----------:| | `bits` + `chunk_while` | 700 ms | 2,210 ms | | `unpack1("b*")` + `scan` | 36 ms | 2,440 ms | | `bit_runs` | 1.1 ms | 260 ms | | `each_bit_run.size` | 1.1 ms | 26 ms | | | Few runs | |----------------------------|---------:| | `unpack1("b*").index("0")` | 8.4 ms | | `bit_run_length(1, 0)` | 4 us | The methods skip a byte of all `0` or all `1` bits in one step. In the many-runs case, the Array for each run bounds the time. `each_bit_run.size` makes no Array, so it is 10 times faster there. -- https://bugs.ruby-lang.org/

2 1

[ruby-dev:52246] [Ruby Feature#22399] Iterator Methods for String Bit Operations
by hasumikin (hitoshi hasumi) 07 Oct '26

07 Oct '26

Issue #22399 has been reported by hasumikin (hitoshi hasumi). ---------------------------------------- Feature #22399: Iterator Methods for String Bit Operations https://bugs.ruby-lang.org/issues/22399 * Author: hasumikin (hitoshi hasumi) * Status: Open ---------------------------------------- PR URL: https://github.com/ruby/ruby/pull/XXXXX [TBD] ### What this adds The original proposal (#22082) listed this iterator group. Matz asked for a minimal subset in #22082#note-13, and the proposal deferred the group in #22082#note-14. The methods build on the single-bit methods of #22118 and on the region arguments of #22279: ``` String#each_bit(lsb_first: true) {|bit| ... } -> self String#each_bit(offset, length, lsb_first: true) {|bit| ... } -> self String#each_bit(range, lsb_first: true) {|bit| ... } -> self String#each_bit(lsb_first: true) -> Enumerator String#each_bit(offset, length, lsb_first: true) -> Enumerator String#each_bit(range, lsb_first: true) -> Enumerator String#bits(lsb_first: true) -> Array String#bits(offset, length, lsb_first: true) -> Array String#bits(range, lsb_first: true) -> Array String#bits(lsb_first: true) {|bit| ... } -> self String#bits(offset, length, lsb_first: true) {|bit| ... } -> self String#bits(range, lsb_first: true) {|bit| ... } -> self String#each_bit_offset(bit, lsb_first: true) {|offset| ... } -> self String#each_bit_offset(bit, offset, length, lsb_first: true) {|offset| ... } -> self String#each_bit_offset(bit, range, lsb_first: true) {|offset| ... } -> self String#each_bit_offset(bit, lsb_first: true) -> Enumerator String#each_bit_offset(bit, offset, length, lsb_first: true) -> Enumerator String#each_bit_offset(bit, range, lsb_first: true) -> Enumerator String#bit_offsets(bit, lsb_first: true) -> Array String#bit_offsets(bit, offset, length, lsb_first: true) -> Array String#bit_offsets(bit, range, lsb_first: true) -> Array String#bit_offsets(bit, lsb_first: true) {|offset| ... } -> self String#bit_offsets(bit, offset, length, lsb_first: true) {|offset| ... } -> self String#bit_offsets(bit, range, lsb_first: true) {|offset| ... } -> self ``` `each_bit` and `bits` visit every bit of the region. They give each bit as `0` or `1`. `each_bit_offset` and `bit_offsets` visit the offsets of the bits that are equal to `bit`. The `bit` argument is `0`, `1`, `true` or `false`. The offsets come in increasing order. ```ruby data = "\xAA\xCC".b data.bits(8, 4) # => [0, 0, 1, 1] data.bit_offsets(1) # => [1, 3, 5, 7, 10, 11, 14, 15] data.bit_offsets(0, 8..) # => [8, 9, 12, 13] data.bit_offsets(1, lsb_first: false) # => [0, 2, 4, 6, 8, 9, 12, 13] data.each_bit_offset(1).size # => 8 ``` The pairing follows `each_byte` / `bytes`. The `each_*` form returns `self` with a block. Without a block, it returns an Enumerator. The Array form returns an Array. Like `bytes`, the Array form also accepts a block. With a block, it behaves like the `each_*` form. ### Use cases #### Columnar data (LSB-first) Apache Arrow stores validity bitmaps and boolean columns as LSB-first packed bitmaps. Arrow pads a bitmap buffer to a multiple of 64 bytes. The padding bits have no defined value, so a read must stop at the column length. The region forms express this limit directly. The typical read patterns are: - **Null positions:** `bit_offsets(0, 0, length)` gives the indexes of the null elements of a column. A Ruby loop over `bit_set?` is not necessary. Without the region, the result includes the padding bits as nulls. ```ruby validity = "\xFD\x03".b # 10 elements, element 1 is null, bits 10..15 are padding validity.bit_offsets(0) # => [1, 10, 11, 12, 13, 14, 15] validity.bit_offsets(0, 0, 10) # => [1] ``` - **Boolean columns:** `bits(0, length)` converts a packed column into an Array of `0` and `1` values. - **Slices:** The region forms scan only the chunk of a bitmap that a batch covers. The offsets stay absolute. Thus they work with `bit_set?` and the mutation methods without adjustment. #### Microcontrollers and wire formats The `each_*` forms walk a buffer and do not allocate an Array. The region forms (with `offset..` or `(offset, length)`) continue from any bit offset. They do not unpack the whole buffer. On CRuby this matters for large buffers. `bits` on a 1 MiB buffer builds an Array of 8 million elements. `each_bit` with a block allocates nothing in the method itself. The same property helps implementations for microcontrollers (mruby, PicoRuby). These implementations take the core API as their specification. On such devices, a String is often the only buffer a program has. - **Protocol headers:** RFC protocol diagrams number bits MSB-first. `bits(offset, length, lsb_first: false)` returns a header field in wire order. A decoder can follow the diagram directly and does not reverse bits. ```ruby hdr = "\x12\x34\x81\x80".b # DNS header: ID, then the 16 flag bits flags = hdr.bits(16, 16, lsb_first: false) qr, rd, ra = flags[0], flags[7], flags[8] # => 1, 1, 1 ``` - **Huffman and other prefix codes:** A decoder reads a code stream one bit at a time and descends a tree. `each_bit` with a region reads the packed codes in the order the encoder wrote them. The decoder allocates nothing per bit. Bit-serial formats differ in packing order. JPEG entropy-coded segments and CCITT fax pack MSB-first, so they need `lsb_first: false`. DEFLATE packs LSB-first and reads with the default. ```ruby tree = ["a", ["b", ["c", "d"]]] # 0 => a, 10 => b, 110 => c, 111 => d stream = "\x5B\x80".b # 0 10 110 111 = "abcd", 9 bits node = tree out = +"" stream.each_bit(0, 9, lsb_first: false) do |bit| node = node[bit] if node.is_a?(String) out << node node = tree end end out # => "abcd" ``` - **Bit-banged buses:** A shift-register chain receives a multi-byte frame MSB-first. `each_bit(lsb_first: false)` over the frame gives one GPIO write per bit. The program does not split the frame into Integers first. - **1 bpp frame buffers:** A display such as the SSD1306 stores a page as one byte per column. Bit 0 is the top pixel. With the bitwise methods of #22118, `bit_offsets(1)` turns a pixel test into a coordinate list. `bitwise_and` of a sprite and the background keeps only the overlapping pixels. `bit_offsets(1)` yields their positions. `i / 8` is the column and `i % 8` is the row within the page. ```ruby sprite = "\x00\x3C\x3C\x00".b background = "\x00\x00\x30\xFF".b hits = sprite.bitwise_and(background) hits.bit_offsets(1).map { |i| [i / 8, i % 8] } # => [[2, 4], [2, 5]] ``` #### Why not String#unpack? `unpack1("b*")` and `unpack1("B*")` already give the bits as a `"0"`/`"1"` String. That form has four limits: - It allocates a String eight times the size of the receiver. - It moves only by whole bytes with `@`. - It cannot list the offsets of the set or cleared bits. - It returns characters, where `bit_get` returns Integers. The iterators read in place. They take bit regions. They list offsets directly. Their results compose with the rest of the bit API. ### API Contracts 1. **Region arguments** are the same as for `bit_count`: no argument (the whole string), `(offset, length)`, or a Range, plus `lsb_first:`. A lone `offset` raises `ArgumentError`, as for `bit_count`. 2. **Reads clamp.** The methods cut a region that extends past the end of the string down to the bits that exist. A region that begins at or beyond the end yields nothing. An empty or inverted region yields nothing. `bit_count`, the other read-only region method, has the same behavior. 3. **Offsets are absolute.** `each_bit_offset` and `bit_offsets` yield offsets relative to the start of the string, in the numbering that `lsb_first:` selects. Thus `data.bit_set?(i, lsb_first: x)` is true for every `i` that `data.each_bit_offset(1, lsb_first: x)` yields. 4. **The `bit` argument** accepts `0`, `1`, `true` or `false`. The method converts any other object with `to_int`, as for an index, with the same exceptions. An Integer other than `0` or `1` raises `ArgumentError`. An object without `to_int` raises `TypeError`. 5. **Errors are raised without a block.** The method validates the arguments before it creates the Enumerator. Thus `"".each_bit(-1, 4)` raises `IndexError` immediately, not on the first `each`. The Enumerator validates the arguments again when it iterates or computes its size. Thus `to_int` conversions can run more than once. 6. **Enumerator#size** is the number of bits in the region for `each_bit`. For `each_bit_offset`, it is the number of matching bits. The Enumerator computes the size on demand from the current contents of the receiver. 7. **The block can modify the receiver.** The iteration reads the string again for every bit. It stops at the current end of the string. A longer receiver does not extend the iteration beyond the original region. 8. **Out-of-range arguments** follow #22279. A negative offset raises `IndexError`. A negative length raises `ArgumentError`. A position or length beyond `2**64 - 1` raises `ArgumentError`. 9. **Frozen receivers** work, because all four methods only read. 10. **Encoding** has no effect. The methods treat the receiver as a byte sequence, as in #22118. 11. **`lsb_first` changes only the bit numbering within each byte.** The byte order does not change, as in #22118 and #22279. The value must be `true` or `false`. Any other value raises `ArgumentError`, as in #22118. ### Performance `each_bit_offset` and `bit_offsets` mask each byte against the region. They skip the bytes with no matching bit. Thus they scan a sparse bitmap byte by byte, not bit by bit. The measurements use an AMD Ryzen 5 5600X, a 1 MiB buffer, and ruby 4.1.0dev with YJIT off. YJIT changes the loop times by less than 15%, because the C method calls dominate the loops. Each alternative produces the same result as the new method. Thus its time includes the conversion of the `unpack1("b*")` bit String to Integers or offsets. The sparse buffer has 1,000 set bits. The dense buffer is `"\xAA" * 1 MiB`. | Operation | `bit_get` / `bit_set?` loop | `unpack1("b*")` based | New method | |--------------------------------------------|--------|----------|----------------------------| | Offsets of the set bits, sparse buffer | 0.56 s | 0.0125 s | `bit_offsets(1)`: 0.0015 s | | Offsets of the set bits, dense buffer | 0.61 s | 0.29 s | `bit_offsets(1)`: 0.052 s | | All bits as an Integer Array, dense buffer | 0.63 s | 0.43 s | `bits`: 0.072 s | | All bits yielded to a block, dense buffer | 0.35 s | 0.51 s | `each_bit { }`: 0.27 s | The block forms gain little speed over a `bit_get` loop, because the block call dominates. Their benefit is that the method itself allocates nothing. -- https://bugs.ruby-lang.org/

2 1

[ruby-dev:52243] [Ruby Feature#22279] Region (Length / Range) Arguments for String Bit Operations
by hasumikin (hitoshi hasumi) 10 Sep '26

10 Sep '26

Issue #22279 has been reported by hasumikin (hitoshi hasumi). ---------------------------------------- Feature #22279: Region (Length / Range) Arguments for String Bit Operations https://bugs.ruby-lang.org/issues/22279 * Author: hasumikin (hitoshi hasumi) * Status: Open ---------------------------------------- PR URL: ## What this adds Region overloads for the methods introduced in #22118, so they act on a contiguous run of bits in a single call: ``` String#bit_set(offset, length, lsb_first: true) -> self String#bit_set(range, lsb_first: true) -> self String#bit_clear(offset, length, lsb_first: true) -> self String#bit_clear(range, lsb_first: true) -> self String#bit_flip(offset, length, lsb_first: true) -> self String#bit_flip(range, lsb_first: true) -> self String#bit_count(offset, length, lsb_first: true) -> Integer String#bit_count(range, lsb_first: true) -> Integer ``` Existing single-offset forms and the no-argument `bit_count` are unchanged. ## Use cases Apache Arrow stores validity masks and boolean columns as LSB-first packed bitmaps (the same layout as Ruby's default `lsb_first: true`) and touches them a range at a time. **Writing a buffer:** mark a contiguous run of elements valid or null in one call (the equivalent of Arrow C++'s `bit_util::SetBitsTo`), or invert a boolean column: ```ruby validity.bit_set(start, run) # mark [start, start+run) valid validity.bit_clear(start, run) # mark them null column.bit_flip(0...n_elements) # boolean NOT ``` **Reading a buffer:** `null_count` is `length - bit_count`, a rank query over the mask. An Arrow array is an O(1) slice with an arbitrary bit offset, so a slice's `null_count` counts a range that starts and ends mid-byte: ```ruby null_count = slice_length - validity.bit_count(slice_offset, slice_length) ``` ## API Contracts - `(offset, length)` and a Range are equivalent; passing both is an `ArgumentError`. The positional form follows `String#[](start, length)`. - Endless / beginless ranges work as usual (`0..` = to the last bit). An empty region (`length == 0`, or an empty Range) is a no-op for the mutations and `0` for `bit_count`. For the mutations, even an empty region must begin within `0..bit_size`: `"\x00".bit_set(9, 0)` and `"\x00".bit_set(9...9)` both raise `IndexError`, mirroring how `"abc"[4, 0]` is `nil` while `"abc"[3, 0]` is `""`. - Mutations require the whole region to be in range and raise `IndexError` on overrun without modifying any bits; `bit_count` clamps to the bits that exist: the same read/write split as #22118. - `bit_count` returns `0` (not `nil`) for a region that lies entirely beyond the end. It is an aggregate, not a slice or a single-position lookup: the "reads return nil" rule of #22118 exists because `bit_get` has no in-domain value for "no such bit", whereas the count of set bits in an empty intersection is legitimately `0`, and the result is meant to feed arithmetic (`length - bit_count(...)`). This also keeps one rule for clamping: whatever part of the region exists is counted, and an empty part counts as `0`. - Mutations raise `FrozenError` on a frozen receiver even for an empty region, following `[].freeze.push` and `"".freeze.concat`. An out-of-range region is still detected first (`IndexError`), as in `String#setbyte`. - Negative offsets and Range endpoints raise `IndexError`, with no count-from-end normalization, as in #22118. A negative `length` raises `ArgumentError`, following the decision in #22881. - An inverted Range (`5..2`) is treated as empty, like `"abcd"[2..0]` returning `""`. - Bit positions beyond `2**64 - 1` raise `ArgumentError` even for the clamping `bit_count` (the representable-range rule of #22118 applies before clamping). - A lone `offset` keeps its #22118 single-bit meaning, so it is valid only for the mutations (`bit_set(offset)` etc.). `bit_count` has no single-bit form: it takes no argument (whole string), `(offset, length)`, or a Range, and `bit_count(offset)` raises `ArgumentError`. A one-bit count would return the same 0/1 as `bit_get(offset)`, and reading it as "from `offset` to the end" would contradict `String#[](index)`; counting to the end is spelled `bit_count(offset..)`. - An explicit `nil` is an argument, not an omission: `bit_set(0, nil)` and `bit_count(0, nil)` raise `TypeError`, as `"abc"[0, nil]` does. - `lsb_first` only changes the bit-to-position mapping within a byte. ## Performance A region call lets the implementation use `memset` / byte-wide `popcount` instead of a Ruby-level per-bit loop. On an AMD Ryzen 5 5600X (x86_64-linux, gcc, `--zjit`), with `buf = "\x00".b * 2**20` (1 MiB), setting 8,388,600 bits at an unaligned offset: ```ruby buf.bit_set(4, 8388600) # 15 us 8388600.times { buf.bit_set(4 + it) } # 440 ms (~29,000x slower) ``` Most of that gap is the per-call overhead of 8 million Ruby method calls rather than the per-byte work; the point is that a region call removes the loop entirely, not that the single-bit form is slow. For `bit_count` the point is addressing rather than raw speed: with `data = Random.bytes(4096)`, a region such as `data.bit_count(12347, 1000)` cannot be expressed with `byteslice` at all, because it starts and ends mid-byte. Even for a byte-aligned region the region form avoids the intermediate String: `data.bit_count(12344, 1008)` 90 ns vs `data.byteslice(1543, 126).bit_count` 130 ns. -- https://bugs.ruby-lang.org/

2 1

[ruby-dev:52244] [Ruby Bug#22285] FIX2INT loses consistency due to sizeof(int)
by YO4 (Yoshinao Muramatsu) 01 Sep '26

01 Sep '26

Issue #22285 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Bug #22285: FIX2INT loses consistency due to sizeof(int) https://bugs.ruby-lang.org/issues/22285 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- In include/ruby/internal/arithmetic/int.h, FIX2INT is resolved to rb_fix2int or RB_FIX2LONG. ``` #define FIX2INT RB_FIX2INT /**< @old{RB_FIX2INT} */ /** * Converts a Fixnum into C's `int`. * * @param[in] x Some Fixnum. * @pre Must not pass anything other than a Fixnum. * @return The passed value converted into C's `int`. */ static inline int RB_FIX2INT(VALUE x) { /* “FIX2INT raises a `TypeError` if passed `nil`,” says rubyspec. Not sure if * that is the intended behavior, but just preserving backward compatibility. */ #if 0 RBIMPL_ASSERT_OR_ASSUME(RB_FIXNUM_P(x)); #endif long ret; if /* constexpr */ (sizeof(int) < sizeof(long)) { ret = rb_fix2int(x); } else { ret = RB_FIX2LONG(x); } return RBIMPL_CAST((int)ret); } ``` RB_FIX2LONG takes a fast path using shift operations, whereas rb_fix2int performs a more complex process by calling rb_num2long based on the result of the FIXNUM_P check. This goes beyond the scope of rdoc and doc/extensions. Is it intended behavior for FIX2INT to call to_int? Since the FIXNUM_P guard has become an idiom, this seems like unnecessary behavior. Note that there is a possibility that an implementer could unintentionally create code that breaks under Windows and probably 32-bit platforms, (see Bug #22284) -- https://bugs.ruby-lang.org/

1 0

[ruby-dev:52235] [Ruby Bug#22123] Ruby::Box + `BUNDLER_SETUP` can evaluate gemspecs before main-box RubyGems initialization
by se4weed.dev＠gmail.com (Yuto NORINAGA) 18 Aug '26

18 Aug '26

Issue #22123 has been reported by se4weed.dev(a)gmail.com (Yuto NORINAGA). ---------------------------------------- Bug #22123: Ruby::Box + `BUNDLER_SETUP` can evaluate gemspecs before main-box RubyGems initialization https://bugs.ruby-lang.org/issues/22123 * Author: se4weed.dev(a)gmail.com (Yuto NORINAGA) * Status: Open * ruby -v: ruby 4.1.0dev (2026-06-13T03:25:52Z master 0e3b8918b3) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- # Ruby::Box + `BUNDLER_SETUP` can evaluate gemspecs before main-box RubyGems initialization ## Subject Ruby::Box + BUNDLER_SETUP can evaluate gemspecs before main-box RubyGems initialization ## Description When Ruby is started with `RUBY_BOX=1` and Bundler's `BUNDLER_SETUP` environment variable is present, RubyGems can require `bundler/setup` while loading `gem_prelude` for the root box. That can make Bundler run code through `TOPLEVEL_BINDING` in the main box before the main box has finished loading RubyGems. In one common case, Bundler evaluates a path gem's `.gemspec`, and a normal gemspec using `Gem::Specification.new` then fails with: ```text uninitialized constant Gem::Specification ``` This happens during Ruby startup, before application code starts. ## Environment Observed with: ```text ruby 4.1.0dev (2026-06-13T03:25:52Z master 0e3b8918b3) +PRISM [arm64-darwin25] macOS 26.4.1 bundler 4.1.0.dev Ruby::Box enabled with RUBY_BOX=1 ``` ## Minimal Reproduction Create a tiny path gem: ```sh rm -rf /tmp/box-bundler-repro mkdir -p /tmp/box-bundler-repro/box_repro cd /tmp/box-bundler-repro ``` `Gemfile`: ```ruby source "https://rubygems.org" gem "box_repro", path: "./box_repro" ``` `box_repro/box_repro.gemspec`: ```ruby Gem::Specification.new do |spec| spec.name = "box_repro" spec.version = "0.1.0" spec.summary = "Ruby::Box BUNDLER_SETUP repro" spec.authors = ["repro"] spec.files = [] end ``` Choose the Ruby executable being tested: ```sh RUBY_UNDER_TEST="${RUBY_UNDER_TEST:-ruby}" ``` Generate the lockfile with the same Ruby/Bundler environment: ```sh "$RUBY_UNDER_TEST" -S bundle lock ``` Then use the same executable to find the bundled `bundler/setup` path: ```sh BUNDLER_SETUP_PATH="$("$RUBY_UNDER_TEST" -rrbconfig -e 'puts File.join(RbConfig::CONFIG["rubylibdir"], "bundler/setup")')" ``` Then run Ruby with `RUBY_BOX=1` and `BUNDLER_SETUP`: ```sh RUBY_BOX=1 \ BUNDLE_GEMFILE="$PWD/Gemfile" \ BUNDLER_SETUP="$BUNDLER_SETUP_PATH" \ "$RUBY_UNDER_TEST" -e 'puts :ok' ``` ## Actual Result ```text [!] There was an error while loading `box_repro.gemspec`: uninitialized constant Gem::Specification. Bundler cannot continue. # from /tmp/box-bundler-repro/box_repro/box_repro.gemspec:1 # ------------------------------------------- > Gem::Specification.new do |spec| # spec.name = "box_repro" # ------------------------------------------- ``` ## Expected Result ```text ok ``` ## Control Cases This succeeds without Ruby::Box: ```sh BUNDLE_GEMFILE="$PWD/Gemfile" \ BUNDLER_SETUP="$BUNDLER_SETUP_PATH" \ "$RUBY_UNDER_TEST" -e 'puts :ok' ``` This also succeeds with Ruby::Box if Bundler is loaded later through `-rbundler/setup` instead of RubyGems' `BUNDLER_SETUP` hook: ```sh RUBY_BOX=1 \ BUNDLE_GEMFILE="$PWD/Gemfile" \ "$RUBY_UNDER_TEST" -rbundler/setup -e 'puts :ok' ``` So this does not appear to be application-specific. The failure is triggered by the Bundler startup environment, especially `BUNDLER_SETUP`. ## Analysis Ruby currently loads `gem_prelude` for both the root and main boxes: ```c // builtin.c rb_load_gem_prelude((VALUE)rb_root_box()); rb_load_gem_prelude((VALUE)rb_main_box()); ``` `gem_prelude.rb` requires RubyGems: ```ruby require "rubygems" ``` At the end of RubyGems, Bundler may be loaded from `BUNDLER_SETUP`: ```ruby # lib/rubygems.rb require ENV["BUNDLER_SETUP"] if ENV["BUNDLER_SETUP"] && !defined?(Bundler) ``` This means the root box's RubyGems load can trigger `bundler/setup` before the main box's RubyGems state is ready. Bundler evaluates path gemspecs with `TOPLEVEL_BINDING`: ```ruby # lib/bundler.rb eval(contents, TOPLEVEL_BINDING.dup, path.expand_path.to_s) ``` To confirm which box evaluated the gemspec and which RubyGems constants were available, I temporarily added the following diagnostics at the beginning of the path gem's `.gemspec`, before `Gem::Specification.new`: ```ruby if defined?(Ruby::Box) warn "box=#{Ruby::Box.current.inspect}" warn "root=#{Ruby::Box.root.inspect}" warn "main=#{Ruby::Box.main.inspect}" else warn "box=nil" warn "root=nil" warn "main=nil" end warn "gem=#{!!defined?(Gem)}" warn "gem_spec=#{!!defined?(Gem::Specification)}" warn "gem_version=#{!!defined?(Gem::VERSION)}" ``` During the failure, that diagnostic output showed: ```text box=#<Ruby::Box:3,user,main> root=#<Ruby::Box:2,root> main=#<Ruby::Box:3,user,main> gem=true gem_spec=false gem_version=false ``` So the gemspec is evaluated in the main box, but before the main box's RubyGems state is ready. The diagnostics suggest that `Gem` is present, but RubyGems constants such as `Gem::Specification` and `Gem::VERSION` are not fully initialized in the main box at that point. ## Why this should not be fixed in gemspecs Normal gemspecs conventionally use: ```ruby Gem::Specification.new do |spec| # ... end ``` Adding `require "rubygems/specification"` to the gemspec does not seem like the right fix. In local testing it only moved the failure further into RubyGems initialization, with other missing pieces such as: ```text Gem::Deprecate Gem::Requirement Gem::VERSION Gem.platforms ``` The problem is that Bundler is evaluating the gemspec before RubyGems is ready in the current box. ## Possible Fix Direction The core issue seems to be that `BUNDLER_SETUP` is consumed from the root box's RubyGems prelude before the main box's RubyGems state is ready. One possible direction is to fix this in Ruby startup / Ruby::Box prelude sequencing. For example, `builtin.c` could prevent the root box `gem_prelude` from consuming `BUNDLER_SETUP`, while still allowing the main box `gem_prelude` to consume it normally. Another possible direction is to fix this on the Bundler/RubyGems side, so that the `BUNDLER_SETUP` hook does not run user/top-level code in the main box before the main box RubyGems state is ready. The invariant I think we want is: `BUNDLER_SETUP` should not be able to trigger main-box code execution before the main box's RubyGems initialization is complete. With a local prototype that temporarily hides `BUNDLER_SETUP` while loading the root box prelude in `builtin.c`, then restores it before loading the main box prelude, the minimal reproduction succeeds. ## Related PR A proposed fix with a regression test is available at: https://github.com/ruby/ruby/pull/17323 -- https://bugs.ruby-lang.org/

3 2

[ruby-dev:52236] [Ruby Bug#22140] Prism rejects a trailing comma in a pattern-matching loop condition before do, while parse.y accepts it
by Kai (kai matsudate) 21 Jul '26

21 Jul '26

Issue #22140 has been reported by Kai (kai matsudate). ---------------------------------------- Bug #22140: Prism rejects a trailing comma in a pattern-matching loop condition before do, while parse.y accepts it https://bugs.ruby-lang.org/issues/22140 * Author: Kai (kai matsudate) * Status: Open * ruby -v: ruby 4.1.0dev (2026-07-02T00:24:43Z master e91f477e21) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- Prism rejects the following code, while parse.y accepts it: Prism: ```sh $ ruby --parser=prism -ce 'while a in b, do end' ruby: -e:1: syntax error found (SyntaxError) > 1 | while a in b, do end | ^~ expected a pattern expression after `,` ``` parse.y: ```sh $ ruby --parser=parse.y -ce 'while a in b, do end' Syntax OK ``` The same appears with `until` and `for` loops. I have a draft patch here: https://github.com/ruby/prism/pull/4154 -- https://bugs.ruby-lang.org/

2 1

[ruby-dev:52239] [Ruby Feature#9755] Thread::Backtrace::Location#defined_class
by shugo (Shugo Maeda) 10 Jul '26

10 Jul '26

Issue #9755 has been updated by shugo (Shugo Maeda). Status changed from Assigned to Rejected Now that a `Location` has a method entry ([Feature #16495]), `defined_class` can be implemented without adding extra information. However, I'm closing this issue because I no longer remember the use case. ---------------------------------------- Feature #9755: Thread::Backtrace::Location#defined_class https://bugs.ruby-lang.org/issues/9755#change-118028 * Author: shugo (Shugo Maeda) * Status: Rejected * Assignee: ko1 (Koichi Sasada) ---------------------------------------- I'd like Thread::Backtrace::Location#defined_class. ```ruby class X def foo p caller_locations.first.defined_class #=> Y end end class Y def bar X.new.foo end end Y.new.bar ``` nobu created a patch: https://github.com/nobu/ruby/compare/backtrace-self%2Bclass But this patch has two problems: 1. The patch adds Thread::Backtrace::Location#self, but it's weird that a location has self. 2. Thread::Backtrace::Location#class conflicts with Kernel#class. So I proposed defined_class as the method name. -- https://bugs.ruby-lang.org/

1 0

[ruby-dev:52221] [Ruby Feature#22082] Introduce Bit Operations into String
by hasumikin (hitoshi hasumi) 10 Jul '26

10 Jul '26

Issue #22082 has been reported by hasumikin (hitoshi hasumi). ---------------------------------------- Feature #22082: Introduce Bit Operations into String https://bugs.ruby-lang.org/issues/22082 * Author: hasumikin (hitoshi hasumi) * Status: Open ---------------------------------------- ## Relevant tickets - Introduce #bit_count method on Integer --- #20163 ## Abstract Ruby's `String` is already a byte sequence, but it lacks high-level bit operations. As a result, packed binary data must be handled with manual byte arithmetic. For example, checking set-bit at offset `10` currently requires calculating the byte position and the bit's position within that byte: ```ruby data = "\xAA\xAA\xAA\xAA" # The "classic" way byte_offset = 10 / 8 byte = data.getbyte(byte_offset) bit_offset = 10 % 8 ((byte >> bit_offset) & 1) == 1 #=> false # The "concise" way using Integer#[] data.getbyte(10 / 8)[10 % 8] == 1 #=> false ``` The concise form is a single line, but it still leaks implementation details into every call site: the caller writes `10 / 8` and `10 % 8` by hand (a recurring source of off-by-one errors at byte boundaries), and the `Integer` result must be compared against `1` to be used as a boolean. With a bit-addressed API, `data.bit_at(10)` takes a bit position and returns `true`/`false` directly. The cost compounds for iteration, run-length scanning, or splicing, where each operation otherwise needs its own byte/bit arithmetic. I propose native bit-level APIs, treating the String class as a first-class bit sequence: ```ruby data = "\xAA\xAA\xAA\xAA" data.bit_at(10) #=> false ``` By providing a flat bit-addressing model that handles the underlying bit-to-byte mapping, we allow developers to focus on the logical layout of their data (e.g., an Apache Arrow bitmap or a pixel buffer), making the code more readable and less error-prone. This proposal presents a family of methods for bit-oriented use of `String`, organized into groups below. The immediate goal is agreement on the overall direction and feedback on which subset should be pursued first. Presenting the full menu matters because some design questions only become clear at that level: - bit numbering keyword (`lsb_first:`) and its consistent application across methods - naming symmetry such as `bits` / `each_bit` - behavior for out-of-range bit indices ## Implementation (Prototype) You can try an actual working implementation with `gem install string_bits`. The source code can be seen in https://github.com/hasumikin/string_bits The reason I want to include this feature in the Ruby core rather than a third-party gem is that I want this API to be common across Ruby implementations such as mruby and Spinel. ## Proposed Methods Full prototype and documentation: https://github.com/hasumikin/string_bits/blob/0.2.0/docs/ProposedMethods.md **Read** - `bit_at(bit_offset, lsb_first: true) -> true | false | nil` -- read a single bit - `bit_count -> Integer` -- count of set-bits (popcount) `bit_count(bit_offset, bit_length, lsb_first: true) -> Integer` `bit_count(bit_range, lsb_first: true) -> Integer` - `bit_run_count(bit, bit_offset, lsb_first: true) -> Integer | nil` -- length of the run of `bit` starting at bit_offset **Iterator** - `each_bit(start_offset=0, lsb_first: true) { |bool| ... } -> self` -- yield each bit as true/false `each_bit(start_offset=0, lsb_first: true) -> Enumerator` - `bits(start_offset=0, lsb_first: true) -> Array` -- Array form of `each_bit` `bits(start_offset=0, lsb_first: true) { |bool| ... } -> self` - `each_bit_run(start_offset=0, lsb_first: true) { |bool, offset, len| } -> self` -- yield `(bool, offset, run_length)` pairs `each_bit_run(start_offset=0, lsb_first: true) -> Enumerator` - `bit_runs(start_offset=0, lsb_first: true) -> Array` -- Array form of `each_bit_run` `bit_runs(start_offset=0, lsb_first: true) { |bool, len| } -> self` - `each_bit_offset(bit, start_offset=0, lsb_first: true) { |offset| ... } -> self` -- yield position of each bit equal to `bit` `each_bit_offset(bit, start_offset=0, lsb_first: true) -> Enumerator` - `bit_offsets(bit, start_offset=0, lsb_first: true) -> Array` -- Array form of `each_bit_offset` `bit_offsets(bit, start_offset=0, lsb_first: true) { |offset| ... } -> self` **Mutation** - `bit_set(bit_offset, bit_length=1, lsb_first: true) -> self` -- set one bit or a logical bit bit_range to 1 `bit_set(bit_range, lsb_first: true) -> self` - `bit_clear(bit_offset, bit_length=1, lsb_first: true) -> self` -- set one bit or a logical bit bit_range to 0 `bit_clear(bit_range, lsb_first: true) -> self` - `bit_flip(bit_offset, bit_length=1, lsb_first: true) -> self` -- toggle one bit or a logical bit bit_range `bit_flip(bit_range, lsb_first: true) -> self` - `bit_splice(bit_offset, bit_length, str, str_bit_offset=0, lsb_first: true) -> self` -- write a sub-sequence of bits in place (bit-granularity `bytesplice`) `bit_splice(bit_range, str, str_bit_offset=0, lsb_first: true) -> self` **Slice** - `bit_slice(bit_offset, bit_length, lsb_first: true) -> String | nil` -- extract a sub-sequence of bits (bit-granularity `byteslice`) `bit_slice(bit_range, lsb_first: true) -> String | nil` **Bitwise** - `bitwise_not -> String` / `bitwise_not! -> self` -- invert every bit - `bitwise_and(other) -> String` / `bitwise_and!(other) -> self` -- bitwise AND - `bitwise_or(other) -> String` / `bitwise_or!(other) -> self` -- bitwise OR - `bitwise_xor(other) -> String` / `bitwise_xor!(other) -> self` -- bitwise XOR ## Performance This is not only about convenience. In a prototype implementation (string_bits gem), bulk operations such as `bitwise_and`, `bitwise_or`, and `bit_count` are also substantially faster than Ruby-level loops over bytes (see the Benchmark link below). I do not think performance alone is the reason to add the feature, but it is a practical benefit. ## Notes Benchmarks, discussion, and prior art: - Proposed methods (with use cases): https://github.com/hasumikin/string_bits/blob/0.2.0/docs/ProposedMethods.md - Benchmark: https://github.com/hasumikin/string_bits/blob/0.2.0/docs/Benchmark.md - Discussion: https://github.com/hasumikin/string_bits/blob/0.2.0/docs/Discussion.md - Why extend `String` rather than introduce a new class? - Naming convention: symmetry with `bytes` / `each_byte` - Bit Position Numbering of the String bit API - Why `lsb_first: true` is the default? - Bit ordering across domains - Apache Arrow Compatibility - Error behavior for out-of-range bit indices - Prior art: https://github.com/hasumikin/string_bits/blob/0.2.0/docs/PriorArt.md -- https://bugs.ruby-lang.org/

6 13

[ruby-dev:52237] [Ruby Feature#17121] Remove ENV#index
by Earlopain (Earlopain _) 02 Jul '26

02 Jul '26

Issue #17121 has been updated by Earlopain (Earlopain _). Status changed from Open to Closed Removed in https://github.com/ruby/ruby/commit/12a121cc0f7f5b438388288383a7b8b9baf3e2fe ---------------------------------------- Feature #17121: Remove ENV#index https://bugs.ruby-lang.org/issues/17121#change-117886 * Author: S_H_ (Shun Hiraoka) * Status: Closed ---------------------------------------- `ENV#index` is deprecated since `ruby-1.9.0-0`. So, remove it. ```bash root@8785b339158a:/all-ruby# env ALL_RUBY_SINCE=ruby-1.8 ./all-ruby -e 'ENV.index("RUBY")' ruby-1.8.0 ... ruby-1.8.7-p374 ruby-1.9.0-0 -e:1: warning: ENV.index is deprecated; use ENV.key ... ruby-2.7.0-rc1 -e:1: warning: ENV.index is deprecated; use ENV.key ruby-2.7.0-rc2 -e:1: warning: ENV.index is deprecated; use ENV.key instead ... ruby-2.7.1 -e:1: warning: ENV.index is deprecated; use ENV.key instead ``` pull request: https://github.com/ruby/ruby/pull/3415 -- https://bugs.ruby-lang.org/

1 0

[ruby-dev:52234] [ANN] Dev.us.kg - A new developer forum for the Ruby community
by amine kheddache 16 Jun '26

16 Jun '26

Hello Ruby development community, I'm excited to announce the launch of **Dev.us.kg**, a new, free community forum for Ruby developers and contributors of all skill levels. 🔗 Forum: https://dev.us.kg **What you can do on the forum:** - Ask Ruby development questions and get help - Share your Ruby projects and get feedback - Discuss implementation details and best practices - Connect with other Ruby developers **Categories include:** - Ruby 💎 - Python 🐍 - PHP 🐘 - Rust 🦀 - C# (.NET) - SQL - Lua - HTML CSS JS - Share Tools The forum is completely free, has no tracking, and is open to everyone. It's a space to learn, share, and build together. Join us and help build a welcoming Ruby community: 👉 https://dev.us.kg Best regards, AmineKHD

1 0