Issue #22493 has been updated by matz (Yukihiro Matsumoto). I agree this is a bug. Case-insensitive matching should not depend on the direction, and `×` and `÷` are not a case pair. Making the two tables agree according to the Unicode case mapping is the right fix. I leave the review of the tables to @duerst and @ima1zumi, and the backport decision to the branch maintainers. Matz. ---------------------------------------- Bug #22493: Case-insensitive matching is inconsistent in some single-byte encodings https://bugs.ruby-lang.org/issues/22493#change-120142 * Author: mame (Yusuke Endoh) * Status: Open * Backport: 3.3: REQUIRED, 3.4: REQUIRED, 4.0: REQUIRED ---------------------------------------- Each single-byte encoding has two case tables: the lowercase table (used for matching and `String#downcase`) and the list of case pairs (used for the search optimization and character classes). They disagree in some encodings, so case-insensitive matching gives different results depending on the direction or the search method, and some characters that are not case pairs are matched with each other. ```ruby e = Encoding::Windows_1251 big = "\x80".force_encoding(e) # "Ђ" small = "\x90".force_encoding(e) # "ђ" Regexp.new(big, Regexp::IGNORECASE) =~ small #=> 0 Regexp.new(small, Regexp::IGNORECASE) =~ big #=> nil (expected: 0) big.start_with?(Regexp.new(small, Regexp::IGNORECASE)) #=> true e = Encoding::Windows_1252 Regexp.new("\xD7".force_encoding(e), Regexp::IGNORECASE) =~ "\xF7".force_encoding(e) #=> 0 (expected: nil; "×" and "÷") ``` @make_now_just found this and wrote the fix. The fix makes the two tables agree, following the Unicode case mapping of each character. All the changes are: * ISO-8859-16 * lowercase table: 0xA4 € no longer maps to 0xA5 „ * Windows-1250 * lowercase table: 0xD7 × no longer maps to 0xF7 ÷ * Windows-1251 * case pairs added: Ђ/ђ (0x80/0x90), Ѓ/ѓ (0x81/0x83), Љ/љ (0x8A/0x9A), Њ/њ (0x8C/0x9C), Ќ/ќ (0x8D/0x9D), Ћ/ћ (0x8E/0x9E), Џ/џ (0x8F/0x9F), Ў/ў (0xA1/0xA2), Ј/ј (0xA3/0xBC), Ґ/ґ (0xA5/0xB4), Є/є (0xAA/0xBA), Ї/ї (0xAF/0xBF), І/і (0xB2/0xB3), Ѕ/ѕ (0xBD/0xBE) (they were already in the lowercase table) * Windows-1252 * lowercase table: 0xAF ¯ no longer maps to 0xBF ¿, and 0xD7 × no longer maps to 0xF7 ÷ * Windows-1253 * lowercase table: 0xB6 ¶ no longer maps to 0xDC ά * case pairs: ¶/ά (0xB6/0xDC) replaced with Ά/ά (0xA2/0xDC) * Windows-1254 * case pairs added: Š/š (0x8A/0x9A), Œ/œ (0x8C/0x9C), Ÿ/ÿ (0x9F/0xFF) (they were already in the lowercase table) The wrong entries seem to come from copying the tables of another encoding: windows_1252.c was made from windows_1251.c (0xAF/0xBF and 0xD7/0xF7 are case pairs in Windows-1251), Windows-1253 from ISO-8859-7 (where 0xB6 is Ά), and Windows-1254 from ISO-8859-9 (which has no characters in 0x80-0x9F). This does not change the Greek final sigma ς and µ, or the Turkish İ, which also differ from Unicode case folding. PR: https://github.com/ruby/ruby/pull/19644 @duerst @ima1zumi Could you review these changes? -- https://bugs.ruby-lang.org/