Issue #19697 has been updated by byroot (Jean Boussier). While I think Ruby should offer a way to handle unicode domains, I'm not sure punycode should be handled at the Resolve layer though. Either way, this looks more like a feature request to me. ---------------------------------------- Bug #19697: Resolv::DNS resolution for international domains fails with "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT" https://bugs.ruby-lang.org/issues/19697#change-103321 * Author: clairity (claire c) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- i try to resolve a registered international domain, and instead of receiving DNS records, get an `incompatible character encodings` error: ``` # irb 3.2.2 :001 > require 'resolv' 3.2.2 :002 > res = Resolv::DNS.open { |dns| dns.getresource '우편.닷컴', Resolv::DNS::Resource::IN::MX } ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `[]=': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `sender' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:527:in `block in fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1125:in `block (3 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `block (2 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `block in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:521:in `fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:507:in `each_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:488:in `getresource' from (irb):3:in `block in <top (required)>' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:298:in `open' from (irb):3:in `<main>' from ~/.rvm/gems/ruby-3.2.2/gems/irb-1.6.4/exe/irb:9:in `<top (required)>' ... 4 levels... ``` i also tried this command in 3.3.0-preview1 and 2.1.4, receiving the same error. the expected response is to get DNS records for `우편.닷컴` (or any registered international domain). `dig 우편.닷컴` does indeed return a short but valid DNS record. -- https://bugs.ruby-lang.org/

Issue #19697 has been updated by chucke (Tiago Cardoso). I agree with byroot, this shouldn't be solved at the resolve layer. FWIW I've proposed the feature request in the stdlib uri library, which would.have worked here, but the core maintainers didn't agree this is a worthy feature to have. Since then, I've released [idnx](https://github.com/HoneyryderChuck/idnx), which uses libidn2 (or winnls) to encode IDNA2008 domains into punycode. I built it for [httpx](https://honeyryderchuck.gitlab.io/httpx/), the http library I'm the maintainer of. The confusion may also come from the fact that getaddrinfo supports IDNA domain name resolution in macOS. However, that's not a DNS feature. ---------------------------------------- Feature #19697: Resolv::DNS resolution for international domains fails with "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT" https://bugs.ruby-lang.org/issues/19697#change-103333 * Author: clairity (claire c) * Status: Open * Priority: Normal ---------------------------------------- i try to resolve a registered international domain, and instead of receiving DNS records, get an `incompatible character encodings` error: ``` # irb 3.2.2 :001 > require 'resolv' 3.2.2 :002 > res = Resolv::DNS.open { |dns| dns.getresource '우편.닷컴', Resolv::DNS::Resource::IN::MX } ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `[]=': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `sender' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:527:in `block in fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1125:in `block (3 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `block (2 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `block in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:521:in `fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:507:in `each_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:488:in `getresource' from (irb):3:in `block in <top (required)>' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:298:in `open' from (irb):3:in `<main>' from ~/.rvm/gems/ruby-3.2.2/gems/irb-1.6.4/exe/irb:9:in `<top (required)>' ... 4 levels... ``` i also tried this command in 3.3.0-preview1 and 2.1.4, receiving the same error. the expected response is to get DNS records for `우편.닷컴` (or any registered international domain). `dig 우편.닷컴` does indeed return a short but valid DNS record. -- https://bugs.ruby-lang.org/

Issue #19697 has been updated by chucke (Tiago Cardoso). byroot (Jean Boussier) wrote in #note-5:

...

FWIW I've touched on the potential feature request in the stdlib uri library which was not further discussed

That issue started as mentioning the removal of URI.escape, which probably prevented it from getting more attention.

I think there is potential for a feature request for IDN support in either URI or Resolv, but IMO a new clean feature request should be started, with a proposed API.

here: https://github.com/ruby/uri/issues/76 ---------------------------------------- Feature #19697: Resolv::DNS resolution for international domains fails with "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT" https://bugs.ruby-lang.org/issues/19697#change-103349 * Author: clairity (claire c) * Status: Open * Priority: Normal ---------------------------------------- i try to resolve a registered international domain, and instead of receiving DNS records, get an `incompatible character encodings` error: ``` # irb 3.2.2 :001 > require 'resolv' 3.2.2 :002 > res = Resolv::DNS.open { |dns| dns.getresource '우편.닷컴', Resolv::DNS::Resource::IN::MX } ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `[]=': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `sender' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:527:in `block in fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1125:in `block (3 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `block (2 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `block in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:521:in `fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:507:in `each_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:488:in `getresource' from (irb):3:in `block in <top (required)>' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:298:in `open' from (irb):3:in `<main>' from ~/.rvm/gems/ruby-3.2.2/gems/irb-1.6.4/exe/irb:9:in `<top (required)>' ... 4 levels... ``` i also tried this command in 3.3.0-preview1 and 2.1.4, receiving the same error. the expected response is to get DNS records for `우편.닷컴` (or any registered international domain). `dig 우편.닷컴` does indeed return a short but valid DNS record. -- https://bugs.ruby-lang.org/