httpx 1.8.0 has been released. ``` HTTPX.get("https://gitlab.com/honeyryderchuck/httpx <https://gitlab.com/honeyryderchuck/httpx>") ``` HTTPX is an HTTP client library for the Ruby programming language. Among its features, it supports: * HTTP/2 and HTTP/1.x protocol versions * Concurrent requests by default * Simple and chainable API * Proxy Support (HTTP(S), CONNECT tunnel, Socks4/4a/5) * Simple Timeout System * Lightweight by default (require what you need) And also: * Compression (gzip, deflate, brotli) * Streaming Requests * Authentication (Basic Auth, Digest Auth, AWS Sigv4) * Expect 100-continue * Multipart Requests * Cookies * HTTP/2 Server Push * H2C Upgrade * Automatic follow redirects * International Domain Names * GRPC * Circuit breaker * WebDAV * SSRF Filter * Response caching * HTTP/2 bidirectional streaming * QUERY HTTP verb * Server Sent Events * Tracing * Request Hedging * Datadog integration * Faraday integration * Webmock integration * Sentry integration Here are the updates since the last release: # 1.9.0 ## Features ### `:hedging` plugin The `:hedging` plugin implements a strategy called "request hedging", whereby a request is sent two times, over different connections, and picking up whichever arrives first. ```ruby http = HTTP.plugin(:hedging) http.get("http://www.example.com") # sent two times, only the fastest one comes back... ``` You can read more about it in https://gitlab.com/os85/httpx/wikis/Hedging . ### `:proxy/http` plugin: new options `:proxy_headers` and `:enable_proxy_tunnel` The `:proxy/http` plugin (which gets automatically loaded by the `:proxy` plugin) gains two new options. #### `:proxy_headers` Can be passed a separate set of HTTP headers, which are going to be sent to the proxy when establishing the CONNECT proxy tunnel. ```ruby http = HTTPX.plugin(:proxy).with(proxy_headers: { "x-proxy-1" => "foo" }, headers: { "x-header-1" => "bar" }).get("https://example.com") # -> CONNECT example.com HTTP/1.1 # -> Host: example.com # -> X-Proxy-1: foo # ... # <- 200 OK # ... # -> GET / HTTP/1.1 # -> X-Headers-1: bar # ... ``` #### `:enable_proxy_tunnel` When set to `true`, it forces the establishment of a CONNECT proxy tunnel before sending the request. Bear in mind that this will be the norm for `https://` requests, so only use this in cases where the requests are to be sent to an `http://` URL and a CONNECT proxy tunnel is required. ## Improvements * `:request_timeout` will also observe the connect phase, in case a request is sent to an idle connection. This means that, if you're using `:request_timeout`, you no longer need to specify `:connect_timeout`. * The same will happen if you're using `:total_request_timeout`. * do not require "io/wait" anymore for ruby 3.2 or higher. * `:proxy/http` plugin: raise a `HTTPX::ConnectTimeoutError` if there was a timeout error while waiting for the CONNECT request to complete. ## Bugfixes * `:auth` plugin: fixed clause inverted check, which caused a new token to always be generated on retry. * `:auth` plugin: take auth_header_expires_at into account when deciding to regenerate on retry (token may have expired by the time the request will be retried). * `:tracing` plugin: fixed request initial time calculation in order to take into account the time to wait for a PING correctly. * `:tracing` plugin: fixed request initial time calculation for pending requests which were waiting for the connect phase and errored meanwhile. * `:proxy/http` plugin: `:headers` will not be sent anymore to the proxy when establishing the CONNECT proxy tunnel. * socket connect timeout bookkeeping is fixed, where in a multi request scenario, connect timeout errors shall not be raised too early if an error happens before the remainder of connections elapses. * `:proxy` plugin: allow proxy params to be frozen, which also fixes an issue where proxies identified as unhealthy by a previous request, would still be retried on for different hostnames. * `:callbacks` plugin: fix broken visibility of session `#close` method. * native resolver: retry dns queries on not-imp and refused errors (treats them as transient resolver errors). * `webmock` adapter: fix emission of `:response` callback so that this only happens after the body is fully buffered (which makes the `:callbacks` plugin `.on_response_completed` callback works the same way when and when not stubbed). * `:addresses`: fix support for the documented variant where a hostname/ips hash is passed. * `http2`: fix interests calculation, so that: * a connection expects to read from connections which still have in-transit streams. * an HTTP/2 with no available flow control window in its streams will wait for incoming WINDOW_UPDATE frames. * `http2`: pause buffering the request body if there's no available control window in the corresponding stream or connection. * `http2`: reset the ping timer too if a connection receives a GOAWAY frame while waiting for a PING to be ack'ed. * wait for read or write for any operation, considering that `OpenSSL::SSL::SSLSocket` objects may return both `.wait_readable` and `:wait_writable` for reads and writes. * when using `:max_requests`, also account for inflight requests when deciding to send it to the parser or the pending queue. * fixed `HTTPX::Options#merge` which was not freezing the resulting options object, something which happens when enabling plugins, which made resulting sessions unusable across Ractors. * fixed the `Alt-Svc` header parser to be able to parse values with multiple authorities. * fixed request params handling so that they are passed to all requests invoked (instead of only the first of them). ## Chore * Emit a warning message whenever the connection is deactivated with pending requests; the message contains an internal state which should help debug it. # 1.8.4 ## Improvements * send requests which were not sent, or were not processed by the server, when the server sends back a GOAWAY frame (which is deemed recoverable, like when there's no error), after forcing a reconnection (instead of bubbling up the GOAWAY error for all of them). ## Bugfixes * `:retries` plugin: also retry requests which failed due to a RST_STREAM frame with REFUSED_STREAM error code. * `:retries` plugin: resend requests which were piped to a closed connection internally. * `:proxy` plugin: fix issue related with reconnecting inactive connections shut down by the peer, when used with the `:persistent` plugin, where the CONNECT request wasn't being sent to establish the tunnel. * fixed compatibility issue starting with the `json` gem 3.0.0, when calling `HTTPX::Response#json`. * avoid bookkeeping issue in session, where the `on_response_arrived` on a request may be called more than once (which it can happen in the `:follow_redirects` plugin, where a request may have completed due to an error, while the request is waiting to be resent due to `retry-after`). # 1.8.3 ## Bugfixes * options print a warning instead of raising an error on unknown option * The approach introduced in the previous version, of erroring on something people have been accidentally relying on, is too strict in hindsight for a patch version. This was reverted, and instead, a deprecation warning will be printed, to give time for folks to migrate. * The format of option messages (exceptions and warnings) also slightly changes to wrap the option in quotes, for proper highlighting. # 1.8.2 ## Improvements * `:proxy` plugin: allow setting `:no_proxy` option to `"*"` (or `["*"]`) to allow bypassing proxy for all requests. * Extending sessions with unknown options will now raise an error, instead of silently ignoring them (consistent with the initialization case now): ```ruby session = HTTPX.with(unknown_option: 1) # will raise a HTTPX::Error: unknown option: unknown_option ``` * Use `HTTP2::Connection#closed?` internally (instead of `#state`) to check for connection state; this will, in a future release of `http-2`, seamlessly support the new `:closing` state. * Add missing `response` attribute reader to `HTTPX::RequestTimeoutError`. * Improve the correctness of calling `#dup` on requests and connections. ## Bugfixes * `:auth` plugin: generate token first, then set the "expires_at" timestamp (bearer tokens, specially if JWT, may carry information about when it expires). * `:auth` plugin: do not check for token freshness on a retry, instead force token regeneration. * `:retries` plugin: do not apply "retry after" delay after a ping timeout error (the ping timeout already implies a delay, so retry on a new connection should be immediate). * `:retries` plugin: do not retry on any HTTP/2 frame error (only on GOAWAY or PING frame errors). * `:persistent` plugin: do not decrement request remaining retries on ping or GOAWAY frame errors. * `:tracing` plugin: set connection initialization time to nil no termination (so that it doesn't get reused on persistent connections after reuse). * `:follow_redirects` plugin: fixed issue when a timeout error would be raised but only bubbles up to the root request, not properly closing the in-flight follow up request, and leaving the request handling phase in a broken state. * datadog adapter: the connection initialization time is now set only on connect or first request received, the reason being, making it more accurate. * webmock adapter: `.to_timeout` helper will make the request raise a `HTTPX::RequestTimeoutError` instead of a plain `TimeoutError` (otherwise, if using persistent sessions, it'd trigger a retry instead and mess with webmock request count bookkeeping). * native resolver: do not transition to `:open` if the socket was closed during resolution (which happens when an error is raised while the query bytes are buffered). * fix for when setting a new session callback on an ssl context object which has been frozen after the first connection attempt (done by binding the ssl context callback to an internal ivar which can be changed). * connection: protect against `Errno::ETIMEDOUT` errors on socket close by forcing transition to closed. * do not set a ping timer for HTTP/1.1 on keep-alive flow, as there's no ping and the connection is reestablished. # 1.8.1 ## Improvements * the session will now eagerly write initial data to connections instead of waiting for I/O the first time (saves a syscall). * removed all rescue blocks rescuing `StandardError` internally; this could interfere with usages of `httpx` inside `Timeout.timeout` blocks, which may emit an exception at any point, including in blocks where that exception could be rescued, leaving the session in a potentially irrecoverable state and looping. * debug logs change the display of ids from 'tid:123, fid:123' to 'tid=123 fid:123'. * and when using the `datadog` adapter, all related correlation ids (trace id, span id, etc) will also be added to the identifiers in the same format * plugins built on top of the `:retries` plugin will skip the `retry_after` interval when certain conditions are met. * `:persistent` plugin: request will be retried immediately, when the error happens after the keep alive interval expired. * `:auth` plugin: request will be retried immediately if it failed due to a "401 Unauthorized Error". * `:ssrf_filter` plugin: removed public/private references of exception message. ## Bugfixes * `:auth` plugin: handle edge case where `:auth_header_expires_at` returns no value. * `:digest_auth` plugin: do not retry if there was an issue building the authorization header after the probe. * `webmock` adapter: prevent mocked requests from opening a connection to the peer server that will not be used. * clear and cancel request timers on completion or reset, ensuring that they won't be retained way past the point where the request is gone. * connection: cancel and discard ping timer before closing (removes timer reference) * connection: fix logic where purging-while-resending requests to a coalesced connection was leaving references in the old connection. * moved an `on_body_chunk` callback to the `:callbacks` plugin, where it's used and expected to be defined. * `http2`: avoid closing the connection on internal errors which are supposed to be specific to a request, such as the max headers or max response body size checks. * system resolver will use an internal exception when using `Timeout.timeout` on resolutions to avoid conflicts with the default timeout exception class. * prevent edge case where `#alpn_protocol` was being called on a `TCPSocket`. * fixed v1.8.0 regression causing connection to loop when an error happens while connecting. * resolver cache: fixed cache eviction logic, where alias answers weren't being evicted, causing resolutions to stall after a certain number of resolutions.