Support for HTTP trailers?

[mccutchen]

Context

Based on some limited experimentation, it appears that Fly’s HTTP proxy/routing layer does not support HTTP trailers. Is that correct? If so, are there any plans to support HTTP trailers, or any tweaks I can make to fly.toml?

As always with the slightly more obscure corners of the HTTP spec(s), this may well be user and/or programmer error on my part, but I’m hoping someone can shed light on the behavior I’m seeing!

---

Experiment

Here’s the lightweight experiment I’m doing, where I’m seeing different (but equally “broken” from my POV) behavior when making a request via HTTP 1.1 vs HTTP 2:

Using HTTP 1.1

Example request:

curl -s -v --raw --no-buffer --http1.1 'https://httpbingo.org/trailers?trailer1=value1&trailer2=value2'

Observed behavior:

• The expected Trailer and Transfer-Encoding: chunked headers are included in the response

  < trailer: trailer1
  < trailer: trailer2
  < transfer-encoding: chunked
  

• … but the actual trailers are missing after the response body.

Using HTTP 2

Example request:

curl -s -v --raw --no-buffer --http2 'https://httpbingo.org/trailers?trailer1=value1&trailer2=value2' 

Observed behavior:

• None of the expected Trailer: or Transfer-Encoding: chunked headers are present in the response
• No trailer values appear after response body.

---

Extra details

In case it helps, httpbingo.org’s fly.toml file (and other source code) can be seen here in mccutchen/httpbingo.org on GitHub.

That app is a thin Fly-specific wrapper around mccutchen/go-httpbin, where the /trailers endpoint I’m using to test has a fairly minimal and hopefully straightforward implementation. (If there’s something obviously wrong with that implementation, I’d be eager to learn and fix it!)

[pavel]

Hey @mccutchen

Yeah, we don’t support http/1.1 trailers. They should work when making an http/2 request, though.

By default our proxy “downgrades” incoming http/2 requests to http/1.1 on their last hop (when making a request to machine).
If you enable h2_backed option the proxy will do an h2c request instead. Make sure your app supports h2c. In Go, you’ll need to use this package - h2c package - golang.org/x/net/http2/h2c - Go Packages

[mccutchen]

Ah, thank you for the explanation and the pointer to the h2c package. With that added and the h2_backend option enabled for the app, I am now seeing the expected trailers on HTTP/2 requests:

$ curl -s -v --raw --no-buffer --http2 'https://httpbingo.org/trailers?Trailer-1=Value-1&Trailer-2=Value-2'
> GET /trailers?Trailer-1=Value-1&Trailer-2=Value-2 HTTP/2
> Host: httpbingo.org
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 200 
< access-control-allow-credentials: true
< access-control-allow-origin: *
< content-type: application/json; charset=utf-8
< date: Thu, 19 Sep 2024 15:08:24 GMT
< server: Fly/bd16e0095 (2024-09-18)
< via: 2 fly.io
< fly-request-id: 01J85D7J4MC9S4V7BZ4CXJ9QFV-bos
< 
{
  [... json body elided ...]
}
< trailer-1: Value-1
< trailer-2: Value-2
* Connection #0 to host httpbingo.org left intact