Constantly getting error in production: "[PU02] could not complete HTTP request to instance: legacy hyper error"

oteiza-a · April 12, 2026, 3:34pm

I have a nodejs app running in 4 performance-cpu-2x@4096MB machines, 2 in LHR and 2 in FRA.

My toml file:

app = ***
primary_region = 'fra'

[build]

[env]
  PORT = '8081'
  SCOPES = "***"
  HOST = 'https://***.com'

[http_service]
  internal_port = 8081
  force_https = true
  auto_stop_machines = "off"
  auto_start_machines = true
  min_machines_running = 1
  processes = ['app']

[[vm]]
  size = 'performance-2x'
  cpu_kind = 'performance'
  memory_mb = 4096

I’m constantly seeing this error in production:

_stream_id: 0000000000000000197fabbc3511f00ff74d3169a9105e2c
code: PU02
error.message: could not complete HTTP request to instance: legacy hyper error: client error (SendRequest) (source: connection error)
event.provider: proxy
fly.region: gru
fly.zone: 89c7
http.request.id: 01KP0RBGAMXZE7SHDWVNHB012D-gru
http.request.method: GET
http.request.version: 1.1
http.response.status_code: 502
log.level: error
detected_level: err.

The problem seems to be related to the fly proxy, as the responses from my machines seem healthy:

Also I’m using uptime services to make sure that my app is not failing to respond from different regions, and there has not been any downtime, even though the error logs are still appearing.

Is there anything I can do to investigate the cause of these errors? Any insight is appreciated, thanks!

halfer · April 12, 2026, 4:33pm

A 502 suggests that the problem is within your own service. Have you checked your Grafana instance for logs?

oteiza-a · April 12, 2026, 4:44pm

Yes, as I showed in the post, there are no 502 in the grafana metrics, and also no unhandled error logs, this is from the last 2 days, all machines:

A single 500 response in the last 2 days:

The [PU02] logs appear alone in the grafana logs, there’s no error showing up before or after them.

halfer · April 12, 2026, 4:48pm

Ah, I meant checking the Grafana logs, not the graphs. There could be some informational or warning log lines there that explain the issue.

(I am not disagreeing with your thesis that the 5xx comes from the proxy, but 9 times out of 10 issues raised in this forum are from user error; I thus think the due diligence is worth it.)

oteiza-a · April 12, 2026, 4:52pm

No problem I can show you an example of the logs too, unfortunately as I mentioned the error appears without any warnings or related errors with it, here’s an example:

As you can see, the other logs around it happen minutes before and after the [PU02] appear.
The “Blocked” logs are no errors by the way, it’s a simple middleware I have to reject suspicious requests.

Cornfield4893 · April 14, 2026, 7:03am

Hello, we do have the same error in our logs in both of our instances. Both in FRA region as well

morse · April 14, 2026, 3:29pm

I am getting a similar intermittent error

[PU02] could not complete HTTP request to instance: legacy hyper error: client error (SendRequest) (source: error from user's Body stream)

oteiza-a · April 14, 2026, 3:42pm

Hi @morse, what region/s is your app running on? Just checking in case it’s FRA too.

morse · April 14, 2026, 5:01pm

Region is IAD for me. The https://fly.io/ website itself is very unresponsive for me too, taking seconds to load and sometimes failing with timeouts. This is an issue with Fly itself and not other websites. I am in Italy.

flyio-debug return

flyio-debug: {"n":"edge-cf-fra2-8893","nr":"fra","ra":"49.236.0.100","rf":"Verbatim","sr":"ams","sdc":"ams1","sid":"0800129b6ee1e8","st":0,"nrtt":0,"bn":null,"mhn":"worker-dp-ams1-270a","mrtt":7}

somt0xhi · April 14, 2026, 5:53pm

Hello, are you still experiencing this?

Please can you curl again and post the headers + fly-request-id too?

olenzma · April 14, 2026, 6:16pm

Also experiencing intermittent failures to establish connection through proxy to my healthy and idle backend running in FRA - caller request times out w/o ever reaching the machine. Let me know if any diagnostics/logs can help

somt0xhi · April 14, 2026, 6:23pm

Yes please can you try

curl -v -H "flyio-debug: doit" debug.fly.dev (or your apps url)

olenzma · April 14, 2026, 6:45pm

- Removed, realize I measured the wrong thing , re-running diagnostic -

oteiza-a · April 14, 2026, 6:54pm

Hi @somt0xhi, OP here, I’m still getting the error (literally 5 minutes ago).

When I ran curl -v -H "flyio-debug: doit" [my-app].fly.dev i got:

* Host ***.fly.dev:80 was resolved.
* IPv6: 2a09:8280:1::49:e828:0
* IPv4: 66.241.125.53
*   Trying [2a09:8280:1::49:e828:0]:80...
* Connected to ***.fly.dev (2a09:8280:1::49:e828:0) port 80
> GET / HTTP/1.1
> Host: ***.fly.dev
> User-Agent: curl/8.7.1
> Accept: */*
> flyio-debug: doit
> 
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< location: https://***.fly.dev/
< server: Fly/b35aee356 (2026-04-14)
< via: 1.1 fly.io
< fly-request-id: 01KP6N86NFH4P8W46DY80SFEN2-gru
< flyio-debug: {"n":"edge-nac-gru1-9568","nr":"gru","ra":"2800:150:132:458:4878:8bf4:99ce:3ebf","rf":"Verbatim","sr":null,"sdc":null,"sid":null,"st":null,"nrtt":null,"bn":null,"mhn":null,"mrtt":null}
< content-length: 0
< date: Tue, 14 Apr 2026 18:49:52 GMT
< 
* Connection #0 to host ***.fly.dev left intact

Should I test destroying the the FRA machines and keeping a new primary region?

somt0xhi · April 14, 2026, 8:33pm

I don’t see any hyper errors for your app in the last one hour . It looks like requests came through an edge in bom that was temporarily unable to connect to other servers but that has resolved :.

morse · April 14, 2026, 8:45pm

Here is mine (but this request was fine. this error is intermittent)

* Host kimaki-gateway-production.fly.dev:443 was resolved.
* IPv6: (none)
* IPv4: 66.241.124.115
*   Trying 66.241.124.115:443...
* Connected to kimaki-gateway-production.fly.dev (66.241.124.115) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [338 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [2032 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.fly.dev
*  start date: Feb 21 23:24:17 2026 GMT
*  expire date: May 22 23:24:16 2026 GMT
*  subjectAltName: host "kimaki-gateway-production.fly.dev" matched cert's "*.fly.dev"
*  issuer: C=US; O=Let's Encrypt; CN=E8
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://kimaki-gateway-production.fly.dev/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: kimaki-gateway-production.fly.dev]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [flyio-debug: doit]
> GET / HTTP/2
> Host: kimaki-gateway-production.fly.dev
> User-Agent: curl/8.7.1
> Accept: */*
> flyio-debug: doit
>
* Request completely sent off
< HTTP/2 200
< content-type: text/html; charset=utf-8
< content-length: 954
< date: Tue, 14 Apr 2026 20:46:50 GMT
< server: Fly/b35aee356 (2026-04-14)
< via: 2 fly.io, 2 fly.io
< fly-request-id: 01KP6VYCAR8MERRX9A4NDRDMR9-fra
< flyio-debug: {"n":"edge-cf-fra2-2430","nr":"fra","ra":"49.236.0.100","rf":"Verbatim","sr":"iad","sdc":"ash1","sid":"0805091bd1e578","st":0,"nrtt":0,"bn":"worker-lsh-ash1-881c","mhn":"edge-cf-iad2-bf81","mrtt":88}
<
{ [954 bytes data]
* Connection #0 to host kimaki-gateway-production.fly.dev left intact

The app with the issue is an app with always on websocket connections (DIscord bot). The failing requests are intermittent REST HTTP requests. All fail with

 could not complete HTTP request to instance: legacy hyper error: client error (SendRequest) (source: error from user's Body stream)

oteiza-a · April 14, 2026, 9:31pm

Hi @somt0xhi, here are the last logs from today:

Here are the details of the last error log:


_stream_id: 0000000000000000197fabbc3511f00ff74d3169a9105e2c
code: PU02
error.message: could not complete HTTP request to instance: legacy hyper error: client error (SendRequest) (source: connection error)
event.provider: proxy
fly.region: bom
fly.zone: ce17
http.request.id: 01KP6N7T5K17NVA8TCQJKXSDET-bom
http.request.method: GET
http.request.version: 1.1
http.response.status_code: 502
log.level: error
url.full: /proxy_route/store-config
detected_level: err.*

I’m currently in (GMT-4) timezone right now, so 14:50 should be 18:50 in UTC time.

somt0xhi · April 15, 2026, 3:16pm

looks like you have a bunch of users in APAC region, and routing those requests to the EU can sometimes fail. The error message can be improved and we’d look into this.

Can you try deploying some apps in sin or bom that’s closer and see if this helps?

oteiza-a · April 16, 2026, 1:05pm

Hi @somt0xhi, currently my database is located in the FRA region, so using machines in distant regions could affect my app’s performance. Is there any alternative?

Also, what’s the cause behind the issues routing from APAC region to EU? Is this particularly a fly proxy issue, or can it still be related to my app?

Thanks!

AdityaMalani · April 17, 2026, 7:46am

Hi,

I am facing the same issue. And it seems like this issue has been on going for quite some time. Attaching screenshot of error from last 24 hours. Like @oteiza-a mentioned, there are no other error messages or warning along with the main error log. And there is nothing in the graphs as well.

Can you please help me too? This is likely the root cause of API call failures we have been experiencing and I would like to debug and fix it asap.