Requests to Flycast IP fail with Connection reset by peer

yaniv · July 9, 2025, 6:41pm

Hello all.

Context: Investigating the react app client connectivity before connecting to Nginx.

App Setup:

Client App: x-realestate-client
- Image contains: EXPOSE 3000 and CMD ["serve", "-s", "dist", "--listen", "tcp://0.0.0.0:3000"]
Environment: App is configured for cold start

External Test Results

Machine Start:

fly machines start 48e355eb39d008 -a x-realestate-client

Public HTTPS Check:

curl -v https://x-realestate-client.fly.dev

Response: 200 OK with valid HTML payload
Proves TLS and public endpoint are functional

Direct IP Access:

Public IPv4 (66.241.124.27 ) fails with TLS handshake error
Public IPv6 (2a09:8280:1::6a:389f:0 ) fails to connect

Internal VM Checks (via SSH)

flyctl ssh console -a x-realestate-client

Tool Installation:

apt-get update && apt-get install -y iputils-ping dnsutils net-tools

Tools installed successfully (ping, dig, netstat)

IPv6 & DNS Diagnostics:

ping6 fdaa:a:ac95:0:1::3

Successful with 0% packet loss

dig x-realestate-client.internal AAAA

Resolved: fdaa:a:ac95:a7b:3bf:c169:9201:2

Flycast Address HTTP Check:

curl http://[fdaa:a:ac95:0:1::3]:3000

Response: Connection reset by peer

App Runtime Test:

curl http://localhost:3000

HTML page returned: Confirms app is running inside the VM

Listening Port Check:

netstat -tulnp | grep 3000

Output: 0.0.0.0:3000 is listening with node as the process — app is correctly bound to all interfaces

Conclusions

App is running and accessible via localhost:3000
Listening on 0.0.0.0:3000 — correct for internal and external binding
Flycast internal IPv6 address for the server is reachable (via ping6 )
Flycast DNS is resolving correctly
Access to internal IPv6 address from client fails (connection reset)
Public direct IP tests fail
Requests to Flycast IP (fdaa:a:ac95:0:1::3) still fail with Connection reset by peer

So I am understand that if all three machines the client the server and NGX configured on cold
start, then you have to use the flycast, So I have understood.

Similar Behavior I have on the Express server

Flycast Failure Indications with Server (Port 8000)

Flycast IPv6 Ping

ping6 fdaa:a:ac95:0:1::2

Successful: Server responds to ICMPv6 echo requests

DNS Resolution via Flycast

dig x-realestate-server.internal AAAA

Resolved: Flycast private IPv6 address is correctly registered

HTTP Request via Flycast IPv6

curl http://[fdaa:a:ac95:0:1::2]:8000

Failed: Connection reset by peer error indicates the server is not handling or is prematurely closing incoming requests on port 8000 via Flycast.

Port Binding Test (from within server)

netstat -tulnp | grep 8000

Confirms port 8000 is listening

Server Summary: Although Flycast network-level connectivity and DNS resolution to the server are functional, the application is rejecting HTTP connections via Flycast address, pointing to an application-level issue or misconfiguration on port 8000.

The Client fly.toml and the Dockerfile.
→ Link

mayailurus · July 9, 2025, 7:50pm

You actually want this to be :80 instead of :3000, due to the Fly Proxy sitting in the middle. (The [http_service] block in fly.toml implicitly tells it to listen on port 80.)

    curl  →  Fly Proxy  →  Machine

        /                 \
       /                   \

  port 80                port 3000

(In contrast, if it was a .internal address instead of Flycast, then you would need :3000—since it’d be going directly to the Machine.)

You generally wouldn’t expect HTTPS requests to work with numeric IP addresses, so this part isn’t unusual.

A plain HTTP request to the second address does work for me:

$ curl 'http://[2a09:8280:1::6a:389f:0]/' | fgrep title
  <title>React | TS | CI-CD | Redux | Node | MongoDB | Cy | Jest</title>

yaniv · July 10, 2025, 5:11pm

Thank very much for participating, if I have understood you correctly {internal: 3000, external: 80}

fly machines start 48e355eb39d008 -a x-realestate-client
48e355eb39d008 has been started
PS C:\> curl http://[fdaa:a:ac95:0:1::3]:80
curl: (7) Failed to connect to fdaa:a:ac95:0:1::3 port 80 after 1 ms: Could not connect to server

What do you reckon?

mayailurus · July 10, 2025, 5:29pm

Close! Do the curl from within fly ssh console -a x-realestate-client…

(Your Windows desktop/laptop doesn’t have access to the internal network, by default.)

yaniv · July 10, 2025, 6:15pm

I am sooooo… exited to see this:
thank so much!

fly ssh console -a x-realestate-client
Connecting to fdaa:a:ac95:a7b:3bf:c169:9201:2... complete
root@48e355eb39d008:/app# curl http://[fdaa:a:ac95:0:1::3]:80
<!doctype html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <link rel="shortcut icon" href="./favicon.ico" />

  <title>React | TS | CI-CD | Redux | Node | MongoDB | Cy | Jest</title>
  <script type="module" crossorigin src="./assets/index-B7nQF9MF.js"></script>
  <link rel="stylesheet" crossorigin href="./assets/index-MbHT80NU.css">
</head>

<body>
  <div id="root"></div>
  <div id="modal-root"></div>
</body>

But… i have tried the same with the express…

d891699f039568:/app#  curl http://[fdaa:a:ac95:0:1::2]:80
d891699f039568:/app#  curl http://[fdaa:a:ac95:0:1::2]:80
d891699f039568:/app#  curl http://[fdaa:a:ac95:0:1::2]:8000
curl: (56) Recv failure: Connection reset by peer
d891699f039568:/app#  curl http://[fdaa:a:ac95:0:1::2]:80/health
d891699f039568:/app#  curl http://[fdaa:a:ac95:0:1::2]:80/api/health
d891699f039568:/app#

What do you reckon?

this is the server fly.toml


app = 'x-realestate-server'
primary_region = 'ams'

    
[build]
  image = "xoxo10/container-server:latest"
  args = { MONGO_CONN = "${MONGO_CONN}", DB_NAME = "${DB_NAME}", JWT_SECRET = "${JWT_SECRET}", ORIGIN = "${ORIGIN}", PORT = "8000", SERVER_DOMAIN = "${SERVER_DOMAIN}", DOMAIN_EXTENSION = "${DOMAIN_EXTENSION}", FIREBASE_UNIVERSE_DOMAIN = "${FIREBASE_UNIVERSE_DOMAIN}", FIREBASE_STORAGE_BUCKET="${FIREBASE_STORAGE_BUCKET}", FIREBASE_CONFIG_BASE64="${FIREBASE_CONFIG_BASE64}"}

[deploy]
  strategy = "immediate"    # Default, no downtime
  wait_timeout = "10m"   # Extra time for slow cold starts
  [deploy.release_command_vm]
    size = "shared-cpu-1x"    
    memory = "256mb"

[http_service]
  internal_port = 8000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 0
  
  [http_service.concurrency]
    type = "requests"  # Better than "connections" for HTTP
    soft_limit = 25    
    hard_limit = 30

  [[http_service.checks]]
    grace_period = "10s"
    interval = "10s"
    method = "GET"
    path = "/api/health"
    timeout = "2s"
    [http_service.checks.headers]
      Content-Type = "application/json"

[[vm]]
  size = "shared-cpu-1x"
  memory = "256mb"
  cpus = 1
  cpu_kind = "shared"

mayailurus · July 10, 2025, 7:12pm

Nice!

You want this to be false, now, since Flycast doesn’t support SSL.

If you use curl -i http://[fdaa:a:ac95:0:1::2]:80, with the -i flag to explicitly show HTTP headers, then you’ll probably see it redirecting to https://. (The Fly Proxy doesn’t include the customary “Redirecting to…” HTTP body, which is why curl was outputting nothing—in the absence of -i.)

yaniv · July 11, 2025, 3:38pm

Sir absolutely splendid! thank you very much, I have enjoyed talking to you.

 curl -i http://[fdaa:a:ac95:0:1::2]:80
HTTP/1.1 200 OK
x-powered-by: Express
cross-origin-opener-policy: same-origin
cross-origin-resource-policy: same-site
origin-agent-cluster: ?1
referrer-policy: same-origin
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
x-dns-prefetch-control: off
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-xss-protection: 0
content-type: application/json; charset=utf-8
content-length: 22
etag: W/"16-20luvwVmABRcNMehXRFal/WNXbM"
vary: Accept-Encoding
date: Fri, 11 Jul 2025 15:36:01 GMT
connection: keep-alive
keep-alive: timeout=5
server: Fly/a608e03f9 (2025-07-10)
via: 1.1 fly.io
fly-request-id: 01JZX225CN8463PT9QG8GPH8HP-ams

{"msg":"Hello World!"}