TCP Port Hangs

Questions / Help
1

I’ve just tried out fly for the first time and I’m running into an issue connecting with my app.

The app exposes an http server but needs to terminate TLS connections directly.

The Docker image works perfectly fine when trying it out locally, but when deployed to fly the connection just seems to hang forever.

And yes, the server is listening on 0.0.0.0:6128, not 127.0.0.1.

Curiously the problem is still present when ssh ing to the instance and trying to connect with curl directly: it still just hangs.

Even more curiously, netstat shows a high RecvQ value, which seems to show that packets are getting stuck in the receive buffer without getting read.

2022-05-13T03:13:14+02:00
2022-05-13T03:13:14+02:00697×71 4.16 KB

Which is very odd to me, since the server works just fine in local Docker, so it must be somehow related to the Firecracker environment.

The server seems to be up and running just fine:
2022-05-13T03:42:41+02:00

Here is the config, maybe I’m missing something?

# fly.toml file generated for fabric on 2022-05-13T00:22:08+02:00

app = "xxx"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[build]
  image = "xxx:v0.0.3"

[env]

[[services]]
  protocol = "tcp"
  internal_port = 6128

  [services.concurrency]
    hard_limit = 100000
    soft_limit = 100000
    type = "connections"

  [[services.ports]]
    port = "6128"

  [[services.tcp_checks]]
    grace_period = "5s"
    interval = "15s"
    restart_limit = 2
    timeout = "2s"
2

I’m having issues sshing as well. just hangs.

3

Perhaps a more general issue? I thought I was having issues with generating SSL cert, but keep getting SSL Connection Error - PR_CONNECT_RESET_ERROR.

This happens on both the fly.dev domain and my custom domain.

UPDATE: I’m still not sure what the issue was. Restarting the apps did not work, but I could ssh into them just fine and all the checks were still passing. I just redeployed them now and they are back up.

UPDATE 2: I edited secret. When the app restarted, it had the same problem. Another deploy didn’t fix it in this case. :confused:

4

Given the follow-up posts which sound more like region/host issues it may well not be this, but could it be worth trying to listen on IPv6? :thinking: Only when I handled TLS in-app (for NodeJS), I did it like this:

https.createServer(httpsOptions, app).listen(port, '::', () => console.log(`Started server`)

… and so listened on ‘::’.

Your app may be entirely different,and not sure how Fly’s proxy works, but that may explain the difference between running locally and when run behind Fly’s proxy. Or not. But can’t hurt to try.

5

I’m still getting a connection error.

Error error connecting to SSH server: connect tcp [fdaa:0:32f2:a7b:2c60:de02:f1:2]:22: operation timed out

Has anyone tried overriding their existing ssh creds? I don’t understand the documentation for override (what would the final command look like?):

fly ssh establish -h
Create a root SSH certificate for your organization. If <override>
is provided, will re-key an organization; all previously issued creds will be
invalidated.

Usage:
  flyctl ssh establish [<org>] [<override>] [flags]

Flags:
  -h, --help   help for establish

Global Flags:
  -t, --access-token string   Fly API Access Token
  -j, --json                  json output
      --verbose               verbose output