Weird log messages

1

Hi!
We are getting constant fly error messages in the logs:

2022-04-03T09:14:06Z proxy[c8ea5c60] fra [error]Error 1: Undocumented
2022-04-03T09:14:17Z proxy[a295f09f] ams [error]Error 1: Undocumented
2022-04-03T09:14:21Z proxy[20ed011c] sea [warn]Error 2004: App connection idle
2022-04-03T09:14:30Z proxy[c8ea5c60] ams [error]Error 1: Undocumented
2022-04-03T09:14:34Z proxy[a295f09f] fra [error]Error 1: Undocumented
2022-04-03T09:14:40Z proxy[20ed011c] nrt [error]Error 1: Undocumented

Can anyone shed a light on this?

2

That might happen if your app is in a restart loop. I don’t think these instances lived for very long.

3

@jerome no, it’s definitly not the restarts.
it happens all the time:

2022-04-04T17:08:21Z proxy[073fec6a] lga [error]Error 1: Undocumented
2022-04-04T17:08:32Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:32Z proxy[25610780] sin [warn]Error 2004: App connection idle
2022-04-04T17:08:36Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:39Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:38Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:40Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:39Z proxy[073fec6a] fra [error]Error 1: Undocumented
2022-04-04T17:08:42Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:46Z proxy[25610780] nrt [warn]Error 2004: App connection idle
2022-04-04T17:08:47Z proxy[25610780] nrt [warn]Error 2004: App connection idle

while flyctl status shows all 3 instances running and healthy for 10 hours

4

App connection idle means the app didn’t send any data over the connection for ~60s. Undocumented is caused by similar problems, we just haven’t written the helpful error messages for that yet.

What does your fly.toml look like? Is this an HTTP service?

5

@kurt it’s a standard HTTP service.
here’s the config:

❯ cat fly.toml
# fly.toml file generated for proserver on 2022-03-30T13:36:34+03:00

app = "proserver"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[metrics]
port = 5001 # default for most prometheus clients
path = "/metrics" # default for most prometheus clients

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  internal_port = 5000
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 2500
    soft_limit = 1000
    type = "connections"

  [[services.ports]]
    force_https = false
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

   [[services.http_checks]]
     interval = 10000
     grace_period = "5s"
     method = "get"
     path = "/health"
     protocol = "http"
     timeout = 2000
     tls_skip_verify = false
6

Thanks, we’re still investigating this, but it looks like the undocumented errors and the app connection idle errors are one in the same.

It appears to happen on POST /user-create. Something in your app isn’t responding fast enough.

What kind of app is it? Something is preventing bodies and connections from closing.