Debugging 502s, confused, caused by deploy? 🤔

1

Hey, we’re trying to understand why we’re having 502s. We have metrics from Fly about it, and we think logs too, but we’re not sure why. Here’s an example:

CleanShot 2023-02-01 at 13.08.20@2x
CleanShot 2023-02-01 at 13.08.20@2x1920×446 95.2 KB

Questions:

  1. Why is the log output messed up with leading error.message that breaks the format? Is that normal?
  2. For timed out while connecting to instance what is the timeout limit? Does it retry? How many times?
  3. Should request.id be used for request tracing?
  4. I see logs with app/runner/proxy following by an ID (?) in brackets. Does the ID relate logs between proxy/runner/app?
  5. Is there a correspondence of one 502 response for every error see in these logs? Meaning here we’d expect 3 502s to be recorded by metrics (yet our metrics suggest there was only one 502 so I’m confused)
  6. While the app was booting around the 17:53:22-53 time range we got an error. Is that related or coincidence? If it’s related, how is a restart/deployment supposed to be made in a flawless way? Is it even possible?

Thanks in advance for helping answer these questions! :pray:

1 Like
2

Just saw this:

2023-02-01T18:22:56Z proxy[e89b8b12] sea [warn]Could not proxy HTTP request. Retrying in 20 ms (attempt 1)
2023-02-01T18:22:56Z proxy[e89b8b12] sea [warn]Could not proxy HTTP request. Retrying in 24 ms (attempt 1)
2023-02-01T18:22:56Z proxy[e89b8b12] sea [warn]Could not proxy HTTP request. Retrying in 24 ms (attempt 1)