error.message="while TCP-proxying data to/from app: failed to copy (direction=server->client, error=timed out)" 2022-07-24T10:39:46Z proxy[37cccb48] ams [error]

Logfiles are currently swamped with

error.message="while TCP-proxying data to/from app: failed to copy (direction=server->client, error=timed out)" 2022-07-24T10:39:46Z proxy[37cccb48] ams [error]

Application is working as expected.

2 Likes

Hi @andig,

Do all your instances look healthy if you run fly status?

If you’re running more than one instance of this app, are you seeing this message for all instances?

1 running and healthy

The best lead I have on this at the moment is our 60s idle timeout. If your app doesn’t close connections, we close them ourselves after 60s of inactivity. We may have changed how/whether we emit a message when that happens. It’s useful to see this when you’re having an issue; I can appreciate that it’s annoying when you’re not.

1 Like

As-is the message seems pretty useless as it doesn’t even tell which of the potential connections is has closed (could we add ports?). App is working time so that’s just noise. It’s also not quite clear if thats an actual data transfer (then the app should handle the error, not the proxy) or an idle timeout (then it is not a data transfer or copy).

It would be good to reduce the noise and make clearer what meaning is conveyed here plus debug details.

All TCP is data transfer. :smiley:

The error means that it timed out waiting for data to transfer.

We’re working on tweaking these to be more human compatible. It’s a pretty useful error when something is misbehaving, though. I don’t think we’ll remove it.

i have same issue. how to solve it? i use redis image docker

Error: error while TCP-proxying data to/from app: failed to copy (direction=server->client, error=Connection reset by peer (os error 104))

I am seeing this issue as well running haproxy. Concurrency is way below limits (according to metrics and logs).

Just suddenly happens and goes away on its own.

2023-10-14T10:53:55.043 proxy[9185e04a22d928] ams [error] could not proxy TCP data to/from instance: failed to copy (direction=client->server, error=Connection reset by peer (os error 104))

2023-10-14T11:07:37.692 proxy[17811ee6a67289] hkg [error] could not proxy TCP data to/from instance: failed to copy (direction=server->client, error=Broken pipe (os error 32)) 

these look like issues between global-lb and vm/machine/container? No errors from the app itself.
Some timeout that can be tweaked?

I am also seeing this in my Envoy container.

Is it because I have TCP service in my fly.toml, but Envoy is only listening to HTTP?

[[services]]
protocol = 'tcp'
internal_port = 1234

[[services.ports]]
port = 443
handlers = ['tls']