Postgres connection issues


I’ve been having some issues with a golang app that connects to a postgres cluster.

I’m using the pgx db driver and using the native version pgxpool interface the error is the unhelpful unexpected EOF, however using the stdlib version the error is driver: bad connection (in an earlier version it appeared as write failed: write tcp broken pipe )

I’ve made a simple program that reproduces the error:

the fly logs output ends up looking like this

2022-03-10T07:57:05Z app[f470e071] syd [info]Starting init (commit: 0c50bff)...
2022-03-10T07:57:05Z app[f470e071] syd [info]Preparing to run: `/bin/app` as root
2022-03-10T07:57:05Z app[f470e071] syd [info]2022/03/10 07:57:05 listening on [fdaa:...]:22 (DNS: [fdaa::3]:53)
2022-03-10T08:14:16Z app[f470e071] syd [info]2022/03/10 08:14:16 Pinged DB successfully
2022-03-10T08:14:16Z app[f470e071] syd [info]2022/03/10 08:14:16 Pinged DB successfully
2022-03-10T08:27:07Z app[f470e071] syd [info]2022/03/10 08:27:07 Pinged DB successfully
2022-03-10T08:57:07Z app[f470e071] syd [info]2022/03/10 08:57:07 driver: bad connection
2022-03-10T09:23:08Z app[f470e071] syd [info]2022/03/10 09:23:08 Pinged DB successfully
2022-03-10T09:27:07Z app[f470e071] syd [info]2022/03/10 09:27:07 Pinged DB successfully
2022-03-10T09:57:08Z app[f470e071] syd [info]2022/03/10 09:57:08 driver: bad connection

Interestingly in my actual program the error occurs every hour like clockwork (a background task runs every 15 minutes), but the errors are more sporadic with just a simple database ping.

I found a bunch of older posts on here that have similar problems but with earlier versions of the postgres image, and it seems like a lot has changed.
for reference the result from fly image show

Deployment Status
  Registry   =                                                     
  Repository = flyio/postgres                                                           
  Tag        = 14.1                                                                     
  Version    = v0.0.16                                                                  
  Digest     = sha256:29c8a9a28c0e70f690ef2f10fec353908fff99e1dc0a048f5087174b561c7bb8  

This issue doesn’t occur when connecting to [database-ip]:5433 to avoid hitting the proxy.

I’m at a bit of a loss as to how to investigate this further, so I’d love any suggestions