I’m running a Postgres instance on fly.io and want to ensure it isn’t exposed to the public internet. My goal is to set up a sort of internal VPC so that only trusted connections can access the database. Specifically, I want to allow a Cloudflare Worker to connect via Cloudflare’s tunnel (using their Hyperdrive connection) into my fly.io setup.
Here’s my plan:
Run an application on fly.io that acts as a tunnel (ingress) into my internal VPC.
Have the Cloudflare tunnel route requests to this tunnel application, which then forwards traffic to the internal Postgres instance.
The issue:
While I can connect to the Postgres database using its IPv4 address or from an application within the same fly.io cluster/VPC (even via SSH), connections coming in through the Cloudflare tunnel are failing—specifically, I’m having trouble with the TLS handshake required for secure Postgres connections.
Has anyone dealt with a similar TLS issue when routing Postgres connections through a Cloudflare tunnel? Any tips on configuring the internal network, adjusting machine settings, or any other practices to ensure the TLS connection works correctly would be hugely appreciated.
It might help if you could say more about your Postgres instance, why TLS is a hard requirement in this context, and the full details of the connection strings have been tried so far: both the cases that succeeded and the cases that failed.
(* out any passwords, of course—and feel free to do the same for any names that you consider sensitive—but show the full structure.)
It’s unusual to use TLS to talk to Postgres within the 6PN, so it may not be the tunnel itself that’s at fault…
if I setup a public IP on the DB, and connect via a xxxx.fly.dev url, i am able to connect with hyperdrive (no tunnel, same credentials).
And SSH-ing into another different machine in the same internal network, and runnning a psql into the db with the internal URL works (psql postgres://user:pass@database.internal/db-name )
So we know credentials work
However, if I setup the tunnel, pointing into database.internal:5432 it does not work.
Able to connect to it from hyperdrive (using postgres://user:pass@my-staging-database.fly.dev/database)
Able to connect to it with psql (using postgres://user:pass@my-staging-database.fly.dev/database)
Able to connect to it from a different machine (ssh-ing into it, running psql postgres://user:pass@my-staging-database.internal/database)
So then I create the tunnel:
This toml file:
# fly.toml app configuration file generated for cloudflared-tunnel on 2025-02-15T22:37:10-08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'cloudflared-tunnel'
primary_region = 'sjc'
[build]
dockerfile = 'Dockerfile'
[[vm]]
memory = '1gb'
cpu_kind = 'shared'
cpus = 1
[[services]]
internal_port = 5432
protocol = "tcp"
[[services.ports]]
port = 443 # or 5432 if you prefer
handlers = ["pg_tls"]
Using this docker file:
FROM cloudflare/cloudflared:latest
# Run the hello-world tunnel command when the container starts.
CMD ["tunnel", "--no-autoupdate", "run", "--token", "MY_TUNNEL_TOKEN"]
If I setup another application (e.g. fly’s demo app) and tunnel to it, the tunnel works without issues.
Thanks for all the details… The .fly.dev address has the Fly Proxy handling TLS, whereas this isn’t true on .internal.
I think the default for psql is to attempt TLS but then silently fall back to plaintext if it’s not available, so to better emulate Hyperdrive’s (apparently) harsher requirements, you would test with the following variant instead:
$ psql 'postgres://user:pass@my-staging-database.internal/database?sslmode=require'
psql: error: server does not support SSL, but SSL was required
That’s the bad news that I just saw on a freshly fly pg created database, anyway.
