Custom Private Networks (6PN) isolation with Managed Postgres - is bridging the only option?

[bradford1]

We’re trying to set up proper environment isolation using custom private networks. The goal is to ensure that if a dev/staging app is compromised, it cannot access production services or databases.

Our setup:

• Multiple apps (dev, staging, prod)
• Managed Postgres (MPG) for databases
• Want prod app + prod database isolated from other environments

The problem:

• MPG clusters appear to live on the org’s default private network
• Apps on custom 6PNs (fly apps create prod-app --network prod-network) cannot reach MPG
• The only workaround mentioned in docs/forums is Flycast bridging

Our concern with bridging:
Doesn’t bridging networks defeat the isolation purpose? If we bridge prod-network to the default network so prod can reach MPG, then any compromised app on the default network could potentially reach prod services through that bridge.

Questions:

1. Is there a way to create MPG clusters on a specific custom 6PN?
2. If not, what’s the recommended pattern for environment isolation with MPG?
3. Is the alternative to deploy Postgres as a generic app on our isolated network instead of using MPG?

Any guidance appreciated!

related: Custom Private Networks with a database

[mayailurus]

@bradford1

If we bridge prod-network to the default network so prod can reach MPG, then any compromised app on the default network could potentially reach prod services through that bridge.

Hi… Flycast is actually one-directional, since it’s a reverse proxy. (It’s the same Fly Proxy that allows the public Internet to reach carefully selected services within your various internal networks.)

$ fly ips allocate-v6 --private --network prod -a pg-db

The above gives everything within the custom private network (prod) access to the registered service ports of the single app (pg-db) in the default network, and there’s no ability granted to open TCP connections in the other direction.

You’re right overall that it’s still less isolated, since a rogue app in the default network could corrupt the Postgres instance (which generally isn’t Fort Knox) and from there use existing connections to get at least a partial toehold ability to talk to prod—much more communication than ideal isolation would allow. (A maliciously populated database is usually already damage enough, of course.)

Also, it’s not clear to me that the Flycast trick directly works with MPG, anyway, since there isn’t an app to name in that case. You might need some kind of repeater app in the default network, along the lines of what people were recommending for old-style egress IPs.

(I don’t have an MPG cluster myself.)

Since Fly MPG comes with Support, you can ask there and get an official answer.

(Let us know in the forum what you hear!)

@bradford1

Is the alternative to deploy Postgres as a generic app on our isolated network instead of using MPG?

Yep, you can install the PG Flex image within the custom private network, but you would lose the fly pg commands, I think. (As I mentioned in the other thread, those are slated to disappear anyway, last I heard.)

For most people, I wouldn’t recommend that as production database, however. It would only be for ones that seemed already inclined to do a lot of tinkering, be their own expert database administrators, etc.

It might be worth looking into whether multiple organizations, with linked billing, would be a tolerable compromise, , …

Hope this helps a little!