How to connect to the GoogleCloud SQL through Wireguard VPN?

1

Hi,

While Fly is working on the PITR functionality, we would like to host our database on the GCP, and I consider that connecting it with our app through a VPN tunnel offers better security than firewall rules.

There is a guide on how to set up a WireGuard server on GCP, and I wonder what is the best option to connect an app to that user on the Fly side?

My biggest concern is that the app should be started after the tunnel is brought up, or it will crash otherwise.
So I assume that tunnel has to be permanent somehow.

2

Look into using Cloud SQL Auth proxy.

It creates the authenticated, encrypted connection to your SQL instance without having to do all the plumbing yourself.

You could run this is its own container/app or in a side-car way by adding the process to your app (probably better). Then use secrets to inject the service account credentials into the app so the proxy can use them to connect.

1 Like
3

Thanks! I quickly glanced through the documentation, which requires a companion server-side setup.
It means a necessity of setting up an HA cluster for it on top of the HA PostgreSQL cluster :sob:

4

Hmm, where did you see/read that?
From my experience there’s no such setup needed. I use the cloudsql proxy with a single postgres instance, smallest size, no HA config or server-side setup required.

5

The Cloud SQL Auth proxy uses a secure tunnel to communicate with its companion process running on the server. Each connection established through the Cloud SQL Auth proxy creates one connection to the Cloud SQL instance.

On the About the Cloud SQL Auth Proxy  |  Cloud SQL for MySQL  |  Google Cloud

6

Ah, I see. This is a bit misleading, all you need to run is the cloudsql proxy, the rest (proxy server, etc) GCP handles for you.

I’d recommend reading the rest of the doc, towards the bottom is a section Requirements for using the Cloud SQL Auth proxy, it has a pretty short list of what you need, it’s really straightforward to setup.

1 Like
7

Got it.

Are you using Public IP though? That’s one of the things that I would like to avoid :slight_smile:

8

You can do this, the most reliable setup is to run wireguard servers within GCP and then make each Fly VM a wireguard client.

However, this is not buying you much. You still need a public IP, it’s just something that Wireguard owns. Wireguard is a little more secure than doing TLS + public IP for Postgres, but that comes at a lot of extra complexity.

If this were me, I’d be inclined to just connect to Cloud SQL over TLS on a public IP. And then make sure Cloud SQL requires client certificate validation to accept a connection.

1 Like
9

brilliant, thank you!

I didn’t know about that routine before :slight_smile:

10

Any advices on storing certificates?

The app secrets might not offer enough space to store them :wink: :smiley:

11

My instance has a public IP but it’s locked down with the firewall rules to block all incoming connections and it’s not used (just an artifact of how I created the instance). Even when I connect from my local machine I use the cloudSQL proxy.

12

I learned that Cloud SQL Proxy adds your public ip to allowed list

13

Could you share where you found that information? Cause I can’t observe any such thing. When I start the proxy the connection is handled entirely by the proxy & GCP infra and the public IP of my instance is not opened up to any outside connections.

14

In the Connections tab of the SQL instance properties :slight_smile:

15

What I see is exactly the opposite:

image
image1096×470 41.3 KB

16

Is this with cloud SQL proxy running?

17

Yup. And an active connection to the DB.

18

Interesting. I wonder what’s the difference in configuration causing exactly opposite effects.

Btw, are you running it on parallel or on the separate VM?

19

@oliver1 I’ve tried a bunch of options to run Cloud SQL Auth Proxy alongside the main app and no cigar so far.

The gist of it would be something like this

RUN curl https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 > /usr/local/bin/cloud_sql_proxy
RUN chmod +x /usr/local/bin/cloud_sql_proxy
RUN /usr/local/bin/cloud_sql_proxy -instances=obfuscated-cloud-project-uri=tcp:127.0.0.1:5433 -credential_file=obfuscated-credentials-file.json  >>/var/log/cloudsql.log 2>&1  &

I also tried to move the last line and launching application into the entry point script but result was the same.

CMD ["/usr/local/bin/entrypoint.prd.sh"]

#!/bin/sh

cd /app
/usr/local/bin/cloud_sql_proxy -instances=obfuscated-cloud-project-uri=tcp:127.0.0.1:5433 -credential_file=obfuscated-credentials-file.json  >>/var/log/cloudsql.log 2>&1  &
/app/bin/server

If you don’t mind sharing details of your setup, I’ll truly appreciate that.

@kurt is it possible to run background process on fly.io?

Update: if I run the same command after the app is deployed by logging in into the instance with flyctl ssh console then it’s all fine :thinking_face:

20

Running /usr/local/bin/cloud_sql_proxy should go into the start script, not the Dockerfile.
Is the cloud proxy really running? Anything in the logs? Hard to say what’s wrong without more information.