Migrate AWS/Docker-Compose stack to Fly.io

[bwoodlt]

Hi All,

Hope you’re all well

In my other post here, I shared a docker-compose showing our current stacks and got great responses from folks on potential options.

We’ve now migrated our docker-compose setup to fly.io. Thanks to @ignoramous @greg and @tj1 for guidance.

The following changes are now in place:

• For the nodeapi instances, we’ve just deployed one instance to fly.io to test the water.
• We’ve connected the fly Postgres instance to the api instance running in a fly app.
• We’ve also migrated our data over to the new Postgres clusters.
• We are now using Upstash Redis instead of the self-managed Redis instance.

What we’re still unsure about:

• We need to deploy a web code (perhaps I just need to deploy from the root of the web project e.g /project/web-code)
• We are using nginx for the reverse proxy, I have no clue how to achieve similar on fly.io or if there are alternative approach? The goal is to deploy another 3 instances of the node api and use nginx reverse-proxy to route traffic to different instance.
• Once we achieve the above, now deploy the entire instances i.e 4 node_api apps, 1 web api into other 3 regions. Again, I’m not sure whether the Postgres cluster has to be the same for all these new regions or need to deploy new Postgres for different regions.
• We run a niche social app in which if we do a good job of providing a great UX, we should be able to scale to a few million users in 2 years, hence the need to deploy closer to the user to minimise latency as much as possible so that if a user is visiting from South Africa, we shouldn’t have to go all the way to Ohio to fetch their payload.
• On the Postgres side, we’re looking at introducing Pgbouncer to manage connections, still reading more about it to learn the benefit and how it reduces load on our databases.

I’d appreciate any pointers to help us further on our journey.

Gracias

[mhostetler]

For deployment, my go-to is using Github Actions. Fly provides a Github Action here: GitHub - superfly/flyctl-actions: :octocat: GitHub Action that wraps the flyctl

I follow a mono repo pattern and have unique Action workflows for each “app” in the monorepo - you can pass arguments to flyctl to have it operate in a subfolder - example:

name: "Deploy Server Prod"
on:
  workflow_dispatch:

env:
  FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

jobs:
  deploy:
    name: Deploy Server to Production
    runs-on: ubuntu-18.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Docker layer caching
        uses: satackey/action-docker-layer-caching@v0.0.11
        continue-on-error: true

      - name: Build app Docker image
        run: docker build ./${MY_PROJECT}_server -t fitclub

      - uses: superfly/flyctl-actions@1.1
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
          FLY_PROJECT_PATH: "./${MY_PROJECT}_server"
        with:
          args: "deploy -a ${MY_PROJECT}-prod --config ./fly.toml"

Reverse Proxy
In a previous life, I used the Nginx Reverse proxy trick quite a bit. On Fly, I no longer use it at all and simply rely on Fly networking. The one case where Fly networking is limited is if you’re trying to stitch together apps under sub-directories on a given domain. If this is the case, simply deploy an app with you Reverse Proxy config pointed at internal addresses.

Scaling Postgres
Depends a lot on your app, but it would probably be easiest to work with a single postgres-flex app and scale read replica’s to the other regions. If your app can be configured to write to a single region, point it back to the primary region for writes. If not, you can look into using the Fly-Replay header.

This is VERY app dependent (workload, etc) so feel free to clarify.

Good luck!

[bwoodlt]

Thanks for this @mhostetler I’ll take a look at Github Actions.

I’m still having issues accessing the deployed instance on the outside world. I can see our node app is deployed yet when I visit the provided deployed link, it just returned 502.

[ignoramous]

In case you’re confused / lost, try deploying sample apps to get a feel for how Fly works. May be that’d help? For example: Running Fly.io Apps On UDP and TCP · Fly Docs or Run a Node App · Fly Docs

If you haven’t already, check this troubleshooting guide: Troubleshooting your deployment · Fly Docs or Get Information about an App · Fly Docs

If you post specific errors from your app and create one topic each for every roadblock you hit, that’d help the community here help you better.

Alternatively, if you’re paying for support, then feel free to email the Fly team your queries.

[bwoodlt]

I don’t think its more deploying or launching an app (that can happen with a few commands), but been able to understand the end-to-end. Fly.io is definitely not for beginners in DevOps having said that, learning as much as… I’ve seen the referenced docs and went through a few repos on fly GitHub.
And yes, I can run some sample projects.Its more connecting the dots considering our use-case.

I’ve now reached out to the support team via their paid offering.

always@ @ignoramous