Cannot connect to postgres during deployment

Questions / Help Build debugging
1

I have a remix app, configured from one of the official “stacks”. It’s deployed through github actions.
When I push to dev/main, the app is deployed to staging/production, respectively.

Everything has been working fine until today… When I pushed to dev, the deploy step “fails”

Error release_command failed running on machine e148ed76db7689 with exit code 1.
Check its logs: here's the last 100 lines below, or run 'fly logs -i e148ed76db7689':
  Pulling container image registry.fly.io/[app]:deployment-01H0R1JWTM0X0X9JZEH5KNMTFE
  Successfully prepared image registry.fly.io/[app]:deployment-01H0R1JWTM0X0X9JZEH5KNMTFE (2.031856037s)
  Starting init (commit: 0ae2eef)...
  Preparing to run: `docker-entrypoint.sh bash ./scripts/migrate.sh` as root
  2023/05/18 18:23:10 listening on [fdaa:1:c7c6:a7b:161:b5ba:2546:2]:22 (DNS: [fdaa::3]:53)
  + fallocate -l 512M /swapfile
  + chmod 0600 /swapfile
  + mkswap /swapfile
  Setting up swapspace version 1, size = 512 MiB (536866816 bytes)
  no label, UUID=3badef6b-905d-4342-9dce-a5a6c1c6c94f
  + echo 10
  + swapon /swapfile
  + echo 1
  + npx prisma migrate deploy
  npm WARN exec The following package was not found and will be installed: prisma@4.14.1
  Prisma schema loaded from prisma/schema.prisma
  Datasource "db": PostgreSQL database "applecart_staging", schema "public" at "applecart-staging-db.flycast:5432"
  Error: P1001: Can't reach database server at `applecart-staging-db.flycast`:`5432`
  Please make sure your database server is running at `applecart-staging-db.flycast`:`5432`.

This is happening in a ./scripts/migrate.sh file, where I am calling npx prisma migrate deploy

Note: this started happening all of a sudden, with no changes to the code. I just redeployed staging because my “prod” environment was failing here first.

I have ssh’d into my machine and ran dig aaaa applecart-staging-db and everything looks correct.
I have also destroyed and recreated the postgres cluster from scratch, and reattached to it.
I have also searched through this forum for anything that might help, and I am at a loss.

Really not sure what to do here. Please help.

EDIT: I was able to resolve this by destroying the “free builder” machine. I guess something got messed up and it couldn’t talk to postgres when running the migration

2

For what its worth, this was a problem from our side. We had a regression earlier that prevented
communication over flycast.

3

Thank you for the response!

I might as well ask - when this was happening, I was looking at the Status page to see if any services were down. Is this not something that could be surfaced? Or did I miss it?

4

Unfortunately we caught it late due to the regression only affecting new hosts we were provisioning but was trivial to fix after we caught wind of it. We’re planning on having better checks specifically for Flycast so in future we can surface it as soon as it happens.

1 Like