When I run fly launch or fly deploy, and I have some build steps on my Dockerfile. Where are these build steps run? In my understanding, it depends on the --remote-only and --local-only flags.
--remote-only runs on a fly thing with access to .flycast and .internal domain names.
--local-only runs on wherever the command is being called. In example, github actions and my PC, and these build don’t have access to .internal and .flycast domain names.
Is this correct?
I need to know this but I can’t find it clearly on your docs anywhere.
My Dockerfile in case it’s relevant to the answer.
# Set Bun and Node version
ARG BUN_VERSION=1.1.13
ARG NODE_VERSION=20.12.2
FROM imbios/bun-node:${BUN_VERSION}-${NODE_VERSION}-slim
# Set production environment
ENV NODE_ENV="production"
ARG DATABASE_URL
ENV DATABASE_URL=${DATABASE_URL}
# Bun app lives here
WORKDIR /usr/src/app
# Copy app files to app directory
COPY / .
# CD into the root directory
RUN cd ../../
# Install node modules
RUN bun install
# Build next js
RUN bunx turbo run --filter @diet-it/web build
# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD [ "bunx", "turbo", "run", "--filter", "@diet-it/web", "start" ]
The bunx turbo run --filter @diet-it/web build invocation triggers prisma migrate deploy. This command needs access to my postgres database.
@rubys is the best writer here, by a wide margin, and I was pleased to see him posting again.
(There used to be several drifting down each day. Like the orange leaves in the fall up here.)
Perhaps you glossed over his earlier response’s most important part, though, two days ago?
I.e., migrations are factored out of the main build process and execute apart, in their own special context.
Cantankerous individuals like myself go further and construct our own builders—in part to have complete control over the exact nitty details of what network access is and [usually more importantly] is not allowed. (I’m using Guix, but he gave the easier RCHAB example.)
That’s a project in its own right, though, so stick with the Fly.io platform feature that was designed specifically for performing migrations, unless there really is a compelling reason to do otherwise…
Maybe it’s my poor knowledge of English or my lack of interpretation skills, but I couldn’t understand the migrations part out of his comment. Not complaining, just thought he had focused on other aspects of my questions and though it would be a good idea to make an objective question.
Thank you for your answer.
First, I’ll start by answering the question you pose in the title: deploy release commands are run on ephemeral machines in your app’s private network, and with access to all of your secrets. And by ephemeral machines, I mean a virtual machine that is created for the sole purpose of running that one command and then destroyed immediately thereafter.
I’ve written and deployed Next.js apps and apps that use prisma, but never a Next.js app using prisma; what’s different (and new to me) here is turbo, which looks cool.
It looks like you define workflows in turbo, in a file called turbo.json. In your application, you have defined two workflows: build and start. Someplace in build you are triggering a prisma migration, I gather that is during a db:push task.
The smallest change you could make that likely get you up and running is to edit those workflows so that the db:push task is run as a part of the start workflow rather than the build workflow. If you are deploying to exactly one machine, that is fine. If you are deploying to multiple machines it is worth splitting that workflow out to a separate step, and running that as a deploy release command.
Awesome. I’d like to skip directly to the deploy_release strategy so I don’t have to revisit this topic in a close future.
So I tried following the referenced docs:
I removed the prisma scripts from my bunx turbo --filter @diet-it/web build pipeline and moved it to its own command. bunx turbo run --filter @diet-it/db db:build which now calls the db:build script:
"db:build": "bun run db:deploy && bun run db:generate",
"db:deploy": "prisma migrate deploy",
"db:generate": "prisma generate",
Added a release_command to my web fly.toml file:
[deploy]
release_command = "bunx turbo run --filter @diet-it/db db:build"
But now github returns a empty error that, if I look into the logs of the referenced machine, I can see the release_command failed because it didn’t have access to the DATABASE_URL environment variable again.
# https://fly-metrics.net/d/fly-logs/fly-logs?instance=6e82570ea76d08&orgId=799116&var-
@diet-it/db:db:build: error: script "db:build" exited with code 1
@diet-it/db:db:build: error: script "db:deploy" exited with code 1
@diet-it/db:db:build: error: "prisma" exited with code 1
@diet-it/db:db:build: Prisma CLI Version : 5.21.1
@diet-it/db:db:build:
@diet-it/db:db:build: [Context: getConfig]
@diet-it/db:db:build: Validation Error Count: 1
@diet-it/db:db:build:
@diet-it/db:db:build: |
@diet-it/db:db:build: 8 | url = env("DATABASE_URL")
@diet-it/db:db:build: 7 | provider = "postgresql"
@diet-it/db:db:build: --> prisma/schema/schema.prisma:8
@diet-it/db:db:build: error: Environment variable not found: DATABASE_URL.
@diet-it/db:db:build: Error code: P1012
@diet-it/db:db:build: Error: Prisma schema validation - (get-config wasm)
@diet-it/db:db:build:
@diet-it/db:db:build: Datasource "db": PostgreSQL database
@diet-it/db:db:build: Prisma schema loaded from prisma/schema
@diet-it/db:db:build: $ bun prisma migrate deploy
@diet-it/db:db:build: $ bun run db:deploy && bun run db:generate
@diet-it/db:db:build: cache bypass, force executing 07a484ce0b1dcd7a
• Remote caching disabled
• Running db:build in 1 packages
• Packages in scope: @diet-it/db
2024/10/28 14:16:54 INFO SSH listening listen_address=[fdaa:9:d488:a7b:342:a100:d647:2]:22 dns_server=[fdaa::3]:53
Machine created and started in 4.667s
INFO [fly api proxy] listening at /.fly/api
INFO Preparing to run: `/usr/local/bin/docker-entrypoint.sh bunx turbo run --filter @diet-it/db db:build` as root
2024-10-28T14:16:53.422744546 [01JB9QS4GKG9D0FXTWMGRJ2153:main] Running Firecracker v1.7.0
Configuring firecracker
Successfully prepared image registry.fly.io/diet-it-backend:deployment-01JB9QRN2KZYA6DYX607STBZG9 (2.646417274s)
Pulling container image registry.fly.io/diet-it-backend:deployment-01JB9QRN2KZYA6DYX607STBZG9
It seems like even on a release_command machine environment, I’m still having trouble to access my DATABASE_URL.
Sorry if I’m overextending this thread and asking too much of you all. Butt I really don’t know how to proceed.
What I’m seeing is that build is running deploy, and deploy is running migrate. Without seeing the full logs, my assumption is that the build is failing, which would make sense as it is running on your build machine which does not have access to secrets.
It’s a bit confusing because I named these command poorly.
Basically, the command bunx trubo run --filter @diet-it/db db:build calls both db:generate and db:deploy. Both these commands needs access to the DATABASE_URL. My fly.toml
[deploy]
release_command = "bunx turbo run --filter @diet-it/db db:build"
calls them both on the release_command.
So, I did just tried my code with your suggestion just to make sure, and the result is the same. I get an error for missing DATABASE_URL on the ephemeral machine fly spins up to run the release_command step.
The complete logs are here just in case you want to look them yourself: Github action logs Logs from the machine fly spins up just for the release_command step
The logs after making yours changes. (Running only prisma migrate deploy on the release_command and prisma generate together with the next build command):
I’m lost. I don’t know how to proceed. There’s either a bug on fly.io where ephemeral machines generated by the release_command are not setting the environment correctly (like on this issue), or your documentation is wrong here.
Or I’m doing something completely wrong and I can’t figure it out.
From what I see in your logs, you never got to the release step, it is the build step that failed. fly console is indeed the way to explore what an ephemeral machine looks like - it is created when you run that command, destroyed when you exit, and has access to your network and secrets.
Rendering static pages will need access to the database, so this step too needs to be moved to the release machine. Unfortunately this is a bit more painful as you will also need to capture the prerendered output and load it on each server. The easiest way to do that is with Tigris. If this is of interest, I can talk you through this.
(Technically, this too could be run on the deployed machine, but that kinda defeats the purpose of prerendering).
But look at the second message logs, isn’t this run after the build step is finished?
I removed all the static pages from this project, and it still fails. Also, on these logs, prisma migrate deploy does not get called until the release_command is triggered.
Even tho I named my command db:build not thinking on the implications. It was called by the release_command specification.
I will re run everything to be sure. But these logs were supposed to happen on the release_command.
By the way, while having fly deploy run as a github action is clearly a desirable place to end up, it might be easier/faster to debug using fly deploy locally. Once you get it working, you can move on to a github action.